Hi Olivier,
Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis).
[RK] Did not realize you thought of this as a bug since it seems very deliberate when I look at the code(even secondary vm's in hafnium have dtbs and are mapped so it sure looks like a feature as opposed to a bug. Why do you think this is a bug and unintentional?
E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, >> FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
[RK] We need a way to pass platform specific arguments to the partitions anyway and manifest is a nice way to do this. The FFA discovery interfaces are good for things common across all partitions but not so great for platform specific thing. Did you have some other mechanism in mind to pass information to the partitions? Section 5.4 of the FF-A spec has a protocol for passing data but that is all implementation specific anyway so portability is gone out the window unless we explicitly make sure OPTEE and hafnium agree on the IMPDEF aspects of it.
[OD] Sorry to say that but this looks very much like.a hack, right?
[RK] Yes 😊 no doubt about it. I was just pointing out that we have something for now to work around the issue 😊 For issue #1, the only possibility is to keep the DTB size < 4K. All the other problems you specify are also problems and we need to solve them. Again, what is the spec alternative to pass arguments in a completely standard way? 5.4 specifies the mechanism but the TLV's are totally IMPDEF, how do we allow for this while meeting the goal to run the same hafnium binary without resorting to platform ports like TF-A?
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 23, 2021 1:46 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a