Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP?
The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains? For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru...) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru...) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manifest-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhykd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP?
The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru...) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru...) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manifest-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhykd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation. For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi Olivier,
Glad to hear that there are no objections for the sptool update to solve the immediate problem.
For passing the SP manifest pointer to the SP we should ask for ideas from the community. [OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[VW] For the second issue, the FF-a v1.1 Beta0, section 5.4 has some guidance. AFAICT, we can use "FF-A boot protocol usage" field from the "Initialization descriptor" to find the register to use to pass the pointer to the boot information. From the earlier analogy, "Initialization descriptor" seems to be the "SP boot data". IIUC, this descriptor can hold the pointer to the SP manifest.
Thoughts?
-Varun
-----Original Message----- From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Tuesday, June 22, 2021 6:18 PM To: 'Olivier Deprez' Olivier.Deprez@arm.com; Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation. For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 22 June 2021 20:38 To: raghu.ncstate@icloud.com; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
Glad to hear that there are no objections for the sptool update to solve the immediate problem.
[OD] Ack. Tbc with OP-TEE folks if there is no concern for an SEL1 SPMC (when SPs are loaded by BL2).
For passing the SP manifest pointer to the SP we should ask for ideas from the community. [OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[VW] For the second issue, the FF-a v1.1 Beta0, section 5.4 has some guidance. AFAICT, we can use "FF-A boot protocol usage" field from the "Initialization descriptor" to find the register to use to pass the pointer to the boot information. From the earlier analogy, "Initialization descriptor" seems to be the "SP boot data". IIUC, this descriptor can hold the pointer to the SP manifest.
[OD] Ack. This remains to implement. This concept somehow exists since FF-A v1.0 but never got priority. Now it seems there is a use case.
Thoughts?
-Varun
-----Original Message----- From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Tuesday, June 22, 2021 6:18 PM To: 'Olivier Deprez' Olivier.Deprez@arm.com; Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation. For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi Olivier,
Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis).
[RK] Did not realize you thought of this as a bug since it seems very deliberate when I look at the code(even secondary vm's in hafnium have dtbs and are mapped so it sure looks like a feature as opposed to a bug. Why do you think this is a bug and unintentional?
E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, >> FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
[RK] We need a way to pass platform specific arguments to the partitions anyway and manifest is a nice way to do this. The FFA discovery interfaces are good for things common across all partitions but not so great for platform specific thing. Did you have some other mechanism in mind to pass information to the partitions? Section 5.4 of the FF-A spec has a protocol for passing data but that is all implementation specific anyway so portability is gone out the window unless we explicitly make sure OPTEE and hafnium agree on the IMPDEF aspects of it.
[OD] Sorry to say that but this looks very much like.a hack, right?
[RK] Yes 😊 no doubt about it. I was just pointing out that we have something for now to work around the issue 😊 For issue #1, the only possibility is to keep the DTB size < 4K. All the other problems you specify are also problems and we need to solve them. Again, what is the spec alternative to pass arguments in a completely standard way? 5.4 specifies the mechanism but the TLV's are totally IMPDEF, how do we allow for this while meeting the goal to run the same hafnium binary without resorting to platform ports like TF-A?
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 23, 2021 1:46 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi Raghu,
See few comments [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 23 June 2021 15:44 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis).
[RK] Did not realize you thought of this as a bug since it seems very deliberate when I look at the code(even secondary vm's in hafnium have dtbs and are mapped so it sure looks like a feature as opposed to a bug. Why do you think this is a bug and unintentional?
[OD] We designed this code path in the early days of turning Hafnium into a SPMC. Yes we left this possibility open for the SP to observe its manifest, but with in mind this would have to be revisited in favor of using the FF-A boot protocol interface.
E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, >> FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
[RK] We need a way to pass platform specific arguments to the partitions anyway and manifest is a nice way to do this. The FFA discovery interfaces are good for things common across all partitions but not so great for platform specific thing. Did you have some other mechanism in mind to pass information to the partitions? Section 5.4 of the FF-A spec has a protocol for passing data but that is all implementation specific anyway so portability is gone out the window unless we explicitly make sure OPTEE and hafnium agree on the IMPDEF aspects of it.
[OD] I had in mind that the FF-A manifest should contain only well known fields based on the FF-A standard. Other platform specific information had to pass through the boot protocol interface. I agree there is a gap in defining common Tags in the more generic TLV encoding defined by the spec.
[OD] Sorry to say that but this looks very much like.a hack, right?
[RK] Yes 😊 no doubt about it. I was just pointing out that we have something for now to work around the issue 😊 For issue #1, the only possibility is to keep the DTB size < 4K. All the other problems you specify are also problems and we need to solve them. Again, what is the spec alternative to pass arguments in a completely standard way? 5.4 specifies the mechanism but the TLV's are totally IMPDEF, how do we allow for this while meeting the goal to run the same hafnium binary without resorting to platform ports like TF-A?
[OD] Fair enough. A possibility is to agree on Tags and describe generically defined values in the documentation. The FF-A spec leaves this impdef, but nothing prevents to define them at project level.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 23, 2021 1:46 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Thanks Olivier. What you have below makes sense. Going back to the original issue in the email (which was about the 4K size limitation), are we suggesting that we leave the SP manifest as 4K in the sp package and revisit later? The second separate but related issue is that we wanted to use the manifest to pass large information to the SP (because of which we hit the first issue), but are we saying we will use the boot protocol to pass arguments/information, initially in a platform specific way, and then later hopefully in a more standard way? Once again, as we have discussed in a few other situations, will OPTEE adopt the same boot protocol? If not portability of the same FF-A partition (specifically S-EL0 partitions) to OPTEE may become an issue. In any case, I'd like to understand how we can pass somewhat large data(>4K) to a partition in a way that is platform agnostic(I don’t want a platform port like TF-A) and that is portable across OPTEE and Hafnium.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, July 27, 2021 4:57 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See few comments [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 23 June 2021 15:44 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis).
[RK] Did not realize you thought of this as a bug since it seems very deliberate when I look at the code(even secondary vm's in hafnium have dtbs and are mapped so it sure looks like a feature as opposed to a bug. Why do you think this is a bug and unintentional?
[OD] We designed this code path in the early days of turning Hafnium into a SPMC. Yes we left this possibility open for the SP to observe its manifest, but with in mind this would have to be revisited in favor of using the FF-A boot protocol interface.
E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, >> FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
[RK] We need a way to pass platform specific arguments to the partitions anyway and manifest is a nice way to do this. The FFA discovery interfaces are good for things common across all partitions but not so great for platform specific thing. Did you have some other mechanism in mind to pass information to the partitions? Section 5.4 of the FF-A spec has a protocol for passing data but that is all implementation specific anyway so portability is gone out the window unless we explicitly make sure OPTEE and hafnium agree on the IMPDEF aspects of it.
[OD] I had in mind that the FF-A manifest should contain only well known fields based on the FF-A standard. Other platform specific information had to pass through the boot protocol interface. I agree there is a gap in defining common Tags in the more generic TLV encoding defined by the spec.
[OD] Sorry to say that but this looks very much like.a hack, right?
[RK] Yes 😊 no doubt about it. I was just pointing out that we have something for now to work around the issue 😊 For issue #1, the only possibility is to keep the DTB size < 4K. All the other problems you specify are also problems and we need to solve them. Again, what is the spec alternative to pass arguments in a completely standard way? 5.4 specifies the mechanism but the TLV's are totally IMPDEF, how do we allow for this while meeting the goal to run the same hafnium binary without resorting to platform ports like TF-A?
[OD] Fair enough. A possibility is to agree on Tags and describe generically defined values in the documentation. The FF-A spec leaves this impdef, but nothing prevents to define them at project level.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 23, 2021 1:46 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Adding extra details to this: we can easily go over 4K with standard FFA entries in the Manifest, for example device regions. A platform may need to define hundreds of them, so 4K would be exceeded very quickly.
Nicolas
-----Original Message----- From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Wednesday, July 28, 2021 5:52 PM To: 'Olivier Deprez' Olivier.Deprez@arm.com; Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Thanks Olivier. What you have below makes sense. Going back to the original issue in the email (which was about the 4K size limitation), are we suggesting that we leave the SP manifest as 4K in the sp package and revisit later? The second separate but related issue is that we wanted to use the manifest to pass large information to the SP (because of which we hit the first issue), but are we saying we will use the boot protocol to pass arguments/information, initially in a platform specific way, and then later hopefully in a more standard way? Once again, as we have discussed in a few other situations, will OPTEE adopt the same boot protocol? If not portability of the same FF-A partition (specifically S-EL0 partitions) to OPTEE may become an issue. In any case, I'd like to understand how we can pass somewhat large data(>4K) to a partition in a way that is platform agnostic(I don’t want a platform port like TF-A) and that is portable across OPTEE and Hafnium.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, July 27, 2021 4:57 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See few comments [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 23 June 2021 15:44 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis).
[RK] Did not realize you thought of this as a bug since it seems very deliberate when I look at the code(even secondary vm's in hafnium have dtbs and are mapped so it sure looks like a feature as opposed to a bug. Why do you think this is a bug and unintentional?
[OD] We designed this code path in the early days of turning Hafnium into a SPMC. Yes we left this possibility open for the SP to observe its manifest, but with in mind this would have to be revisited in favor of using the FF-A boot protocol interface.
E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, >> FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
[RK] We need a way to pass platform specific arguments to the partitions anyway and manifest is a nice way to do this. The FFA discovery interfaces are good for things common across all partitions but not so great for platform specific thing. Did you have some other mechanism in mind to pass information to the partitions? Section 5.4 of the FF-A spec has a protocol for passing data but that is all implementation specific anyway so portability is gone out the window unless we explicitly make sure OPTEE and hafnium agree on the IMPDEF aspects of it.
[OD] I had in mind that the FF-A manifest should contain only well known fields based on the FF-A standard. Other platform specific information had to pass through the boot protocol interface. I agree there is a gap in defining common Tags in the more generic TLV encoding defined by the spec.
[OD] Sorry to say that but this looks very much like.a hack, right?
[RK] Yes 😊 no doubt about it. I was just pointing out that we have something for now to work around the issue 😊 For issue #1, the only possibility is to keep the DTB size < 4K. All the other problems you specify are also problems and we need to solve them. Again, what is the spec alternative to pass arguments in a completely standard way? 5.4 specifies the mechanism but the TLV's are totally IMPDEF, how do we allow for this while meeting the goal to run the same hafnium binary without resorting to platform ports like TF-A?
[OD] Fair enough. A possibility is to agree on Tags and describe generically defined values in the documentation. The FF-A spec leaves this impdef, but nothing prevents to define them at project level.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 23, 2021 1:46 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus... tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr%... ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%252 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...
Hi Raghu, Nicolas,
I'm fine with allocating more space to the manifest/dtb in general so it's ok to submit changes accordinly. I guess this is a flexibility that may be needed in different situations, like as Nicolas says it's understandable an FF-A manifest may exceed 4KB in a real platform.
The spec is there to reduce fragmentation and interopeability issues so indeed ideally we may want to adopt common approaches between Hafnium or an OP-TEE based SPMC.
Regards, Olivier.
________________________________________ From: Nicolas Benech nbenech@nvidia.com Sent: 29 July 2021 03:23 To: raghu.ncstate@icloud.com; Olivier Deprez; Varun Wadekar; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Adding extra details to this: we can easily go over 4K with standard FFA entries in the Manifest, for example device regions. A platform may need to define hundreds of them, so 4K would be exceeded very quickly.
Nicolas
-----Original Message----- From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Wednesday, July 28, 2021 5:52 PM To: 'Olivier Deprez' Olivier.Deprez@arm.com; Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Thanks Olivier. What you have below makes sense. Going back to the original issue in the email (which was about the 4K size limitation), are we suggesting that we leave the SP manifest as 4K in the sp package and revisit later? The second separate but related issue is that we wanted to use the manifest to pass large information to the SP (because of which we hit the first issue), but are we saying we will use the boot protocol to pass arguments/information, initially in a platform specific way, and then later hopefully in a more standard way? Once again, as we have discussed in a few other situations, will OPTEE adopt the same boot protocol? If not portability of the same FF-A partition (specifically S-EL0 partitions) to OPTEE may become an issue. In any case, I'd like to understand how we can pass somewhat large data(>4K) to a partition in a way that is platform agnostic(I don’t want a platform port like TF-A) and that is portable across OPTEE and Hafnium.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, July 27, 2021 4:57 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See few comments [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 23 June 2021 15:44 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis).
[RK] Did not realize you thought of this as a bug since it seems very deliberate when I look at the code(even secondary vm's in hafnium have dtbs and are mapped so it sure looks like a feature as opposed to a bug. Why do you think this is a bug and unintentional?
[OD] We designed this code path in the early days of turning Hafnium into a SPMC. Yes we left this possibility open for the SP to observe its manifest, but with in mind this would have to be revisited in favor of using the FF-A boot protocol interface.
E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, >> FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
[RK] We need a way to pass platform specific arguments to the partitions anyway and manifest is a nice way to do this. The FFA discovery interfaces are good for things common across all partitions but not so great for platform specific thing. Did you have some other mechanism in mind to pass information to the partitions? Section 5.4 of the FF-A spec has a protocol for passing data but that is all implementation specific anyway so portability is gone out the window unless we explicitly make sure OPTEE and hafnium agree on the IMPDEF aspects of it.
[OD] I had in mind that the FF-A manifest should contain only well known fields based on the FF-A standard. Other platform specific information had to pass through the boot protocol interface. I agree there is a gap in defining common Tags in the more generic TLV encoding defined by the spec.
[OD] Sorry to say that but this looks very much like.a hack, right?
[RK] Yes 😊 no doubt about it. I was just pointing out that we have something for now to work around the issue 😊 For issue #1, the only possibility is to keep the DTB size < 4K. All the other problems you specify are also problems and we need to solve them. Again, what is the spec alternative to pass arguments in a completely standard way? 5.4 specifies the mechanism but the TLV's are totally IMPDEF, how do we allow for this while meeting the goal to run the same hafnium binary without resorting to platform ports like TF-A?
[OD] Fair enough. A possibility is to agree on Tags and describe generically defined values in the documentation. The FF-A spec leaves this impdef, but nothing prevents to define them at project level.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 23, 2021 1:46 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus... tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr%... ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%252 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...
Thanks Olivier. Based on below, we can submit changes to remove the 4K limitation on the DTB size.
How about passing platform and SP specific information to the SP? For now, are we agreeing that passing this information in the partition manifest (that is mapped into the SP) is okay? If so, do we want to define a boot protocol for Hafnium/OPTEE to adopt to pass information? As suggested below, as a start should we just pass a pointer to the SP manifest to the partition since it is part of the sp pkg(as defined by TF-A's SP tool) and the fact that it is mapped into the partition already? Another option would be to have SP's carve a memory region for passing boot data. Hafnium already has support to allocate a memory region without a base address and has a TODO to let partition know about these memory regions. We can then extract impdef regions in the partition manifest and copy those portions into the allocated memory regions. This is of course assuming we don’t want to simply pass the partition manifest to the partition(I don’t see any downside of passing the partition manifest, so long as hafnium does not ever consume it after the partition starts running).
I'm happy to hear other proposals too. The requirements to meet at the moment would be: 1) standard(and portable to say OPTEE at least) way to pass large IMPDEF data to a partition. 2) should not require a platform specific code in hafnium. (it would be ideal if hafnium worked like the x86 linux kernel where you just build a kernel and drop it in with the proper device tree/acpi and it works but that’s a different discussion).
Thanks -Raghu -----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, July 29, 2021 9:00 AM To: Nicolas Benech nbenech@nvidia.com; raghu.ncstate@icloud.com; Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu, Nicolas,
I'm fine with allocating more space to the manifest/dtb in general so it's ok to submit changes accordinly. I guess this is a flexibility that may be needed in different situations, like as Nicolas says it's understandable an FF-A manifest may exceed 4KB in a real platform.
The spec is there to reduce fragmentation and interopeability issues so indeed ideally we may want to adopt common approaches between Hafnium or an OP-TEE based SPMC.
Regards, Olivier.
________________________________________ From: Nicolas Benech nbenech@nvidia.com Sent: 29 July 2021 03:23 To: raghu.ncstate@icloud.com; Olivier Deprez; Varun Wadekar; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Adding extra details to this: we can easily go over 4K with standard FFA entries in the Manifest, for example device regions. A platform may need to define hundreds of them, so 4K would be exceeded very quickly.
Nicolas
-----Original Message----- From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Wednesday, July 28, 2021 5:52 PM To: 'Olivier Deprez' Olivier.Deprez@arm.com; Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Thanks Olivier. What you have below makes sense. Going back to the original issue in the email (which was about the 4K size limitation), are we suggesting that we leave the SP manifest as 4K in the sp package and revisit later? The second separate but related issue is that we wanted to use the manifest to pass large information to the SP (because of which we hit the first issue), but are we saying we will use the boot protocol to pass arguments/information, initially in a platform specific way, and then later hopefully in a more standard way? Once again, as we have discussed in a few other situations, will OPTEE adopt the same boot protocol? If not portability of the same FF-A partition (specifically S-EL0 partitions) to OPTEE may become an issue. In any case, I'd like to understand how we can pass somewhat large data(>4K) to a partition in a way that is platform agnostic(I don’t want a platform port like TF-A) and that is portable across OPTEE and Hafnium.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, July 27, 2021 4:57 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See few comments [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 23 June 2021 15:44 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis).
[RK] Did not realize you thought of this as a bug since it seems very deliberate when I look at the code(even secondary vm's in hafnium have dtbs and are mapped so it sure looks like a feature as opposed to a bug. Why do you think this is a bug and unintentional?
[OD] We designed this code path in the early days of turning Hafnium into a SPMC. Yes we left this possibility open for the SP to observe its manifest, but with in mind this would have to be revisited in favor of using the FF-A boot protocol interface.
E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, >> FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
[RK] We need a way to pass platform specific arguments to the partitions anyway and manifest is a nice way to do this. The FFA discovery interfaces are good for things common across all partitions but not so great for platform specific thing. Did you have some other mechanism in mind to pass information to the partitions? Section 5.4 of the FF-A spec has a protocol for passing data but that is all implementation specific anyway so portability is gone out the window unless we explicitly make sure OPTEE and hafnium agree on the IMPDEF aspects of it.
[OD] I had in mind that the FF-A manifest should contain only well known fields based on the FF-A standard. Other platform specific information had to pass through the boot protocol interface. I agree there is a gap in defining common Tags in the more generic TLV encoding defined by the spec.
[OD] Sorry to say that but this looks very much like.a hack, right?
[RK] Yes 😊 no doubt about it. I was just pointing out that we have something for now to work around the issue 😊 For issue #1, the only possibility is to keep the DTB size < 4K. All the other problems you specify are also problems and we need to solve them. Again, what is the spec alternative to pass arguments in a completely standard way? 5.4 specifies the mechanism but the TLV's are totally IMPDEF, how do we allow for this while meeting the goal to run the same hafnium binary without resorting to platform ports like TF-A?
[OD] Fair enough. A possibility is to agree on Tags and describe generically defined values in the documentation. The FF-A spec leaves this impdef, but nothing prevents to define them at project level.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 23, 2021 1:46 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Raghu,
See inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 19:17 To: Olivier Deprez; 'Varun Wadekar'; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Thanks Olivier. Solving #2 on hafnium today is not so bad. The entire SP package is mapped into the partition by hafnium
[OD] Right, this is what I was alluding to at the beginning of this thread. Notice this is very much an Hafnium implementation specific side-effect of the manifest parsing code. Somehow a "bug" turned into a "feature" :) E.g. I was wondering if there would be any sensitive information in the manifest that we don't necessarily want to expose to a partition (need-to-know basis). The necessary information found in the SP FF-A manifest fields can be gathered by well-defined interfaces (FFA_ID_GET, FFA_PARTITION_INFO_GET etc.). So exposing through the memory map looks redundant.
so we just scan for a DTB magic word(Nico's idea) to figure out where it is and where to start parsing and we have successfully used it.
[OD] Sorry to say that but this looks very much like.a hack, right? I don't bother too much about what an SP does although I'm more concerned about the SPMC/SP interface. Nothing prevents the Hafnium manifest parsing code to be refactored some day and this unspecified property (mapping the SP dtb in the SP space) to be silently lost, breaking vendor SP implementations. Also this doesn't help scaling/migrating the SP from an SEL1 SPMC to an SEL2 SPMC (or a non-Hafnium implementation). I'm also thinking that a PIE built SP doesn't necessarily know its target address. So unless being a random search, the way to scan the SP memory requires build-time platform specific definitions (which can be ok but calls for scalability problems IMO).
For #1, I would suggest(as has been previously suggested) that we fix SP tool to patch the offset based on the dtb size during partition package creation.
[OD] Ok.
For #2, A hafnium specific boot protocol to pass the base address of the partition would be the simples to implement. This already exists for normal world primary partition where the address of the manifest is passed to it. We just need to extend to SP's.
[OD] Best is to stick to the FF-A spec IMO. In Hafnium, SPs are more looking like secondary VMs for which there is no such passing of a DTB (but rather the total size of the VM workspace if I recall).
Thanks -Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:35 AM To: raghu.ncstate@icloud.com; 'Varun Wadekar' vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy' raghupathyk@nvidia.com; 'Nicolas Benech' nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
I agree those are two different problems though I'm trying to bring the discussion farther to elaborate on the rationale and a plan. If you fix the first problem (move the entry point farther in the image, leaving room for the manifest), it remains the second problem that the SP doesn't know where to find the manifest. Hence solving 1. without 2. doesn't help you IMO. See more comments inline [OD]
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 22 June 2021 17:15 To: 'Varun Wadekar'; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: 'Raghupathy Krishnamurthy'; 'Nicolas Benech' Subject: RE: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Our initial ask was to only bump the SP image entrypoint past the DTB
boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise. [RK] Agree here. Passing information through the manifest to the SP is orthogonal to the main problem at hand where we are assuming that the size of manifest cannot be > 4K and the entrypoint offset is always at 0x100.
-Raghu
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Varun Wadekar via TF-A Sent: Tuesday, June 22, 2021 7:35 AM To: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hi Olivier,
[OD] The two questions are which entity is loading boot data (can be a DT
blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[VW] You are right - bootloader loads SP.pkg (header+manifest+image) into memory. Can you please help me understand what "SP boot data" contains?
[OD] By SP boot data I mean any SP manifest node which is not by FF-A standard and hence not consumed by the SPMC (but could be consumed by the SP).
For passing the SP manifest pointer to the SP we should ask for ideas from the community.
Some ideas to get the discussion started: 1. SPMC creates a mapping for the manifest in the SP's virtual address space. SP issues a SVC call to get the virtual address from the SPMC 2. SPMC creates a mapping for the manifest in the SP's virtual address space and passes the pointer to SP via X0 3. SP manifest contains an entry to publish the virtual address for its manifest. SPMC reads this entry and maps the manifest at that virtual address during boot.
[OD] From a spec perspective it is an already solved problem, see DEN0077A FF-A v1.1 Alpha0 section 3.8 but it's not using a DT approach.
[OD] If the assumptions (implementation defined) are that the SP boot
data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
[VW] Our initial ask was to only bump the SP image entrypoint past the DTB boundary while generating SP.pkg using sptool. IIUC, the issue you brought up is for passing the manifest base address to the SP at runtime. I think these two are independent issues. Please advise.
[OD] Yes I agree. As said above, when you've done this (move the entry point), how does it help the SP to find the data?
-Varun
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, June 22, 2021 8:17 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
(sorry last message sent too fast) See few comments inline [OD].
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 17 June 2021 12:33 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
[OD] Ok. How is the SP informed about where to find the dtb (IPA
address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
[VW] We don't have a policy for all SPs, yet. At least I don't know of one. But in our experimental setup we plan to work on some assumptions e.g. assume the manifest blob to always be present at a certain address.
[OD] The two questions are which entity is loading boot data (can be a DT blob) that is to be consumed by the SP? And how is the address for boot data conveyed to the SP? The SP manifest is made for the SPMC consumption. I guess it's ok to conflate DT information required by the SP into the SP manifest. It means the bootloader loads both the SP manifest and the SP boot data in a single "SP package". In current implementation the SP has no standard way to know where the DT blob is loaded in its own memory range. We have experimental patches around the FF-A boot protocol which may help.
[OD] I think review link is mysteriously broken and the review lost. But
Joao can provide the new link if necessary. I understand the intent of not making things too complex.
[VW] Can we list down the next steps? Do we agree in principle that sptool needs to be upgraded to handle this use case? We can collaborate on an implementation, once we agree on the direction.
[OD] If the assumptions (implementation defined) are that the SP boot data fits into the SP manifest, and that the SP finds this data at a fixed address (e.g. offset 0 in the SP package) then yes it can be a way to go.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Thursday, June 17, 2021 8:40 AM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
Few comments [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 14:37 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
Hi,
Just a small heads up VHE SEL0 changes are tested within the Hafnium CI
presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI.
[VW] Thanks for the heads up.
what is the order in size for the manifest dtb you require in Tegra
platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[VW] We can assume tens of KB. In the worst case, a few hundred KB. The manifest is targeted towards the SP and not meant for Hafnium. Just curious, do you see any pitfalls?
[OD] Ok. How is the SP informed about where to find the dtb (IPA address)? Saying this because the boot protocol between Hafnium and SPs is very much implementation specific at this moment and does not strictly follow the FF-A spec recommendations (data passing as TLVs). There is a trick that the manifest blob remains fortunately mapped from offset 0 in the SP IPA range. So it may be able to extract manifest data by mapping it the Stage-1 translation regime. But as said this does not follow a standard.
To set the complete picture, Joao had made some exploration around more
dynamic SP configuration [2], maybe there are ideas to gather from there.
[VW] Thanks for the links. I will review and leave comments - although we are not looking at introducing yet another script.
[OD] I think review link is mysteriously broken and the review lost. But Joao can provide the new link if necessary. I understand the intent of not making things too complex.
Cheers.
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 12:59 PM To: Varun Wadekar vwadekar@nvidia.com; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Nicolas Benech nbenech@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi Varun,
See inline [OD]
Regards, Olivier.
________________________________________ From: Varun Wadekar vwadekar@nvidia.com Sent: 16 June 2021 13:07 To: Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: RE: SP manifest: avoid manual updates to "entrypoint-offset"
HI Olivier,
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim
No, this issue can be seen even without the shim. I don't think the usage of shim or VHE really matters here.
[OD] All right. Just a small heads up VHE SEL0 changes are tested within the Hafnium CI presently and do not relate to TF-A CI. sptool is a TF-A tool and there is no link to the Hafnium CI. Not an issue as such but just making sure you have the full picture. At some point in time, when Hf VHE changes are merged we can create a specific TF-A test config to cover this case if necessary. Out of curiosity what is the order in size for the manifest dtb you require in Tegra platform? Are the supplementary nodes in this manifest consumed by Hafnium or the SP itself?
[OD] This does not relate exactly to your problem but the effect is the
"same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
I agree, this approach is good for the short term. But I propose we craft a forward-looking solution. Are you planning on merging the sptool patch[1]? If not, I propose we come up with a way to dynamically update the SP manifest - I understand this involves adding libfdt to update the manifest.
[OD] There is no pressing need to merge [1] in the short term so this can wait for a better solution. To set the complete picture, Joao had made some exploration around more dynamic SP configuration [2], maybe there are ideas to gather from there.
-Varun
[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... stedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7C 01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d157273 40c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8ey JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0 [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus... tedfirmware.org%2Fpipermail%2Fhafnium%2F2020-August%2F000078.html&data=0 4%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15 727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3 d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& amp;sdata=bWHEuLPln2Ke6KI5YXAdCRpa%2FY1w17ZVguZv8H864EM%3D&reserved=0
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Wednesday, June 16, 2021 11:42 AM To: tf-a@lists.trustedfirmware.org; Nicolas Benech nbenech@nvidia.com Cc: Raghupathy Krishnamurthy raghupathyk@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: SP manifest: avoid manual updates to "entrypoint-offset"
External email: Use caution opening links or attachments
Hi,
See few comments inline [OD]
Regards, Olivier.
________________________________________ From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Varun Wadekar via TF-A tf-a@lists.trustedfirmware.org Sent: 16 June 2021 11:41 To: tf-a@lists.trustedfirmware.org Cc: Raghupathy Krishnamurthy; Nicolas Benech Subject: [TF-A] SP manifest: avoid manual updates to "entrypoint-offset"
Hello,
We (Nico/Raghu) have been implementing SEL0 partitions for Tegra platforms
[OD] Just to be clear, is this about SEL0 partitions using an SEL1 shim (as per https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tru... stedfirmware.org%2Fq%2Ftopic%3A%2522db%25252Fsel0%2522&data=04%7C01%7Cvw adekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727340c1b7d b39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A PxWmULcUDvPixco3iSBvl8D%2BVN%2B3x8kMy%2Fxk1Mmju8%3D&reserved=0+(status:o pen%20OR%20status:merged) )?
and recently hit an issue, where the "entrypoint-offset" field of the SP manifest [1] cannot cope with increasing manifest blobs. The way the SP manifest is created today, the "entrypoint-offset" field is set to a value *statically* by the implementer. Down the line if the manifest grows past the value written in the "entrypoint-offset", we must manually update it. This needs to be fixed.
We believe there is an opportunity to upgrade the sptool to handle this situation during SP package creation, where sptool calculates the manifest size and bumps the "entrypoint-offset" past the end of the manifest. There are other ways of patching the SP manifest at runtime, but they seem sub-optimal.
Please let me know if there are other ideas to solve this problem. I will post a patch to update the sptool shortly but wanted to get the ball rolling.
[OD] We had an attempt (https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr%... ustedfirmware.org%2Fc%2FTF-A%2Ftrusted-firmware-a%2F%2B%2F8536&data=04%7 C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a755e508d9354dc537%7C43083d15727 340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533388766%7CUnknown%7CTWFpbGZsb3d8e yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& ;sdata=Zp%2FSn5Mgj5OIkeVuVVoGm5%2BWqWCP9v02Whcf8%2FKlvVg%3D&reserved=0) to move the entry point farther in the image such that we can support 64KB granules in the Stage-1 translation regime. This does not relate exactly to your problem but the effect is the "same" (aka there is more room for the manifest dtb). Agree this still requires hard-coding the entry point. At this stage I had thought of pushing the entry point even farther at a reasonably large 64KB aligned offset such that it helps with both problems.
Cheers.
[1] 14. FF-A manifest binding to device tree - Trusted Firmware-A documentationhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%252 F%2Ftrustedfirmware-a.readthedocs.io%2Fen%2Flatest%2Fcomponents%2Fffa-manife st-binding.html&data=04%7C01%7Cvwadekar%40nvidia.com%7C2aee1716913143a75 5e508d9354dc537%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637599430533398 725%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h aWwiLCJXVCI6Mn0%3D%7C3000&sdata=0DzGamn2KakN5A7n42%2BI9n5UUaPwG3BXg6NLhy kd8NE%3D&reserved=0 -- TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...
tf-a@lists.trustedfirmware.org
-
Nicolas Benech -
Olivier Deprez -
raghu.ncstate＠icloud.com -
Varun Wadekar