Hi,
I noticed OP-TEE accesses CNTPCT_EL1 aka the physical system counter for different reasons (small delay loops, user session time, debug/trace,...)
I wonder how this should be supported when OP-TEE runs as a Secure Partition (and more generally for other TEEs and bare metal SPs) under the SPMC.
I guess this is ok as long as SPs are all statically loaded at boot time (that is all SPs use the same t0 reference time), and thus the virtual system time is similar to the physical system time.
For a VM running under Hafnium, EL1 physical counter and timer accesses are:
-not trapped when accessed from the PVM.
-trapped when accessed from secondary VMs.
(https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/arch/aarch64/h…)
Hafnium (or the SPMC):
-does not access the EL1 physical counter (CNTPCT), nor the EL1 physical timer (CNTP)
-does not setup the virtual counter offset (CNTVOFF_EL2=0 which leads to CNTVCT=CNTPCT)
-emulates the EL1 timer using virtual timer registers (CNTVCT and CNTV), and Hypervisor timer (CNTHP) for the primary VM.
Considering OP-TEE currently uses the physical counter (and does not use any architected timer), options are:
-do not trap CNTPCT accesses. The problem I see here in the longer term, the virtual time does no longer match physical time (e.g. consider dynamic loading of SPs).
-the SPMC traps CNTPCT accesses for all SPs (and maybe emulate like non-VHE KVM does, but this incurs latency in returning the virtual counter value).
-OP-TEE replaces its CNTPCT usage by CNTVCT. The latter behaves such that if virtualization is disabled accesses are made with CNTVCT=CNTPCT. I tend to think that's the approach used by linux (needs further check though).
Let me know your thoughts (@Achin, @Andrew, @JensW, others ...)
Other considerations:
-With S-EL2 enabled, the EL3 physical timer cannot be accessed from lower ELs whichever SCR_EL3.ST (the behavior is slightly different when virtualization is disabled).
-S-EL2 physical (CNTHPS) and virtual (CNTHVS) timers are unused.
-The EL1 virtual timer interrupt can fire only under the condition that the SP is voluntarily scheduled either by a direct message request or ffa_run. In other words such interrupt is never triggered by HW, but because the SPMC injects it to an SP vCPU which explicitly enabled it, when the virtual timer expires.
- I noticed CNTPCT/CNTVCT direct reads within OP-TEE and Hafnium code bases. Arm ARM quite clearly mentions that such reads must be preceded by an ISB to forbid re-ordering. I think it's worth doing the change.
Regards,
Olivier.
Sorry for chiming in late. Whether or not the FF-A spec forbids exchanging executable pages, it would be better as a general rule to limit execute permissions to an SP, unless explicitly requested by the SP. I cant think of a reason why an SP would need to take donated NS memory and map it as an executable page unless it generates code or loads new code at run time. This, in my mind, is just another attack vector and potential "feature" to misuse, in case SP's don't sanitize pages for ex.
I would be okay with limiting all memory exchanges between NWd and SWd to only operate on RW and have NX enabled always, unless there is an explicit use case some one comes up with that requires it.
Also, shared pages between NWd and SWd should definitely not have execute permissions.
Thanks
Raghu
-----Original Message-----
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> On Behalf Of Andrew Walbran via Hafnium
Sent: Tuesday, October 20, 2020 9:07 AM
To: Olivier Deprez <Olivier.Deprez(a)arm.com>
Cc: hafnium(a)lists.trustedfirmware.org
Subject: Re: [Hafnium] FFA Memory Management Interfaces
On Tue, 20 Oct 2020 at 16:51, Olivier Deprez <Olivier.Deprez(a)arm.com> wrote:
> Hi Andrew,
>
> On the first question it relates to this line
>
> https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008/8/src/arch
> /aarch64/hypervisor/other_world.c#92
>
> and whether we should map the "Hypervisor VM" pages as RWX provided a
> SP must never execute from an NS memory region. So it should never
> happen that a VM grants execute permissions to an NS buffer shared to a SP.
>
I don't think it can happen if the memory region is shared, but why shouldn't an SP be allowed to retrieve a memory region donated or lent from a normal world VM to map as executable? Is there something in the FF-A spec that forbids that, or some reason we wouldn't want to allow it?
>
> Or should this be enforced by the Relayer/SPMC on a mem retrieve that
> an NS memory region will always be mapped RW / NX?
>
> Regards,
> Olivier.
>
> ________________________________________
> From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of
> Andrew Walbran via Hafnium <hafnium(a)lists.trustedfirmware.org>
> Sent: 20 October 2020 17:37
> To: Joao Alves
> Cc: hafnium(a)lists.trustedfirmware.org
> Subject: Re: [Hafnium] FFA Memory Management Interfaces
>
> On Tue, 20 Oct 2020 at 11:18, Joao Alves <Joao.Alves(a)arm.com> wrote:
>
> > Hello All,
> >
> > We have been working on enabling the use memory management
> > interfaces between SWd and NWd, on Hafnium as SPMC. We identified a
> > few points that would benefit from a general
> > discussion/clarification through the Mailing List for this work to progress on time for the first release.
> >
> > The whole memory has been mapped to a VM representative of the NWd
> > FFA endpoints (in the code can be found as a global variable called
> > 'other_world_vm') in this patch
> > <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> > All the memory is configured as RW memory, to be permissive enough
> > to
> ease
> > the validation of the sender's original permissions vs the requested
> > sharing permissions.
> > This approach seems valid given the following assumptions: the
> > 'other_world_vm' is never scheduled, which means that there is no
> > risk of memory leakage; the hypervisor should do the necessary
> > validation of the permissions that a given NWd FFA endpoint has on a
> > given memory region, before forwarding the handling of the operation to the SPMC.
> >
> > The first point to bring to discussion is regarding the RW configuration.
> >
> > *Is this really the best configuration we would expect for memory to
> > be shared from NWd to the SWd? Or is this excluding any use-case we
> > are not aware of (e.g. at this point we are restricting instruction
> > permissions
> to
> > NX)?*
> >
> Is there any reason we need to restrict it to NX? Usually a VM owning
> memory implies that it has full RWX access to it, as it would without
> a hypervisor involved. We assume that it should be able to grant any
> subset of those permissions when donating or lending the memory,
> subject to FF-A or other policy restrictions.
>
> Whilst testing the memory management interfaces, I noticed that if the
> > instruction access on the memory region descriptor (in the spec has
> > been described in section 5.12, and in the code defined as 'struct
> > ffa_memory_region') is set as 'not specified' for the Lend and
> > Donate operations, the receiver would acquire permission to execute
> > the memory region. This is due to the implementation of the function '
> > ffa_memory_permissions_to_mode
> > <
> https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memor
> y.c#n393
> >
> > <
> https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memor
> y.c#n393
> >',
> > which sets the memory mode to MM_MODE_X for case instruction access
> >FFA_INSTRUCTION_ACCESS_NOT_SPECIFIED. My understanding is that in
> >such
> case
> > MM_MODE_X shouldn't be set. Please check this change
> > <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> > *@Andrew Walbran <qwandor(a)google.com>, is the change I provided correct?
> > Or is there a reason for the current implementation of
> > 'ffa_memory_permissions_to_mode'?*
> >
> Again, the assumption is that the owner of a page always has full
> access to it. From the FF-A spec, section 5.11.2.2 says "In a
> transaction to donate memory... if the receiver is a PE, the owner
> must not specify the level of access". Section 5.11.3.2 says the same
> thing for the execute permission in the case of donating or lending to
> a single borrower. The spec says that the retriever should specify the
> permissions it wants, but it is not required. So in these cases the
> retriever may specify any permissions, and if it doesn't specify any then we give it full access.
>
> I also noticed that after a memory reclaim, the sender/owner of a
> memory
> > region acquires RW and X permissions, even if these were not part of
> > the original permissions when sending the memory region. This is due
> > to definition of the variable 'memory_to_attributes' in the function '
> > ffa_memory_reclaim
> > <
> https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memor
> y.c#n2576
> >
> > ':
> > uint32_t memory_to_attributes = MM_MODE_R | MM_MODE_W | MM_MODE_X;
> > Down the call stack, the value of this variable is used to update
> > the variable 'to_mode' in function 'ffa_retrieve_check_transition <
> https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memor
> y.c#n665
> >'.
> > The value of variable 'to_mode' is later passed to
> >'ffa_region_group_identity_map'
> > <
> https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memor
> y.c#n1018
> >
> > to update the owner's memory mapping. My understanding is that the
> sender's
> > memory mode should be reset to the mode it had prior to the original
> memory
> > send operation, and that this is going to be a problem for memory
> > management operations (even between between VMs and between SPs). I
> > have seen this behavior on my set, however I would like to validate
> > my
> analysis.
> > *@Andrew Walbran <qwandor(a)google.com> would you be able to provide
> > any comments on this?*
> >
> Our assumption, as above, is that the initial owner of a page starts
> with full access to it, and so this is what should be restored when it
> is reclaimed.
>
> Please let me know if anyone has any comments/questions.
> >
> > Best regards,
> > João Alves
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
Hello,
In the specification, section 5.4, after the definition of access there is a list of all accesses, and the missing combination is RWX.
Someone else could comment here, this might have been a decision with security in mind, to avoid a writable section that is also executable. At least according to the spec, the owner could be sharing something else other than an RWX memory region.
A partition's manifest may contain memory regions that will be owned by the partition. The memory attributes for that memory is also specified in the manifest. My understanding is that there is nothing preventing that memory from being used for memory management operations, whilst having configuration different than RWX. Am I thinking correctly?
Also, currently the configuration of SCR_EL3.SIF (configured when booting TF-A) defines that no SWd component can execute from a NS memory segment. Does it make sense to enforce this at the 2nd stage of translation as well now that we are concerned with memory sharing between NWd and SWd? This is not to be changed now, but could be that SCR_EL3.SIF will need to be updated?
I think that making the memory management operations implementation generic (i.e. not tied to any assumption), would prevent the above points to be a problem, even if in practice what you say is the case (i.e. owned memory is always RWX) because that would also be preserved.
Best regards,
João Alves
________________________________
From: Andrew Walbran
Sent: Thursday, October 22, 2020 2:01 PM
To: Joao Alves
Cc: hafnium(a)lists.trustedfirmware.org
Subject: Re: [Hafnium] FFA Memory Management Interfaces
Hello,
I still don't understand why a VM which owns a memory region and has exclusive access to it shouldn't have full RWX access permissions to it. What purpose does this serve? If the VM wants to restrict its own access it can do so with the permissions in its stage 1 page table. What other component should be controlling the VM's access to its own exclusively owned memory?
On Thu, 22 Oct 2020 at 10:31, Joao Alves <Joao.Alves(a)arm.com<mailto:Joao.Alves@arm.com>> wrote:
Hello Andrew,
Thank you for your addressing our questions on the previous email.
The section 5.4 of the spec defines ownership as "... software attribute that determines if a component can grant access to a memory region to anothercomponent."
The same section also defines the accesses that a component may have on a memory region.
I prepared the following patch: https://review.trustedfirmware.org/c/hafnium/hafnium/+/6521. It explicitly saves the access the sender had prior to the memory send operation, and restores that same value at a memory reclaim. This change makes the code generic, as it complies with all use-cases.
Also the patch: https://review.trustedfirmware.org/c/hafnium/hafnium/+/6471. Assuming the use-cases where the owner does not have execute permissions, the receiver's permissions will be set as NX if the Instruction Access is not specified.
Would you be able to review the changes?
Let us know if you have any concerns regarding those patches.
Best regards,
João Alves
________________________________
From: Andrew Walbran
Sent: Tuesday, October 20, 2020 5:06 PM
To: Olivier Deprez
Cc: Joao Alves; hafnium(a)lists.trustedfirmware.org<mailto:hafnium@lists.trustedfirmware.org>
Subject: Re: [Hafnium] FFA Memory Management Interfaces
On Tue, 20 Oct 2020 at 16:51, Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>> wrote:
Hi Andrew,
On the first question it relates to this line
https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008/8/src/arch/aarc…
and whether we should map the "Hypervisor VM" pages as RWX provided a SP must never execute from an NS memory region. So it should never happen that a VM grants execute permissions to an NS buffer shared to a SP.
I don't think it can happen if the memory region is shared, but why shouldn't an SP be allowed to retrieve a memory region donated or lent from a normal world VM to map as executable? Is there something in the FF-A spec that forbids that, or some reason we wouldn't want to allow it?
Or should this be enforced by the Relayer/SPMC on a mem retrieve that an NS memory region will always be mapped RW / NX?
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org<mailto:hafnium-bounces@lists.trustedfirmware.org>> on behalf of Andrew Walbran via Hafnium <hafnium(a)lists.trustedfirmware.org<mailto:hafnium@lists.trustedfirmware.org>>
Sent: 20 October 2020 17:37
To: Joao Alves
Cc: hafnium(a)lists.trustedfirmware.org<mailto:hafnium@lists.trustedfirmware.org>
Subject: Re: [Hafnium] FFA Memory Management Interfaces
On Tue, 20 Oct 2020 at 11:18, Joao Alves <Joao.Alves(a)arm.com<mailto:Joao.Alves@arm.com>> wrote:
> Hello All,
>
> We have been working on enabling the use memory management interfaces
> between SWd and NWd, on Hafnium as SPMC. We identified a few points that
> would benefit from a general discussion/clarification through the Mailing
> List for this work to progress on time for the first release.
>
> The whole memory has been mapped to a VM representative of the NWd FFA
> endpoints (in the code can be found as a global variable called
> 'other_world_vm') in this patch
> <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> All the memory is configured as RW memory, to be permissive enough to ease
> the validation of the sender's original permissions vs the requested
> sharing permissions.
> This approach seems valid given the following assumptions: the
> 'other_world_vm' is never scheduled, which means that there is no risk of
> memory leakage; the hypervisor should do the necessary validation of the
> permissions that a given NWd FFA endpoint has on a given memory region,
> before forwarding the handling of the operation to the SPMC.
>
> The first point to bring to discussion is regarding the RW configuration.
>
> *Is this really the best configuration we would expect for memory to be
> shared from NWd to the SWd? Or is this excluding any use-case we are not
> aware of (e.g. at this point we are restricting instruction permissions to
> NX)?*
>
Is there any reason we need to restrict it to NX? Usually a VM owning
memory implies that it has full RWX access to it, as it would without a
hypervisor involved. We assume that it should be able to grant any subset
of those permissions when donating or lending the memory, subject to FF-A
or other policy restrictions.
Whilst testing the memory management interfaces, I noticed that if the
> instruction access on the memory region descriptor (in the spec has been
> described in section 5.12, and in the code defined as 'struct
> ffa_memory_region') is set as 'not specified' for the Lend and Donate
> operations, the receiver would acquire permission to execute the memory
> region. This is due to the implementation of the function '
> ffa_memory_permissions_to_mode
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>',
> which sets the memory mode to MM_MODE_X for case instruction access
> FFA_INSTRUCTION_ACCESS_NOT_SPECIFIED. My understanding is that in such case
> MM_MODE_X shouldn't be set. Please check this change
> <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> *@Andrew Walbran <qwandor(a)google.com<mailto:qwandor@google.com>>, is the change I provided correct?
> Or is there a reason for the current implementation of
> 'ffa_memory_permissions_to_mode'?*
>
Again, the assumption is that the owner of a page always has full access to
it. From the FF-A spec, section 5.11.2.2 says "In a transaction to donate
memory... if the receiver is a PE, the owner must not specify the level of
access". Section 5.11.3.2 says the same thing for the execute permission in
the case of donating or lending to a single borrower. The spec says that
the retriever should specify the permissions it wants, but it is not
required. So in these cases the retriever may specify any permissions, and
if it doesn't specify any then we give it full access.
I also noticed that after a memory reclaim, the sender/owner of a memory
> region acquires RW and X permissions, even if these were not part of the
> original permissions when sending the memory region. This is due to
> definition of the variable 'memory_to_attributes' in the function '
> ffa_memory_reclaim
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> ':
> uint32_t memory_to_attributes = MM_MODE_R | MM_MODE_W | MM_MODE_X;
> Down the call stack, the value of this variable is used to update the
> variable 'to_mode' in function 'ffa_retrieve_check_transition
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>'.
> The value of variable 'to_mode' is later passed to
> 'ffa_region_group_identity_map'
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> to update the owner's memory mapping. My understanding is that the sender's
> memory mode should be reset to the mode it had prior to the original memory
> send operation, and that this is going to be a problem for memory
> management operations (even between between VMs and between SPs). I have
> seen this behavior on my set, however I would like to validate my analysis.
> *@Andrew Walbran <qwandor(a)google.com<mailto:qwandor@google.com>> would you be able to provide any
> comments on this?*
>
Our assumption, as above, is that the initial owner of a page starts with
full access to it, and so this is what should be restored when it is
reclaimed.
Please let me know if anyone has any comments/questions.
>
> Best regards,
> João Alves
>
>
>
>
>
>
>
>
>
>
Hello Andrew,
Thank you for your addressing our questions on the previous email.
The section 5.4 of the spec defines ownership as "... software attribute that determines if a component can grant access to a memory region to anothercomponent."
The same section also defines the accesses that a component may have on a memory region.
I prepared the following patch: https://review.trustedfirmware.org/c/hafnium/hafnium/+/6521. It explicitly saves the access the sender had prior to the memory send operation, and restores that same value at a memory reclaim. This change makes the code generic, as it complies with all use-cases.
Also the patch: https://review.trustedfirmware.org/c/hafnium/hafnium/+/6471. Assuming the use-cases where the owner does not have execute permissions, the receiver's permissions will be set as NX if the Instruction Access is not specified.
Would you be able to review the changes?
Let us know if you have any concerns regarding those patches.
Best regards,
João Alves
________________________________
From: Andrew Walbran
Sent: Tuesday, October 20, 2020 5:06 PM
To: Olivier Deprez
Cc: Joao Alves; hafnium(a)lists.trustedfirmware.org
Subject: Re: [Hafnium] FFA Memory Management Interfaces
On Tue, 20 Oct 2020 at 16:51, Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>> wrote:
Hi Andrew,
On the first question it relates to this line
https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008/8/src/arch/aarc…
and whether we should map the "Hypervisor VM" pages as RWX provided a SP must never execute from an NS memory region. So it should never happen that a VM grants execute permissions to an NS buffer shared to a SP.
I don't think it can happen if the memory region is shared, but why shouldn't an SP be allowed to retrieve a memory region donated or lent from a normal world VM to map as executable? Is there something in the FF-A spec that forbids that, or some reason we wouldn't want to allow it?
Or should this be enforced by the Relayer/SPMC on a mem retrieve that an NS memory region will always be mapped RW / NX?
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org<mailto:hafnium-bounces@lists.trustedfirmware.org>> on behalf of Andrew Walbran via Hafnium <hafnium(a)lists.trustedfirmware.org<mailto:hafnium@lists.trustedfirmware.org>>
Sent: 20 October 2020 17:37
To: Joao Alves
Cc: hafnium(a)lists.trustedfirmware.org<mailto:hafnium@lists.trustedfirmware.org>
Subject: Re: [Hafnium] FFA Memory Management Interfaces
On Tue, 20 Oct 2020 at 11:18, Joao Alves <Joao.Alves(a)arm.com<mailto:Joao.Alves@arm.com>> wrote:
> Hello All,
>
> We have been working on enabling the use memory management interfaces
> between SWd and NWd, on Hafnium as SPMC. We identified a few points that
> would benefit from a general discussion/clarification through the Mailing
> List for this work to progress on time for the first release.
>
> The whole memory has been mapped to a VM representative of the NWd FFA
> endpoints (in the code can be found as a global variable called
> 'other_world_vm') in this patch
> <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> All the memory is configured as RW memory, to be permissive enough to ease
> the validation of the sender's original permissions vs the requested
> sharing permissions.
> This approach seems valid given the following assumptions: the
> 'other_world_vm' is never scheduled, which means that there is no risk of
> memory leakage; the hypervisor should do the necessary validation of the
> permissions that a given NWd FFA endpoint has on a given memory region,
> before forwarding the handling of the operation to the SPMC.
>
> The first point to bring to discussion is regarding the RW configuration.
>
> *Is this really the best configuration we would expect for memory to be
> shared from NWd to the SWd? Or is this excluding any use-case we are not
> aware of (e.g. at this point we are restricting instruction permissions to
> NX)?*
>
Is there any reason we need to restrict it to NX? Usually a VM owning
memory implies that it has full RWX access to it, as it would without a
hypervisor involved. We assume that it should be able to grant any subset
of those permissions when donating or lending the memory, subject to FF-A
or other policy restrictions.
Whilst testing the memory management interfaces, I noticed that if the
> instruction access on the memory region descriptor (in the spec has been
> described in section 5.12, and in the code defined as 'struct
> ffa_memory_region') is set as 'not specified' for the Lend and Donate
> operations, the receiver would acquire permission to execute the memory
> region. This is due to the implementation of the function '
> ffa_memory_permissions_to_mode
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>',
> which sets the memory mode to MM_MODE_X for case instruction access
> FFA_INSTRUCTION_ACCESS_NOT_SPECIFIED. My understanding is that in such case
> MM_MODE_X shouldn't be set. Please check this change
> <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> *@Andrew Walbran <qwandor(a)google.com<mailto:qwandor@google.com>>, is the change I provided correct?
> Or is there a reason for the current implementation of
> 'ffa_memory_permissions_to_mode'?*
>
Again, the assumption is that the owner of a page always has full access to
it. From the FF-A spec, section 5.11.2.2 says "In a transaction to donate
memory... if the receiver is a PE, the owner must not specify the level of
access". Section 5.11.3.2 says the same thing for the execute permission in
the case of donating or lending to a single borrower. The spec says that
the retriever should specify the permissions it wants, but it is not
required. So in these cases the retriever may specify any permissions, and
if it doesn't specify any then we give it full access.
I also noticed that after a memory reclaim, the sender/owner of a memory
> region acquires RW and X permissions, even if these were not part of the
> original permissions when sending the memory region. This is due to
> definition of the variable 'memory_to_attributes' in the function '
> ffa_memory_reclaim
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> ':
> uint32_t memory_to_attributes = MM_MODE_R | MM_MODE_W | MM_MODE_X;
> Down the call stack, the value of this variable is used to update the
> variable 'to_mode' in function 'ffa_retrieve_check_transition
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>'.
> The value of variable 'to_mode' is later passed to
> 'ffa_region_group_identity_map'
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> to update the owner's memory mapping. My understanding is that the sender's
> memory mode should be reset to the mode it had prior to the original memory
> send operation, and that this is going to be a problem for memory
> management operations (even between between VMs and between SPs). I have
> seen this behavior on my set, however I would like to validate my analysis.
> *@Andrew Walbran <qwandor(a)google.com<mailto:qwandor@google.com>> would you be able to provide any
> comments on this?*
>
Our assumption, as above, is that the initial owner of a page starts with
full access to it, and so this is what should be restored when it is
reclaimed.
Please let me know if anyone has any comments/questions.
>
> Best regards,
> João Alves
>
>
>
>
>
>
>
>
>
>
Hi Andrew,
On the first question it relates to this line
https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008/8/src/arch/aarc…
and whether we should map the "Hypervisor VM" pages as RWX provided a SP must never execute from an NS memory region. So it should never happen that a VM grants execute permissions to an NS buffer shared to a SP.
Or should this be enforced by the Relayer/SPMC on a mem retrieve that an NS memory region will always be mapped RW / NX?
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Andrew Walbran via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 20 October 2020 17:37
To: Joao Alves
Cc: hafnium(a)lists.trustedfirmware.org
Subject: Re: [Hafnium] FFA Memory Management Interfaces
On Tue, 20 Oct 2020 at 11:18, Joao Alves <Joao.Alves(a)arm.com> wrote:
> Hello All,
>
> We have been working on enabling the use memory management interfaces
> between SWd and NWd, on Hafnium as SPMC. We identified a few points that
> would benefit from a general discussion/clarification through the Mailing
> List for this work to progress on time for the first release.
>
> The whole memory has been mapped to a VM representative of the NWd FFA
> endpoints (in the code can be found as a global variable called
> 'other_world_vm') in this patch
> <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> All the memory is configured as RW memory, to be permissive enough to ease
> the validation of the sender's original permissions vs the requested
> sharing permissions.
> This approach seems valid given the following assumptions: the
> 'other_world_vm' is never scheduled, which means that there is no risk of
> memory leakage; the hypervisor should do the necessary validation of the
> permissions that a given NWd FFA endpoint has on a given memory region,
> before forwarding the handling of the operation to the SPMC.
>
> The first point to bring to discussion is regarding the RW configuration.
>
> *Is this really the best configuration we would expect for memory to be
> shared from NWd to the SWd? Or is this excluding any use-case we are not
> aware of (e.g. at this point we are restricting instruction permissions to
> NX)?*
>
Is there any reason we need to restrict it to NX? Usually a VM owning
memory implies that it has full RWX access to it, as it would without a
hypervisor involved. We assume that it should be able to grant any subset
of those permissions when donating or lending the memory, subject to FF-A
or other policy restrictions.
Whilst testing the memory management interfaces, I noticed that if the
> instruction access on the memory region descriptor (in the spec has been
> described in section 5.12, and in the code defined as 'struct
> ffa_memory_region') is set as 'not specified' for the Lend and Donate
> operations, the receiver would acquire permission to execute the memory
> region. This is due to the implementation of the function '
> ffa_memory_permissions_to_mode
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>',
> which sets the memory mode to MM_MODE_X for case instruction access
> FFA_INSTRUCTION_ACCESS_NOT_SPECIFIED. My understanding is that in such case
> MM_MODE_X shouldn't be set. Please check this change
> <https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
> *@Andrew Walbran <qwandor(a)google.com>, is the change I provided correct?
> Or is there a reason for the current implementation of
> 'ffa_memory_permissions_to_mode'?*
>
Again, the assumption is that the owner of a page always has full access to
it. From the FF-A spec, section 5.11.2.2 says "In a transaction to donate
memory... if the receiver is a PE, the owner must not specify the level of
access". Section 5.11.3.2 says the same thing for the execute permission in
the case of donating or lending to a single borrower. The spec says that
the retriever should specify the permissions it wants, but it is not
required. So in these cases the retriever may specify any permissions, and
if it doesn't specify any then we give it full access.
I also noticed that after a memory reclaim, the sender/owner of a memory
> region acquires RW and X permissions, even if these were not part of the
> original permissions when sending the memory region. This is due to
> definition of the variable 'memory_to_attributes' in the function '
> ffa_memory_reclaim
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> ':
> uint32_t memory_to_attributes = MM_MODE_R | MM_MODE_W | MM_MODE_X;
> Down the call stack, the value of this variable is used to update the
> variable 'to_mode' in function 'ffa_retrieve_check_transition
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>'.
> The value of variable 'to_mode' is later passed to
> 'ffa_region_group_identity_map'
> <https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>
> to update the owner's memory mapping. My understanding is that the sender's
> memory mode should be reset to the mode it had prior to the original memory
> send operation, and that this is going to be a problem for memory
> management operations (even between between VMs and between SPs). I have
> seen this behavior on my set, however I would like to validate my analysis.
> *@Andrew Walbran <qwandor(a)google.com> would you be able to provide any
> comments on this?*
>
Our assumption, as above, is that the initial owner of a page starts with
full access to it, and so this is what should be restored when it is
reclaimed.
Please let me know if anyone has any comments/questions.
>
> Best regards,
> João Alves
>
>
>
>
>
>
>
>
>
>
Hello All,
We have been working on enabling the use memory management interfaces between SWd and NWd, on Hafnium as SPMC. We identified a few points that would benefit from a general discussion/clarification through the Mailing List for this work to progress on time for the first release.
The whole memory has been mapped to a VM representative of the NWd FFA endpoints (in the code can be found as a global variable called 'other_world_vm') in this patch<https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
All the memory is configured as RW memory, to be permissive enough to ease the validation of the sender's original permissions vs the requested sharing permissions.
This approach seems valid given the following assumptions: the 'other_world_vm' is never scheduled, which means that there is no risk of memory leakage; the hypervisor should do the necessary validation of the permissions that a given NWd FFA endpoint has on a given memory region, before forwarding the handling of the operation to the SPMC.
The first point to bring to discussion is regarding the RW configuration.
Is this really the best configuration we would expect for memory to be shared from NWd to the SWd? Or is this excluding any use-case we are not aware of (e.g. at this point we are restricting instruction permissions to NX)?
Whilst testing the memory management interfaces, I noticed that if the instruction access on the memory region descriptor (in the spec has been described in section 5.12, and in the code defined as 'struct ffa_memory_region') is set as 'not specified' for the Lend and Donate operations, the receiver would acquire permission to execute the memory region. This is due to the implementation of the function 'ffa_memory_permissions_to_mode<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…><https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>', which sets the memory mode to MM_MODE_X for case instruction access FFA_INSTRUCTION_ACCESS_NOT_SPECIFIED. My understanding is that in such case MM_MODE_X shouldn't be set. Please check this change<https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
@Andrew Walbran<mailto:qwandor@google.com>, is the change I provided correct? Or is there a reason for the current implementation of 'ffa_memory_permissions_to_mode'?
I also noticed that after a memory reclaim, the sender/owner of a memory region acquires RW and X permissions, even if these were not part of the original permissions when sending the memory region. This is due to definition of the variable 'memory_to_attributes' in the function 'ffa_memory_reclaim<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>':
uint32_t memory_to_attributes = MM_MODE_R | MM_MODE_W | MM_MODE_X;
Down the call stack, the value of this variable is used to update the variable 'to_mode' in function 'ffa_retrieve_check_transition<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>'. The value of variable 'to_mode' is later passed to 'ffa_region_group_identity_map'<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…> to update the owner's memory mapping. My understanding is that the sender's memory mode should be reset to the mode it had prior to the original memory send operation, and that this is going to be a problem for memory management operations (even between between VMs and between SPs). I have seen this behavior on my set, however I would like to validate my analysis.
@Andrew Walbran<mailto:qwandor@google.com> would you be able to provide any comments on this?
Please let me know if anyone has any comments/questions.
Best regards,
João Alves
Hi,
This is to notify that we are planning to target the Trusted Firmware-A 2.4 release during the second week of November 2020 as part of the regular 6 month cadence. The aim is to consolidate all TF-A work since the 2.3 release. As part of this, a release candidate tag will be created and release activities will commence from Friday 30th Oct 2020 (code freeze date). Essentially we will not merge any major enhancements from this date until the release is made. Please ensure any Pull Requests (PR’s) desired to make the 2.4 release are submitted in good time, to be completed by Tuesday 27th Oct 2020. Any major enhancement PR’s still open after that date will not be merged until after the release
Please note, there is no guarantee that all patches under review will be merged before the code freeze date. Any patches that don’t make the code freeze date will complete their review and merge post the release date.
Thanks
Manish Badarkhe
Hi Varun,
Memory footprint will be a function of three build-time constants:
MAX_CPUS, MAX_VMS and HEAP_PAGES. They're defined in the build rule of each
target platform, e.g. for RPi4 it is here:
https://git.trustedfirmware.org/hafnium/project/reference.git/tree/BUILD.gn…
Hope this helps,
David
On Tue, Sep 15, 2020 at 1:08 AM Varun Wadekar via Hafnium <
hafnium(a)lists.trustedfirmware.org> wrote:
> Hi,
>
> We are trying to figure out the memory footprint (.text + .data) for the
> Hafnium image. I couldn't find a place where the number of CPUs are
> specified for a platform. I assume that the footprint will increase as we
> introduce more CPUs in the mix.
>
> Can someone please help me figure this out?
>
> Thanks.
> --
> Hafnium mailing list
> Hafnium(a)lists.trustedfirmware.org
> https://lists.trustedfirmware.org/mailman/listinfo/hafnium
>
Hi,
We are trying to figure out the memory footprint (.text + .data) for the Hafnium image. I couldn't find a place where the number of CPUs are specified for a platform. I assume that the footprint will increase as we introduce more CPUs in the mix.
Can someone please help me figure this out?
Thanks.
