Hello, I've run into a few issues while implementing fragmentation for memory sharing based on the FF-A 1.0 spec, which I think need clarifying or fixing.
1. From section 13.2.2.3 points 2-6, it sounds like the 'sender' in the case of a FFA_MEM_RETRIEVE_RESP_32 from the SPM to the non-secure hypervisor is the SPM. Am I correct in understanding that this means that the sender id for FFA_MEM_FRAG_TX and FFA_MEM_FRAG_RX should thus always be the ID of the SPM, when the hypervisor is retrieving a memory region from the SPM for the purposes of a reclaim operation from a normal world VM?
2. When a normal world VM tries to share memory with a secure partition via FFA_MEM_SHARE, it may be that the buffer between the hypervisor and the SPM is busy because another VM is also sharing memory or doing something else that uses the buffer, on a different physical CPU. This could happen either for the initial fragment sent via FFA_MEM_SHARE or for a subsequent fragment sent with FFA_MEM_FRAG_TX. The spec currently says in section 12.3.1.2, point 13.2 that the hypervisor must return ABORTED in the FFA_MEM_SHARE case, and doesn't allow any relevant error codes in the description of FFA_MEM_FRAG_TX in section 13.2.2.5, though it does mention ABORTED in section 13.2.2.3 point 8.
However, the buffer being busy need not mean that the whole transaction is aborted; it should be possible for the sender to try again after a short time, when hopefully the buffer is available again. So it would make more sense for the hypervisor to return an FFA_BUSY error in this case, as it does in other cases where a buffer is currently unavailable but the caller can try again.
Hi Andrew,
CIL...
On Tue, May 19, 2020 at 01:28:43PM +0100, Andrew Walbran wrote:
Hello, I've run into a few issues while implementing fragmentation for memory sharing based on the FF-A 1.0 spec, which I think need clarifying or fixing.
- From section 13.2.2.3 points 2-6, it sounds like the 'sender' in the
Did you mean Section 12.2.2.3 in the publicly available spec [1]?
case of a FFA_MEM_RETRIEVE_RESP_32 from the SPM to the non-secure hypervisor is the SPM. Am I correct in understanding that this means that the sender id for FFA_MEM_FRAG_TX and FFA_MEM_FRAG_RX should thus always be the ID of the SPM, when the hypervisor is retrieving a memory region from the SPM for the purposes of a reclaim operation from a normal world VM?
The Sender ID is really meant to be used when the Hypervisor forwards the TX and RX operations to the SPM on behalf of a VM.
In all other cases, the Sender ID field in a RX/TX operation MBZ. If this makes sense then I could update the spec.
- When a normal world VM tries to share memory with a secure partition
via FFA_MEM_SHARE, it may be that the buffer between the hypervisor and the SPM is busy because another VM is also sharing memory or doing something� else that uses the buffer, on a different physical CPU. This could happen either for the initial fragment sent via FFA_MEM_SHARE or for a subsequent fragment sent with FFA_MEM_FRAG_TX. The spec currently says in section 12.3.1.2, point 13.2 that the hypervisor must return ABORTED in the FFA_MEM_SHARE case, and doesn't allow any relevant error codes in the description of FFA_MEM_FRAG_TX in section 13.2.2.5, though it does mention ABORTED in section 13.2.2.3 point 8. However, the buffer being busy need not mean that the whole transaction is aborted; it should be possible for the sender to try again after a short time, when hopefully the buffer is available again. So it would make more sense for the hypervisor to return an FFA_BUSY error in this case, as it does in other cases where a buffer is currently unavailable but the caller can try again.
In the case of MEM_SHARE, either the first fragment or the entire descriptor is passed. It makes sense for the callee to just abort. This is no different from asking the caller to RETRY. The caller will retry with the same information in any case.
For FRAG_TX and RX, the ABORTED error code is meant to be used for critical errors e.g. Hypervisor runs out of memory.
For the case you describe, the Hypervisor could simply replay the FRAG_RX call before the FRAG_TX that failed. Effectively, this is a request to resend the last frag.
What do you reckon?
cheers, Achin
[1] https://developer.arm.com/docs/den0077/a IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Tue, 19 May 2020 at 15:43, Achin Gupta achin.gupta@arm.com wrote:
Hi Andrew,
CIL...
On Tue, May 19, 2020 at 01:28:43PM +0100, Andrew Walbran wrote:
Hello, I've run into a few issues while implementing fragmentation for memory sharing based on the FF-A 1.0 spec, which I think need clarifying or fixing.
- From section 13.2.2.3 points 2-6, it sounds like the 'sender' in
the
Did you mean Section 12.2.2.3 in the publicly available spec [1]?
Yes, I was looking at a different version.
case of a FFA_MEM_RETRIEVE_RESP_32 from the SPM to the non-secure
hypervisor is the SPM. Am I correct in understanding that this means that the sender id for FFA_MEM_FRAG_TX and FFA_MEM_FRAG_RX should thus always be the ID of the SPM, when the hypervisor is retrieving a
memory
region from the SPM for the purposes of a reclaim operation from a normal world VM?
The Sender ID is really meant to be used when the Hypervisor forwards the TX and RX operations to the SPM on behalf of a VM.
In all other cases, the Sender ID field in a RX/TX operation MBZ. If this makes sense then I could update the spec.
I think that makes sense. Arve, do you agree?
It would be good to clarify this in the spec, as it is currently not very clear.
- When a normal world VM tries to share memory with a secure partition
via FFA_MEM_SHARE, it may be that the buffer between the hypervisor
and
the SPM is busy because another VM is also sharing memory or doing something� else that uses the buffer, on a different physical CPU.
This
could happen either for the initial fragment sent via FFA_MEM_SHARE or for a subsequent fragment sent with FFA_MEM_FRAG_TX. The spec
currently
says in section 12.3.1.2, point 13.2 that the hypervisor must return ABORTED in the FFA_MEM_SHARE case, and doesn't allow any relevant
error
codes in the description of FFA_MEM_FRAG_TX in section 13.2.2.5,
though
it does mention ABORTED in section 13.2.2.3 point 8. However, the buffer being busy need not mean that the whole
transaction
is aborted; it should be possible for the sender to try again after a short time, when hopefully the buffer is available again. So it would make more sense for the hypervisor to return an FFA_BUSY error in this case, as it does in other cases where a buffer is currently
unavailable
but the caller can try again.
In the case of MEM_SHARE, either the first fragment or the entire descriptor is passed. It makes sense for the callee to just abort. This is no different from asking the caller to RETRY. The caller will retry with the same information in any case.
I guess so, but it seems more consistent to use FFA_BUSY, as that is what FFA_MSG_SEND would return in the equivalent situation, and it makes the reason for the temporary failure clear.
For FRAG_TX and RX, the ABORTED error code is meant to be used for critical
errors e.g. Hypervisor runs out of memory.
If the hypervisor runs out of memory, then it should return FFA_NO_MEMORY.
For the case you describe, the Hypervisor could simply replay the FRAG_RX
call before the FRAG_TX that failed. Effectively, this is a request to resend the last frag.
What do you reckon?
I guess that would also be possible, but it would be more difficult for me to implement as in my current design I don't have access to the information about what the current offset at the point where I take the lock and check whether the buffer is available. I still think it would be more consistent to use FFA_BUSY though.
On Wed, May 20, 2020 at 4:31 AM Andrew Walbran qwandor@google.com wrote:
On Tue, 19 May 2020 at 15:43, Achin Gupta achin.gupta@arm.com wrote:
Hi Andrew,
CIL...
On Tue, May 19, 2020 at 01:28:43PM +0100, Andrew Walbran wrote:
Hello, I've run into a few issues while implementing fragmentation for
memory
sharing based on the FF-A 1.0 spec, which I think need clarifying or fixing.
- From section 13.2.2.3 points 2-6, it sounds like the 'sender' in
the
Did you mean Section 12.2.2.3 in the publicly available spec [1]?
Yes, I was looking at a different version.
case of a FFA_MEM_RETRIEVE_RESP_32 from the SPM to the non-secure
hypervisor is the SPM. Am I correct in understanding that this means that the sender id for FFA_MEM_FRAG_TX and FFA_MEM_FRAG_RX should
thus
always be the ID of the SPM, when the hypervisor is retrieving a
memory
region from the SPM for the purposes of a reclaim operation from a normal world VM?
The Sender ID is really meant to be used when the Hypervisor forwards the TX and RX operations to the SPM on behalf of a VM.
In all other cases, the Sender ID field in a RX/TX operation MBZ. If this makes sense then I could update the spec.
I think that makes sense. Arve, do you agree?
It would be good to clarify this in the spec, as it is currently not very clear.
The spec already says sender-id mbz if the client is not a hypervisor, but to the SPM, that is not easily known. For frag-rx calls it would be easier if sender-if was retriever-id (the current client) and always passed. For frag-rx return values, i'm not sure it has any use. Similar for frag-tx.
hafnium@lists.trustedfirmware.org
-
Achin Gupta -
Andrew Walbran -
Arve Hjønnevåg