Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It's because I'm using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT - number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD - defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST - specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it's important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it's just the waste of resources in such case. But it can bring a benefit that it's possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
Hi Roman,
Much appreciated. Your suggestions make lots of sense to me. Can I suggest continuing to discuss the improvements in detail in the mailing list?
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1. But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I’m glad to convert this blocking limitation into a non-blocking information message, and let users decide.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible. It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I’d suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible. Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests. Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure.
Best regards, Hu Ziji
From: Roman.Mazurak--- via TF-M tf-m@lists.trustedfirmware.org Date: Wednesday, November 15, 2023 at 04:09 To: tf-m@lists.trustedfirmware.org tf-m@lists.trustedfirmware.org Subject: [TF-M] Reduce dependency of NS Mailbox structure on non-secure configurations Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It’s because I’m using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT – number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD – defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST – specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it’s important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it’s just the waste of resources in such case. But it can bring a benefit that it’s possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
Hi David,
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1.
But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I'm glad to convert this blocking limitation into a non-blocking information message, and let users decide. Good to hear it.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible.
It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I'd suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number. I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member. Than it will be good to cache this number in mailbox partition to avoid changing this number on the fly and creating an unwanted backdoor.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible.
Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests.
Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure. Good point. I think this problem will be solved if the previous one will be solved by splitting shared and non-secure data. I mean that this data will not affect secure side.
Regards, Roman.
From: David Hu David.Hu@arm.com Sent: Thursday, November 16, 2023 09:52 To: Mazurak Roman (CSS ICW SW FW 3) Roman.Mazurak@infineon.com; tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
Much appreciated. Your suggestions make lots of sense to me. Can I suggest continuing to discuss the improvements in detail in the mailing list?
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1. But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I'm glad to convert this blocking limitation into a non-blocking information message, and let users decide.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible. It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I'd suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible. Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests. Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure.
Best regards, Hu Ziji
From: Roman.Mazurak--- via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Date: Wednesday, November 15, 2023 at 04:09 To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Reduce dependency of NS Mailbox structure on non-secure configurations Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It's because I'm using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT - number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD - defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST - specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it's important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it's just the waste of resources in such case. But it can bring a benefit that it's possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
Hi Roman,
I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member.
Any particular reason to fix this value as the first member? NS and S can also be incompatible, if any other fields change in the NS mailbox queue structure definition. How about adding a hash into ns_mailbox_queue_t as the first number, to check if NS and S are using the compatible NS mailbox definitions? Therefore, it is more flexible to organize members in ns_mailbox_queue_t.
Best regards, Hu Ziji
From: Roman.Mazurak@infineon.com Roman.Mazurak@infineon.com Date: Thursday, November 16, 2023 at 21:08 To: David Hu David.Hu@arm.com, tf-m@lists.trustedfirmware.org tf-m@lists.trustedfirmware.org Subject: RE: Reduce dependency of NS Mailbox structure on non-secure configurations Hi David,
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1.
But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I’m glad to convert this blocking limitation into a non-blocking information message, and let users decide. Good to hear it.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible.
It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I’d suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number. I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member. Than it will be good to cache this number in mailbox partition to avoid changing this number on the fly and creating an unwanted backdoor.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible.
Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests.
Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure. Good point. I think this problem will be solved if the previous one will be solved by splitting shared and non-secure data. I mean that this data will not affect secure side.
Regards, Roman.
From: David Hu David.Hu@arm.com Sent: Thursday, November 16, 2023 09:52 To: Mazurak Roman (CSS ICW SW FW 3) Roman.Mazurak@infineon.com; tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
Much appreciated. Your suggestions make lots of sense to me. Can I suggest continuing to discuss the improvements in detail in the mailing list?
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1. But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I’m glad to convert this blocking limitation into a non-blocking information message, and let users decide.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible. It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I’d suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible. Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests. Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure.
Best regards, Hu Ziji
From: Roman.Mazurak--- via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Date: Wednesday, November 15, 2023 at 04:09 To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Reduce dependency of NS Mailbox structure on non-secure configurations Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It’s because I’m using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT – number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD – defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST – specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it’s important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it’s just the waste of resources in such case. But it can bring a benefit that it’s possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
Hi David,
I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member.
Any particular reason to fix this value as the first member? NS and S can also be incompatible, if any other fields change in the NS mailbox queue structure definition.
Probably "first member" is too strong constrain. We can replace this condition by "put before queue member". So, we can estimate number of NS slots on runtime during initialization without need to agree this number on build time.
How about adding a hash into ns_mailbox_queue_t as the first number, to check if NS and S are using the compatible NS mailbox definitions? Therefore, it is more flexible to organize members in ns_mailbox_queue_t.
What kind of hash do you mean?
Best regards, Roman.
From: David Hu David.Hu@arm.com Sent: Monday, November 20, 2023 08:11 To: Mazurak Roman (CSS ICW SW FW 3) Roman.Mazurak@infineon.com; tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member.
Any particular reason to fix this value as the first member? NS and S can also be incompatible, if any other fields change in the NS mailbox queue structure definition. How about adding a hash into ns_mailbox_queue_t as the first number, to check if NS and S are using the compatible NS mailbox definitions? Therefore, it is more flexible to organize members in ns_mailbox_queue_t.
Best regards, Hu Ziji
From: Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com> Date: Thursday, November 16, 2023 at 21:08 To: David Hu <David.Hu@arm.commailto:David.Hu@arm.com>, tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Reduce dependency of NS Mailbox structure on non-secure configurations Hi David,
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1.
But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I'm glad to convert this blocking limitation into a non-blocking information message, and let users decide. Good to hear it.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible.
It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I'd suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number. I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member. Than it will be good to cache this number in mailbox partition to avoid changing this number on the fly and creating an unwanted backdoor.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible.
Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests.
Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure. Good point. I think this problem will be solved if the previous one will be solved by splitting shared and non-secure data. I mean that this data will not affect secure side.
Regards, Roman.
From: David Hu <David.Hu@arm.commailto:David.Hu@arm.com> Sent: Thursday, November 16, 2023 09:52 To: Mazurak Roman (CSS ICW SW FW 3) <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
Much appreciated. Your suggestions make lots of sense to me. Can I suggest continuing to discuss the improvements in detail in the mailing list?
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1. But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I'm glad to convert this blocking limitation into a non-blocking information message, and let users decide.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible. It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I'd suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible. Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests. Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure.
Best regards, Hu Ziji
From: Roman.Mazurak--- via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Date: Wednesday, November 15, 2023 at 04:09 To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Reduce dependency of NS Mailbox structure on non-secure configurations Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It's because I'm using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT - number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD - defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST - specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it's important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it's just the waste of resources in such case. But it can bring a benefit that it's possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
Hi Roman,
What kind of hash do you mean?
This patch https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/25337 shows my idea. Add a hash value as the version of mailbox definitions. Enable TF-M SPE to check if NS mailbox is using the same mailbox definition. Therefore, it is more flexible to organize the other members in NS mailbox queue.
Best regards, Hu Ziji
From: Roman.Mazurak@infineon.com Roman.Mazurak@infineon.com Date: Monday, December 4, 2023 at 22:22 To: David Hu David.Hu@arm.com, tf-m@lists.trustedfirmware.org tf-m@lists.trustedfirmware.org Subject: RE: Reduce dependency of NS Mailbox structure on non-secure configurations Hi David,
I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member.
Any particular reason to fix this value as the first member? NS and S can also be incompatible, if any other fields change in the NS mailbox queue structure definition.
Probably “first member” is too strong constrain. We can replace this condition by “put before queue member”. So, we can estimate number of NS slots on runtime during initialization without need to agree this number on build time.
How about adding a hash into ns_mailbox_queue_t as the first number, to check if NS and S are using the compatible NS mailbox definitions? Therefore, it is more flexible to organize members in ns_mailbox_queue_t.
What kind of hash do you mean?
Best regards, Roman.
From: David Hu David.Hu@arm.com Sent: Monday, November 20, 2023 08:11 To: Mazurak Roman (CSS ICW SW FW 3) Roman.Mazurak@infineon.com; tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member.
Any particular reason to fix this value as the first member? NS and S can also be incompatible, if any other fields change in the NS mailbox queue structure definition. How about adding a hash into ns_mailbox_queue_t as the first number, to check if NS and S are using the compatible NS mailbox definitions? Therefore, it is more flexible to organize members in ns_mailbox_queue_t.
Best regards, Hu Ziji
From: Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com> Date: Thursday, November 16, 2023 at 21:08 To: David Hu <David.Hu@arm.commailto:David.Hu@arm.com>, tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Reduce dependency of NS Mailbox structure on non-secure configurations Hi David,
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1.
But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I’m glad to convert this blocking limitation into a non-blocking information message, and let users decide. Good to hear it.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible.
It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I’d suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number. I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member. Than it will be good to cache this number in mailbox partition to avoid changing this number on the fly and creating an unwanted backdoor.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible.
Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests.
Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure. Good point. I think this problem will be solved if the previous one will be solved by splitting shared and non-secure data. I mean that this data will not affect secure side.
Regards, Roman.
From: David Hu <David.Hu@arm.commailto:David.Hu@arm.com> Sent: Thursday, November 16, 2023 09:52 To: Mazurak Roman (CSS ICW SW FW 3) <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
Much appreciated. Your suggestions make lots of sense to me. Can I suggest continuing to discuss the improvements in detail in the mailing list?
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1. But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I’m glad to convert this blocking limitation into a non-blocking information message, and let users decide.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible. It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I’d suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible. Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests. Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure.
Best regards, Hu Ziji
From: Roman.Mazurak--- via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Date: Wednesday, November 15, 2023 at 04:09 To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Reduce dependency of NS Mailbox structure on non-secure configurations Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It’s because I’m using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT – number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD – defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST – specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it’s important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it’s just the waste of resources in such case. But it can bring a benefit that it’s possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
Hi David,
This patch https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/25337 shows my idea. Add a hash value as the version of mailbox definitions. Enable TF-M SPE to check if NS mailbox is using the same mailbox definition.
Maybe it worse to refactor mailbox queue to separate shared data from data used by non-secure side in the first place. Than it will be easier to see which part must be hashed.
Best regards, Roman.
From: David Hu David.Hu@arm.com Sent: Tuesday, December 5, 2023 08:40 To: Mazurak Roman (CSS ICW SW FW 3) Roman.Mazurak@infineon.com; tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
What kind of hash do you mean?
This patch https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/25337 shows my idea. Add a hash value as the version of mailbox definitions. Enable TF-M SPE to check if NS mailbox is using the same mailbox definition. Therefore, it is more flexible to organize the other members in NS mailbox queue.
Best regards, Hu Ziji
From: Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com> Date: Monday, December 4, 2023 at 22:22 To: David Hu <David.Hu@arm.commailto:David.Hu@arm.com>, tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Reduce dependency of NS Mailbox structure on non-secure configurations Hi David,
I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member.
Any particular reason to fix this value as the first member? NS and S can also be incompatible, if any other fields change in the NS mailbox queue structure definition.
Probably "first member" is too strong constrain. We can replace this condition by "put before queue member". So, we can estimate number of NS slots on runtime during initialization without need to agree this number on build time.
How about adding a hash into ns_mailbox_queue_t as the first number, to check if NS and S are using the compatible NS mailbox definitions? Therefore, it is more flexible to organize members in ns_mailbox_queue_t.
What kind of hash do you mean?
Best regards, Roman.
From: David Hu <David.Hu@arm.commailto:David.Hu@arm.com> Sent: Monday, November 20, 2023 08:11 To: Mazurak Roman (CSS ICW SW FW 3) <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member.
Any particular reason to fix this value as the first member? NS and S can also be incompatible, if any other fields change in the NS mailbox queue structure definition. How about adding a hash into ns_mailbox_queue_t as the first number, to check if NS and S are using the compatible NS mailbox definitions? Therefore, it is more flexible to organize members in ns_mailbox_queue_t.
Best regards, Hu Ziji
From: Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com> Date: Thursday, November 16, 2023 at 21:08 To: David Hu <David.Hu@arm.commailto:David.Hu@arm.com>, tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Reduce dependency of NS Mailbox structure on non-secure configurations Hi David,
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1.
But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I'm glad to convert this blocking limitation into a non-blocking information message, and let users decide. Good to hear it.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible.
It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I'd suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number. I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member. Than it will be good to cache this number in mailbox partition to avoid changing this number on the fly and creating an unwanted backdoor.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible.
Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests.
Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure. Good point. I think this problem will be solved if the previous one will be solved by splitting shared and non-secure data. I mean that this data will not affect secure side.
Regards, Roman.
From: David Hu <David.Hu@arm.commailto:David.Hu@arm.com> Sent: Thursday, November 16, 2023 09:52 To: Mazurak Roman (CSS ICW SW FW 3) <Roman.Mazurak@infineon.commailto:Roman.Mazurak@infineon.com>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
Much appreciated. Your suggestions make lots of sense to me. Can I suggest continuing to discuss the improvements in detail in the mailing list?
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1. But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I'm glad to convert this blocking limitation into a non-blocking information message, and let users decide.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible. It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I'd suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible. Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests. Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure.
Best regards, Hu Ziji
From: Roman.Mazurak--- via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Date: Wednesday, November 15, 2023 at 04:09 To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Reduce dependency of NS Mailbox structure on non-secure configurations Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It's because I'm using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT - number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD - defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST - specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it's important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it's just the waste of resources in such case. But it can bring a benefit that it's possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it's just important to validate that number of slots doesn't not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it's important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
Hi Roman,
For your first suggestion to enable multiple mailbox queue slots in NS bare metal environment, here is the patch: https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/25338
Comments are welcome. 😊
Thanks, Xinyu
From: Roman.Mazurak--- via TF-M tf-m@lists.trustedfirmware.org Date: Thursday, November 16, 2023 at 21:09 To: David Hu David.Hu@arm.com, tf-m@lists.trustedfirmware.org tf-m@lists.trustedfirmware.org Subject: [TF-M] Re: Reduce dependency of NS Mailbox structure on non-secure configurations Hi David,
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1.
But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I’m glad to convert this blocking limitation into a non-blocking information message, and let users decide. Good to hear it.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible.
It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I’d suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number. I think we can solve this problem by refactoring ns_mailbox_queue_t. We can add a number of slots allocated by NS to this structure as a first member. Than it will be good to cache this number in mailbox partition to avoid changing this number on the fly and creating an unwanted backdoor.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible.
Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests.
Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure. Good point. I think this problem will be solved if the previous one will be solved by splitting shared and non-secure data. I mean that this data will not affect secure side.
Regards, Roman.
From: David Hu David.Hu@arm.com Sent: Thursday, November 16, 2023 09:52 To: Mazurak Roman (CSS ICW SW FW 3) Roman.Mazurak@infineon.com; tf-m@lists.trustedfirmware.org Subject: Re: Reduce dependency of NS Mailbox structure on non-secure configurations
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Roman,
Much appreciated. Your suggestions make lots of sense to me. Can I suggest continuing to discuss the improvements in detail in the mailing list?
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive.
This limitation assumes that bare metal environment usually runs in sequence. Since PSA APIs are blocking, it would be waste memory if mailbox slots are more than 1. But it makes sense to me to support integrating a TF-M binary with various NS use cases, which is also the one of enhancement goals of build split. I’m glad to convert this blocking limitation into a non-blocking information message, and let users decide.
More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
It is straightforward to add a step for NS and S to negotiate the mailbox slot number in runtime. TF-M can add a field to set NS mailbox slot number in runtime. It is initialized to S number until NS mailbox requests to update it.
Mailbox initialization is platform specific implementation right now, while an additional initialization step might not be backward compatible. It could break existing NS integration projects on dual-core platforms, which starts submitting mailbox messages after TF-M secure receives NS mailbox addresses. I’d suggest using the LSB bit in the NS in the mailbox queue address value, which is passed from NS to S during mailbox initialization, to notify TF-M S side if NS is going to negotiate the mailbox slot number.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
I personally prefer to the 2nd option to restructure the NS mailbox structures a bit, since it is more extensible. Of course, 2nd option is more complex. We can take a try at first. If 2nd option is too complicated, we can take 1st option.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only,
It makes sense to me to move them to tf-m-tests. Some fields, such as nr_used_slots, can only be updated inside NS mailbox operation. Otherwise, it requires complex synchronization between test threads and weaken the multi-core stress tests. Those fields can be moved to a test specific structure from NS mailbox queue. But NS mailbox implementation has to be updated to access this test specific structure.
Best regards, Hu Ziji
From: Roman.Mazurak--- via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Date: Wednesday, November 15, 2023 at 04:09 To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Reduce dependency of NS Mailbox structure on non-secure configurations Hi all,
I faced the problem that mailbox configuration of secure image differs from one provided for non-secure. It’s because I’m using a split-build but a little bit different that is prepared for v2.0. I think we can minimize dependencies and unexpected impacts between different images when common data structures will have less customization.
Currently we have three options that can change mailbox structures:
1. NUM_MAILBOX_QUEUE_SLOT – number of mailbox slots 2. TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD – defines which NS client implementation is selected. 3. TFM_MULTI_CORE_TEST – specify whether NS multi-core test suite is built.
There is requirement that NUM_MAILBOX_QUEUE_SLOT must be set to 1 if NS bare metal environment is used. But this limitation is excessive. Because it’s important that secure side is not using slots that are not used by non-secure side. It can be possible to use NS bare metal client even if mailbox queue size is more than one, it’s just the waste of resources in such case. But it can bring a benefit that it’s possible to build secure image with default settings (aka 4 mailbox slots) and there is no need to rebuild it if there will be decision to switch from RTOS to bare metal environment which can be useful for some end-user use cases. More flexible update will be to pass number of allocated slots from NS side to TF-M during initialization, it’s just important to validate that number of slots doesn’t not exceed maximum supported by design.
TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD is another problem, because mailbox_reply_t allocates data that are not shared but used by non-secure side only. Which means that it’s important to decide which NS client implementation is going to be used when TF-M is built. I see two different solutions for this problem:
1. Use union to allocate space for both of them and let decide NS client implementation which on to use. Something like this: struct mailbox_reply_t { union { //#ifdef TFM_MULTI_CORE_NS_OS_MAILBOX_THREAD uint8_t *woken_flag; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#else bool is_woken; /* Indicate that owner task has been * or should be woken up, after the * reply is received. */ //#endif }; };
1. Redesign mailbox by separating data that are used by NS client from data that are shared between cores. So, it will be much easier to update non-secure client without touching secure image.
It looks like there is data needed for test suite only (nr_tx and nr_used_slots fields of ns_mailbox_queue_t) defined by TFM_MULTI_CORE_TEST. I think we can allocate it in test suite only, so there will be no need to allocate this data in shared structure and there will not be the case when location of is_full field of ns_mailbox_queue_t accessed by both cores have different location if TFM_MULTI_CORE_TEST configuration is not applied the same way for both secure and non-secure images.
Regards, Roman.
tf-m@lists.trustedfirmware.org
-
David Hu -
Roman.Mazurak＠infineon.com -
Xinyu Zhang