Hi Danny,
Could you please check my comments on [Mailbox design](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/mailbox-hal/) in below?
I’d like to discuss about some high-level designs at the very first time. We can discuss about details later. 😊
1. As designed in Concepts section, secure core has a single dispatcher thread, which is the main thread on the secure core.
AFAIK, there is no such main thread in current TF-M Inter-Process Communication model implementation. Do you mind taking a look at the section "Interaction in thread model" in our [TF-M presentation](https://events.linuxfoundation.org/wp-content/uploads/2017/12/OpenIoT-Summit...
On the other hand, implementing a dedicated thread in TF-M to handling the mailbox events can be a solution. But it might also bring some difficulties in implementation. I'd like to discuss in detail with Transport Layer together.
2. I like the concept of mailbox event. One of the key points is, after the information carried via mailbox arrives in TF-M, how to deliver it to TF-M core. We can have further discussions on the details.
3. PSA Firmware Framework specifies the parameters and return value for PSA Client APIs. Do you think if it is necessary to cover the mailbox message structure containing information of parameters and return value in mailbox design?
Thank you.
Best regards, Hu Ziji
Hi David,
I am glad you could review it. Great to get these inputs from you!
Please see my answers below. Will be happy for any kind of further discussion regarding the details.
1. I mean the main() in the secure core (CM0+). This system boots on the secure core, and main() runs and enables the non-secure core.
The wait loop I described should be in this secure core main() right after enabling the non-secure core.
Let me know if I am missing your point here…
1. Sure. I will be happy to discuss the details.
Basically, the idea is:
* The queue items carry data / pointers to data * After an item is added to the queue, an event is sent to notify about this new queue item (can be optimized by notifying only if this item was added to an empty queue) * SPM checks data validity and might copy data from non-secure memory to secure memory
1. I can go on and add these details. But maybe it is better if we can get some kind of approval to the direction we are taking.
Thanks, Danny
From: David Hu (Arm Technology China) David.Hu@arm.com Sent: Wednesday, February 20, 2019 13:11 To: tf-m@lists.trustedfirmware.org; Danny Shavit Danny.Shavit@arm.com Cc: nd nd@arm.com Subject: Some comments on twincpu mailbox design
Hi Danny,
Could you please check my comments on [Mailbox design](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/mailbox-hal/) in below?
I’d like to discuss about some high-level designs at the very first time. We can discuss about details later. 😊
1. As designed in Concepts section, secure core has a single dispatcher thread, which is the main thread on the secure core.
AFAIK, there is no such main thread in current TF-M Inter-Process Communication model implementation. Do you mind taking a look at the section "Interaction in thread model" in our [TF-M presentation](https://events.linuxfoundation.org/wp-content/uploads/2017/12/OpenIoT-Summit...
On the other hand, implementing a dedicated thread in TF-M to handling the mailbox events can be a solution. But it might also bring some difficulties in implementation. I'd like to discuss in detail with Transport Layer together.
2. I like the concept of mailbox event. One of the key points is, after the information carried via mailbox arrives in TF-M, how to deliver it to TF-M core. We can have further discussions on the details.
3. PSA Firmware Framework specifies the parameters and return value for PSA Client APIs. Do you think if it is necessary to cover the mailbox message structure containing information of parameters and return value in mailbox design?
Thank you.
Best regards, Hu Ziji IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Danny,
Maybe off the topic. If you are interested in more helpful threads in TF-M, please subscribe the mailing list in https://www.trustedfirmware.org/contact/. It won't cost much time, I promise. 😉 I also cc Ashu, Ken and Miklos. So we can receive your email if you just directly reply this email. Hope it can save your time.
Could you please check my inline comments in below? Thank you.
Best regards, Hu Ziji
From: Danny Shavit Danny.Shavit@arm.com Sent: Wednesday, February 20, 2019 10:07 PM To: David Hu (Arm Technology China) David.Hu@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: RE: Some comments on twincpu mailbox design
Hi David,
I am glad you could review it. Great to get these inputs from you!
Please see my answers below. Will be happy for any kind of further discussion regarding the details.
1. I mean the main() in the secure core (CM0+). This system boots on the secure core, and main() runs and enables the non-secure core. The wait loop I described should be in this secure core main() right after enabling the non-secure core. Let me know if I am missing your point here… [Ziji]: That is a tricky place here. main() is just an initialization function/routine, instead of a real thread. In current single Armv-8M implementation, main() never returns after jumping to non-secure core. Secure Partitions (Secure threads) are scheduled by TF-M core when NSPE asserts a service event. In dual-core scenario, in theory, we can hack the main() function into a special thread to loop for the mailbox message. Some workarounds to TF-M core are required. However, it will cost a lot of effort to verify if the workaround will break current TF-M single Armv-8M implementation, since such a hack doesn't quite fit in current TF-M design. Furthermore, the current main() routine itself is a temporary implementation in TF-M Inter-Process Communication (Thread) model. There are other technical issues if we make main() function as a thread. For example, it will become very complicated to implement such a privileged thread in PSA Isolation Level 3. It may also bring in large latency. In my very own opinion, *main thread* looks like a more straightforward solution at current stage. But it concerns me that it might conflict with development of other modules in TF-M. May I know if you considered putting the mailbox event handling inside the IPC interrupt handler?
2. Sure. I will be happy to discuss the details. Basically, the idea is: • The queue items carry data / pointers to data • After an item is added to the queue, an event is sent to notify about this new queue item (can be optimized by notifying only if this item was added to an empty queue) [Ziji] IMO, two more steps can be added here - Fetch the item and data via mailbox - Translate the mailbox specific item/data into generic data format/parameters required by TF-M. • SPM checks data validity and might copy data from non-secure memory to secure memory [Ziji] Basically, I have the similar idea on the rough flow.
3. I can go on and add these details. But maybe it is better if we can get some kind of approval to the direction we are taking. [Ziji] Yes. Sure.
Thanks, Danny
From: David Hu (Arm Technology China) mailto:David.Hu@arm.com Sent: Wednesday, February 20, 2019 13:11 To: mailto:tf-m@lists.trustedfirmware.org; Danny Shavit mailto:Danny.Shavit@arm.com Cc: nd mailto:nd@arm.com Subject: Some comments on twincpu mailbox design
Hi Danny,
Could you please check my comments on [Mailbox design](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/mailbox-hal/) in below? I’d like to discuss about some high-level designs at the very first time. We can discuss about details later. 😊
1. As designed in Concepts section, secure core has a single dispatcher thread, which is the main thread on the secure core. AFAIK, there is no such main thread in current TF-M Inter-Process Communication model implementation. Do you mind taking a look at the section "Interaction in thread model" in our [TF-M presentation](https://events.linuxfoundation.org/wp-content/uploads/2017/12/OpenIoT-Summit... On the other hand, implementing a dedicated thread in TF-M to handling the mailbox events can be a solution. But it might also bring some difficulties in implementation. I'd like to discuss in detail with Transport Layer together.
2. I like the concept of mailbox event. One of the key points is, after the information carried via mailbox arrives in TF-M, how to deliver it to TF-M core. We can have further discussions on the details.
3. PSA Firmware Framework specifies the parameters and return value for PSA Client APIs. Do you think if it is necessary to cover the mailbox message structure containing information of parameters and return value in mailbox design?
Thank you.
Best regards, Hu Ziji
Hi David,
* I am now registered to the trusted-firmware mailing list.
It seems the most "interesting" part is the thread in S which handles the requests from the NS side. Let me just say that I am not familiar as you are with current TF-M design and code, and with future plans, so your inputs are really important! The design page I shared focuses on the dual-core solution. Abstraction for dual-core vs. arm-v8 is another thing we need to address of course. In the dual- core solution, the main() should be able to enable the NS core and then wait for event. I was under the impression that there is no schedular / threads on the S core. If we have a scheduler on the S core, then we can consider the option of creating a dedicated thread for the dispatcher. Putting the mailbox event handling inside the IPC interrupt handler: Isn't it a too long task to do in an handler context?
Thanks, Danny
-----Original Message----- From: David Hu (Arm Technology China) David.Hu@arm.com Sent: Thursday, February 21, 2019 11:43 To: Danny Shavit Danny.Shavit@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com; Ashutosh Singh Ashutosh.Singh@arm.com; Ken Liu (Arm Technology China) Ken.Liu@arm.com; Miklos Balint Miklos.Balint@arm.com Subject: RE: Some comments on twincpu mailbox design
Hi Danny,
Maybe off the topic. If you are interested in more helpful threads in TF-M, please subscribe the mailing list in https://www.trustedfirmware.org/contact/. It won't cost much time, I promise. 😉 I also cc Ashu, Ken and Miklos. So we can receive your email if you just directly reply this email. Hope it can save your time.
Could you please check my inline comments in below? Thank you.
Best regards, Hu Ziji
From: Danny Shavit Danny.Shavit@arm.com Sent: Wednesday, February 20, 2019 10:07 PM To: David Hu (Arm Technology China) David.Hu@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: RE: Some comments on twincpu mailbox design
Hi David,
I am glad you could review it. Great to get these inputs from you!
Please see my answers below. Will be happy for any kind of further discussion regarding the details.
1. I mean the main() in the secure core (CM0+). This system boots on the secure core, and main() runs and enables the non-secure core. The wait loop I described should be in this secure core main() right after enabling the non-secure core. Let me know if I am missing your point here… [Ziji]: That is a tricky place here. main() is just an initialization function/routine, instead of a real thread. In current single Armv-8M implementation, main() never returns after jumping to non-secure core. Secure Partitions (Secure threads) are scheduled by TF-M core when NSPE asserts a service event. In dual-core scenario, in theory, we can hack the main() function into a special thread to loop for the mailbox message. Some workarounds to TF-M core are required. However, it will cost a lot of effort to verify if the workaround will break current TF-M single Armv-8M implementation, since such a hack doesn't quite fit in current TF-M design. Furthermore, the current main() routine itself is a temporary implementation in TF-M Inter-Process Communication (Thread) model. There are other technical issues if we make main() function as a thread. For example, it will become very complicated to implement such a privileged thread in PSA Isolation Level 3. It may also bring in large latency. In my very own opinion, *main thread* looks like a more straightforward solution at current stage. But it concerns me that it might conflict with development of other modules in TF-M. May I know if you considered putting the mailbox event handling inside the IPC interrupt handler?
2. Sure. I will be happy to discuss the details. Basically, the idea is: • The queue items carry data / pointers to data • After an item is added to the queue, an event is sent to notify about this new queue item (can be optimized by notifying only if this item was added to an empty queue) [Ziji] IMO, two more steps can be added here - Fetch the item and data via mailbox - Translate the mailbox specific item/data into generic data format/parameters required by TF-M. • SPM checks data validity and might copy data from non-secure memory to secure memory [Ziji] Basically, I have the similar idea on the rough flow.
3. I can go on and add these details. But maybe it is better if we can get some kind of approval to the direction we are taking. [Ziji] Yes. Sure.
Thanks, Danny
From: David Hu (Arm Technology China) mailto:David.Hu@arm.com Sent: Wednesday, February 20, 2019 13:11 To: mailto:tf-m@lists.trustedfirmware.org; Danny Shavit mailto:Danny.Shavit@arm.com Cc: nd mailto:nd@arm.com Subject: Some comments on twincpu mailbox design
Hi Danny,
Could you please check my comments on [Mailbox design](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/mailbox-hal/) in below? I’d like to discuss about some high-level designs at the very first time. We can discuss about details later. 😊
1. As designed in Concepts section, secure core has a single dispatcher thread, which is the main thread on the secure core. AFAIK, there is no such main thread in current TF-M Inter-Process Communication model implementation. Do you mind taking a look at the section "Interaction in thread model" in our [TF-M presentation](https://events.linuxfoundation.org/wp-content/uploads/2017/12/OpenIoT-Summit... On the other hand, implementing a dedicated thread in TF-M to handling the mailbox events can be a solution. But it might also bring some difficulties in implementation. I'd like to discuss in detail with Transport Layer together.
2. I like the concept of mailbox event. One of the key points is, after the information carried via mailbox arrives in TF-M, how to deliver it to TF-M core. We can have further discussions on the details.
3. PSA Firmware Framework specifies the parameters and return value for PSA Client APIs. Do you think if it is necessary to cover the mailbox message structure containing information of parameters and return value in mailbox design?
Thank you.
Best regards, Hu Ziji IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Danny,
IMHO, there could be multiple design options of mailbox, such as a dedicate thread and processing in interrupt handler. Each option might not be perfect. Do you mind summarizing some options thus we can discuss in detail later in workshop?
Could you please also check my inline comments in below? Thank you.
Best regards, Hu Ziji
-----Original Message----- From: Danny Shavit Danny.Shavit@arm.com Sent: Thursday, February 21, 2019 7:45 PM To: David Hu (Arm Technology China) David.Hu@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com; Ashutosh Singh Ashutosh.Singh@arm.com; Ken Liu (Arm Technology China) Ken.Liu@arm.com; Miklos Balint Miklos.Balint@arm.com Subject: RE: Some comments on twincpu mailbox design
Hi David,
* I am now registered to the trusted-firmware mailing list.
It seems the most "interesting" part is the thread in S which handles the requests from the NS side. Let me just say that I am not familiar as you are with current TF-M design and code, and with future plans, so your inputs are really important! The design page I shared focuses on the dual-core solution. Abstraction for dual-core vs. arm-v8 is another thing we need to address of course. [David]: From the perspective of TF-M, dual-core mailbox and single Armv8-M veneer are just two different interfaces to NSPE. It is transparent to TF-M core and Secure Partitions. The dual-core implementation will eventually be merged into TF-M mainstream, instead of being held as an independent code base. Ideally, TF-M will smoothly co-work with popular RTOS, including mbed of course, on Arm cores, no matter in dual-core system or on single Armv8-M, or in any common scenario. Thus it is essential to make sure that the dual-core design and other TF-M modules development will not disturb each other. It is, actually, a bit difficult since TF-M is not implemented as a normal OS or RTOS. As a secured firmware, TF-M has more limitations and special designs. But it won't be a blocking issue. Ken and Miklos will help us understand more details in current TF-M design and future plan.
In the dual- core solution, the main() should be able to enable the NS core and then wait for event. I was under the impression that there is no schedular / threads on the S core. If we have a scheduler on the S core, then we can consider the option of creating a dedicated thread for the dispatcher. [David]: Yes. A dedicated thread is definitely an option. It should own privileged permissions to access memory regions and devices. But as discussed in above, it is a little different to create a secure thread (Secure Partition) in TF-M, compared with that in normal RTOS, according to TF-M designers. Some security requirements might make it complicated. Besides, the inter-activities between core and SPs might bring in longer latency. But we can leave the details in workshop.
Putting the mailbox event handling inside the IPC interrupt handler: Isn't it a too long task to do in an handler context? [David]: Yes. It is a little abnormal to do those things in an interrupt handler. But it can work if IPC interrupt is put in an proper priority level. The scheduler in TF-M Thread model is triggered by request event, instead of tick. So the scheduling won't be impacted, in theory. One of the disadvantages is that some time-sensitive Secure Services might be delayed. But it is quite implementation specific.
Thanks, Danny
-----Original Message----- From: David Hu (Arm Technology China) David.Hu@arm.com Sent: Thursday, February 21, 2019 11:43 To: Danny Shavit Danny.Shavit@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com; Ashutosh Singh Ashutosh.Singh@arm.com; Ken Liu (Arm Technology China) Ken.Liu@arm.com; Miklos Balint Miklos.Balint@arm.com Subject: RE: Some comments on twincpu mailbox design
Hi Danny,
Maybe off the topic. If you are interested in more helpful threads in TF-M, please subscribe the mailing list in https://www.trustedfirmware.org/contact/. It won't cost much time, I promise. 😉 I also cc Ashu, Ken and Miklos. So we can receive your email if you just directly reply this email. Hope it can save your time.
Could you please check my inline comments in below? Thank you.
Best regards, Hu Ziji
From: Danny Shavit Danny.Shavit@arm.com Sent: Wednesday, February 20, 2019 10:07 PM To: David Hu (Arm Technology China) David.Hu@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: RE: Some comments on twincpu mailbox design
Hi David,
I am glad you could review it. Great to get these inputs from you!
Please see my answers below. Will be happy for any kind of further discussion regarding the details.
1. I mean the main() in the secure core (CM0+). This system boots on the secure core, and main() runs and enables the non-secure core. The wait loop I described should be in this secure core main() right after enabling the non-secure core. Let me know if I am missing your point here… [Ziji]: That is a tricky place here. main() is just an initialization function/routine, instead of a real thread. In current single Armv-8M implementation, main() never returns after jumping to non-secure core. Secure Partitions (Secure threads) are scheduled by TF-M core when NSPE asserts a service event. In dual-core scenario, in theory, we can hack the main() function into a special thread to loop for the mailbox message. Some workarounds to TF-M core are required. However, it will cost a lot of effort to verify if the workaround will break current TF-M single Armv-8M implementation, since such a hack doesn't quite fit in current TF-M design. Furthermore, the current main() routine itself is a temporary implementation in TF-M Inter-Process Communication (Thread) model. There are other technical issues if we make main() function as a thread. For example, it will become very complicated to implement such a privileged thread in PSA Isolation Level 3. It may also bring in large latency. In my very own opinion, *main thread* looks like a more straightforward solution at current stage. But it concerns me that it might conflict with development of other modules in TF-M. May I know if you considered putting the mailbox event handling inside the IPC interrupt handler?
2. Sure. I will be happy to discuss the details. Basically, the idea is: • The queue items carry data / pointers to data • After an item is added to the queue, an event is sent to notify about this new queue item (can be optimized by notifying only if this item was added to an empty queue) [Ziji] IMO, two more steps can be added here - Fetch the item and data via mailbox - Translate the mailbox specific item/data into generic data format/parameters required by TF-M. • SPM checks data validity and might copy data from non-secure memory to secure memory [Ziji] Basically, I have the similar idea on the rough flow.
3. I can go on and add these details. But maybe it is better if we can get some kind of approval to the direction we are taking. [Ziji] Yes. Sure.
Thanks, Danny
From: David Hu (Arm Technology China) mailto:David.Hu@arm.com Sent: Wednesday, February 20, 2019 13:11 To: mailto:tf-m@lists.trustedfirmware.org; Danny Shavit mailto:Danny.Shavit@arm.com Cc: nd mailto:nd@arm.com Subject: Some comments on twincpu mailbox design
Hi Danny,
Could you please check my comments on [Mailbox design](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/mailbox-hal/) in below? I’d like to discuss about some high-level designs at the very first time. We can discuss about details later. 😊
1. As designed in Concepts section, secure core has a single dispatcher thread, which is the main thread on the secure core. AFAIK, there is no such main thread in current TF-M Inter-Process Communication model implementation. Do you mind taking a look at the section "Interaction in thread model" in our [TF-M presentation](https://events.linuxfoundation.org/wp-content/uploads/2017/12/OpenIoT-Summit... On the other hand, implementing a dedicated thread in TF-M to handling the mailbox events can be a solution. But it might also bring some difficulties in implementation. I'd like to discuss in detail with Transport Layer together.
2. I like the concept of mailbox event. One of the key points is, after the information carried via mailbox arrives in TF-M, how to deliver it to TF-M core. We can have further discussions on the details.
3. PSA Firmware Framework specifies the parameters and return value for PSA Client APIs. Do you think if it is necessary to cover the mailbox message structure containing information of parameters and return value in mailbox design?
Thank you.
Best regards, Hu Ziji
tf-m@lists.trustedfirmware.org
-
Danny Shavit -
David Hu (Arm Technology China)