Hi Chris,
Sorry that the format in previous reply is broken. Re-send the reply. Please check my inline comments in below.
On 3/27/2019 1:44 AM, Christopher Brand wrote:
Hi David,
-----Original Message----- From: David Hu (Arm Technology China) David.Hu@arm.com Sent: Tuesday, March 26, 2019 12:06 AM To: tf-m@lists.trustedfirmware.org; Christopher Brand chris.brand@cypress.com Cc: Danny Shavit Danny.Shavit@arm.com; nd nd@arm.com Subject: Re: [RFC] twin cpu bootloader design document
Hi Chris,
Sorry for the delayed reply. Please check my comments in below. Just about small details. Please correct me if I misunderstand anything.
- In my own opinion, it can be possible to use IPC to synchronize mailbox
services between two cores, during initialization.
It certainly *may* be possible, but I'm not convinced that it's *always* possible.
I mean if the synchronization is used to make sure the IPC/mailbox mechanism is ready, then the IPC should be ready when synchronization starts. Any corner case?
The synchronization is only trigged when the mailbox mechanism is readyon mailbox server or client. It means that the IPC module should be also configured.
If the non-secure side is particularly fast, it could attempt to use the IPC mechanism before the secure side has finished setting it up, which could lead to problems.
Yes and the attempt will be blocked until the mailbox/IPC mechanism is ready on peer core. We definitely can implement shared memory in TF-M as a reference. May I suggest that it would be more reasonable to allow diverse mechasims to implement the synchronization? As long as we keep the API(s) and sequence generic enough, platform vendors can implement thier own optimizations.
Based on the above assumption, using IPC to synchronize the two cores ismore generic and convenient than accessing shared memory.
It's platform-specific code, so each platform can use the method that works best there, that could be IPC for some platforms, but I'm not convinced that we can rely on using IPC on platforms where the NS CPU is much faster than the secure CPU.
1. I aggree that IPC might not be always available on every platform while some platform vendors may prefer it. So I wonder if we can keep shared memory as the default choice and implement it as a reference implementation in TF-M. Besides, we can add a comment that platform vendor can implement their own synchronization besides default shared memory mechanism. 2. IMO, it is irrelevant to the speed/performance of two cores. No matter whether NS CPU is faster, slower, or at the same speed, the synchronization for mailbox communication is always essential. Even if secure CPU is more powerful, NS CPU still needs to check whether mailbox is ready in secure side. ( I guess it is a little off the topic.)
If using shared memory to pass status flag, it can be necessary to adjust theaddress of the shared memory occasionally according to the memory assignment in different applications.
The shared memory would only be needed during early initialization. As soon as both sides have passed that point, the memory could be re-assigned for any other use. The memory would always be available for other uses before the NS OS has started running.
I aggree that the shared memory for synchronization can re-used as other init module. Actually I care about more about the details before the shared memory is used, such as 1. The address is defined in secure side and has to be exported to NS side. How does NS side code import the address? 2. The value in shared memory address should be initialized by SPE side before the NSPE side kicks off 3. The shared memory should be put in a privileged region in NSPE. However, it is a little difficult for SPE to know the region assignment in NSPE. Do you think if we can specify more requirements on the shared memory implementations? It will help enhance the security of our developement.
- I'd like to suggest that we shall discuss more about when the booting HAL
APIs are invoked in TF-M. `tfm_core_init()` initializes the TF-M core. Thus in theory, `tfm_core_init()` is irrelevant to the system topology or platform implementations. As a result, IMO, it can be more reasonable to put the HAL APIs outside the `tfm_core_init()`.
This could well be a misunderstanding of the terminology on my part (I'm fairly new here 😊). Does "core" here mean "CPU"? I was reading it as "the core part of TFM", which then would include the parts on both CPUs. I note that tfm_core_init() already makes a call to configure_ns_code(), which is not part of initializing the secure CPU...
Moving this new call from within tfm_core_init() to main() immediately after the call to tfm_core_init() doesn't look like it would make much difference from a performance perspective.
Sorry for misleading you :) In my very own opinion, since the topology concept is brought into TF-M, the init sequence might be sightly adjusted. We can adjust timing of calling APIs if we find a more appropriate place.
- ` tfm_spm_hal_wait_for_ns_cpu_ready()` can be optional.
Right. As the final sentence of the document states, my intent is to allow platforms to not define any or all of the three new functions as needed.
The secure core acts as a server and it is driven by the request from NScore. The secure core actually doesn't have to wait for an explicit signal to know NS is ready.
My concern is that I want to be as flexible as possible regarding platform architecture.
The synchronization can be guaranteed if NS core starts request viamailbox only after secure core is available.
Definitely true for some implementations. I'm not confident that I can say that it's true for all implementations on all platforms.
I guess there are two ways to go - we can include this call now and remove it after we see that there are several twin CPU platforms and none of them need it, or we can exclude it now and add it when and if a platform needs it.
Yep. We can keep it and just comment that it is not a must.
- It can be unnecessary to require calling
`tfm_spm_hal_wait_for_s_cpu_ready()` in NS `main()`. It might be too early to wait in `main()` and may block other initializations which don't rely on mailbox.
We definitely want to make this call as late as possible in NS main(). In the existing TF-M code, I don't think we can assume that the call to osKernelInitialise() from NS main() won't ever use any secure services, so we need to know that the secure code is ready to provide those services. If that's an acceptable constraint then we can move this call later.
Conceptually at least, this call could just record that the secure side is not yet ready and that the check needs to be made again before trying to communicate with it, with corresponding code in the mailbox/IPC code to make the check again (and to wait if needed) in that case. That would be a platform-specific optimization. Note, though, that doing so would mean that any shared memory would be tied up for longer. It really does feel like the kind of decision that has to be made by the system designer. My goal with the design document is to specify a system that is simple and guaranteed to work while allowing the system designer to optimize it for their system as needed.
Similary as above, I'd like to suggest that we can implement a reference design which puts the synchonrization in main() by default. In the meantime, I wonder if we can make the statement about sequence more flexible that different implementations can be allowed. Some requirments and limitiations can be added.
This API can be invoked in mailbox functionalities. The whole NSinitialization can continue, including enabling application threads, until a NS application requests Secure services via mailbox at the very first time.
Again, my concern is that the NS side could boot so fast that it attempts to use the mailbox/IPC mechanism before the secure side has set it up - you don't want the NS side generating an IPC interrupt before the secure side has installed a handler for that interrupt, for example.
Thus the whole dual core design can be more generic since the mailboxworkflow should be identical on diverse platforms. And we can save the time and effort to hack each RTOS initialization. In other words, I wonder if we can make calling `tfm_spm_hal_wait_for_s_cpu_ready()` in NS `main() as an option and allow other implementations.
Again, the document states that none of these new functions need to be provided If they're not needed on the platform.
Actually, in contrast, I prefer to mark this function as mandatory to protect SPE from being disturbed by NSPE. :P
Chris
Thank you.
Best regards, Hu Ziji
Date: Thu, 14 Mar 2019 18:50:56 +0000 From: Christopher Brand chris.brand@cypress.com To: "tf-m@lists.trustedfirmware.org" tf-m@lists.trustedfirmware.org Subject: [TF-M] [RFC] twin cpu bootloader design document Message-ID: <BYAPR06MB5301EBF02F4C0B60A9BB7742FE4B0@BYAPR06MB5301.n amprd06.prod.outlook.com>
Content-Type: text/plain; charset="us-ascii"
Hi,
I've posted a design document for bootloader changes to support twin cpu at https://developer.trustedfirmware.org/w/tf_m/design/twin- cpu/bootloader/ Comments appreciated!
Thanks,
Chris
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.