Hi Devaraj,
Happy to merge it for you, but first please can you add a comment to the gerrit review with details of testing done for the patch?
Kind regards,
Jamie
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Devaraj Ranganna via TF-M
Sent: 10 February 2020 09:50
To: TF-M(a)lists.trustedfirmware.org
Subject: Re: [TF-M] Test: Update test framework API
Hi TF-M maintainers,
It looks like there is no objection to the patchset https://review.trustedfirmware.org/c/trusted-firmware-m/+/3172. Can you please merge it?
Thanks,
Dev
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> on behalf of Devaraj Ranganna via TF-M <TF-M(a)lists.trustedfirmware.org<mailto:TF-M@lists.trustedfirmware.org>>
Reply to: Devaraj Ranganna <Devaraj.Ranganna(a)arm.com<mailto:Devaraj.Ranganna@arm.com>>
Date: Tuesday, 4 February 2020 at 16:30
To: "TF-M(a)lists.trustedfirmware.org<mailto:TF-M@lists.trustedfirmware.org>" <TF-M(a)lists.trustedfirmware.org<mailto:TF-M@lists.trustedfirmware.org>>
Subject: [TF-M] Test: Update test framework API
Hi,
Currently the test framework which executes test suites doesn't return anything. Therefore it is not possible for application layer to know the status of test cases. The patchset https://review.trustedfirmware.org/c/trusted-firmware-m/+/3172 is intended to export the test case pass/fail status to application layer and beyond (if any test framework is used by Non-secure side).
If there are no objections then can the patchset be merged?
Thanks,
Dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Vikas,
I'm seeing two topics here if I understand correctly.
The first one is how to deal with memory management in the os wrapper implementation.
The second one is how to safely "exit" and "delete" a child thread.
Please first check if my understanding is correct.
For the first topic, except the handles of threads, semaphores or mutex (which are the OS handles I guess), the OS wrapper needs some extra handles for resource handling.
For example the pointer of the memory allocated in the OS wrapper API. The two handles forms the new OS wrapper handle you proposed.
For the second topic, can we just terminate or delete the child thread by itself after it has done its job?
Best Regards,
Kevin
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Vikas Katariya via TF-M
Sent: Friday, February 7, 2020 12:55 AM
To: Gyorgy Szing <Gyorgy.Szing(a)arm.com>; Jamie Fox <Jamie.Fox(a)arm.com>; Anton Komlev <Anton.Komlev(a)arm.com>; tf-m(a)lists.trustedfirmware.org
Cc: nd <nd(a)arm.com>
Subject: Re: [TF-M] Changes to OS wrapper
Hi all,
Thanks for the e-mail.
I would like to highlight a few issues with the current API implementation.
* It isn't good for OS which relies on manual memory management, the current API doesn't give an opportunity to free any resources which were in use.
* The os-wrapper handle must be different from the actual underlying OS handle on a manually-memory-managed system in order to allow the resources to be freed which are not managed by the OS.
So a change in the API implementation is required to address the above issues.
* Use `os_wrapper_current_thread_suspend()` and `os_wrapper_thread_terminate(handle)` API, ensuring we suspend and terminate safely, enabling it to free allocated resources.
* Remove `os_wrapper_get_handle()` to make sure we differentiate between OS wrapper handle and OS handle. The os-wrapper knows more about the resources being managed than the OS itself. It is supposed to return an OS Wrapper handle than OS handle because implementations can't always create an OS-wrapper handle from an OS handle.
Comments towards the arguments:
* If we go with `os_wrapper_thread_exit()` to exit a child, then OS can allocate that resource to another purpose instantly and if we were to pass that handle to `os_wrapper_thread_delete(handle)`, we still risk corruption.
* If we go with `os_wrapper_thread_exit()` suspending the thread Or `os_wrapper_thread_delete(handle)` performing a NOP, it changes the semantics of what API intends to do and is not a natural way of moving forward. Also, it may confuse developers by making exit suspend instead of exit, or delete not delete anything.
* If we go with `os_wrapper_thread_exit()` performing a NOP and `os_wrapper_thread_delete(handle)` performing a termination of the thread, there are few things to consider here:
* It changes the semantics again for exit, but on some OS if the thread has finished its operations it will either exit or suspend itself as there is nothing to execute further.
* If the thread exits then os_wrapper_thread_delete(handle) will result in error.
* If we add a wrapper variable that captures info on if the thread has been exited to check deletion is safe or not, there few things to consider here:
* Adds an additional maintainability burden.
* Tracking if a thread has been exited or not adds a cost in RAM that would not be needed if the API had a good shape. Seeing as we all develop for embedded devices here, we should be very careful with RAM use.
* It can't figure out its identification because the OS-wrapper handle is not passed in `os_wrapper_thread_exit()`. Differentiating between the right handle is required to ensure we track the right thread information.
Depreciating the old API will ensure the following:
* Future applications are portable to any OS that needs manual memory management.
* It forces out-of-tree applications using the wrappers to know they need to make a change in order to be portable to operating systems that require manual memory management.
* The in-tree applications will be refactored as part of this API change.
Further, if there is a matter of handling deprecation of API, then I would like to know how that can be achieved in TF-M?
Thanks & Best Regards,
Vikas Katariya
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Anton Komlev via TF-M
Sent: Thursday, February 6, 2020 12:16
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: [TF-M] Changes to OS wrapper
Hello,
Agree with Jamie seeing not enough arguments for extension of existing API. Looks like the required functionality can be hidden inside a specific os_wrapper, which is a main purpose of it.
@Vikas, could you explain a bit differently why you are blocked with the current API?
Cheers,
Anton
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Gyorgy Szing via TF-M
Sent: 06 February 2020 09:10
To: Vikas Katariya <Vikas.Katariya(a)arm.com<mailto:Vikas.Katariya@arm.com>>; Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: [TF-M] Changes to OS wrapper
Hi,
>From the architecture point of view, when an "owner" (sw entity) defines an API, the best is to do that based on it's own needs. This gives the most flexibility and makes the API best withstand future challenges. Sometimes this is not possible. An example for this is the TRIM functionality of SSD storage, where the file-system must give extra information to the storage. In this case the extension of the API is driven by the implementation and thus implementation details. This is risky as different implementations may have conflicting needs, and sometimes the API cannot fulfill both.
If your case, a possible solution is to implement the os_theread_delete() and an empty function. If that is not possible, then a wrapper can be added where a variable captures info on if the thread has been exited, and thus if deletion is safe or not. If it is, then os_thread_delete() can exit without doing anything.
The call sequence of suspend and the delete is specific to the OS you are using or to the way you use it. Do you think does here a strong reason exist to go for an API extension driven by the implementation? I don't have all details, but as far as I understand the specific cases you described I don't see an imminent need for the API change.
/George
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Vikas Katariya via TF-M
Sent: 06 February 2020 08:47
To: Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: [TF-M] Changes to OS wrapper
Hi Jamie,
Thanks for getting back.
It's because of os_wrapper_thread_exit() is used to exit the child thread in the SST test, which means the handle is no longer valid for os_wrapper_thread_delete() to operate on, resulting in an error.
The ideal way to do this properly is to use os_wrapper_thread_suspend() and then os_wrapper_thread_delete() from the parent thread.
Thanks & Best Regards,
Vikas Katariya
From: Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>
Sent: Wednesday, February 5, 2020 17:16
To: Vikas Katariya <Vikas.Katariya(a)arm.com<mailto:Vikas.Katariya@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: RE: Changes to OS wrapper
Hi Vikas,
I still do not really understand the rationale for these changes. If dynamic memory allocation inside the os_wrapper shim is really what you want to do, then what is stopping you from implementing the following?
os_wrapper_thread_new()
{
malloc(external_to_os_resource)
/* create thread */
}
os_wrapper_thread_delete()
{
free(external_to_os_resource)
}
Kind regards,
Jamie
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Vikas Katariya via TF-M
Sent: 05 February 2020 10:20
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Subject: Re: [TF-M] Changes to OS wrapper
Hi all,
The patch set has been updated with further changes:
* https://review.trustedfirmware.org/c/trusted-firmware-m/+/3294
OS wrapper layers help to create Mutex, Semaphores, and Thread on an OS. The wrapper was designed to be implemented on platforms that dynamically allocate memory or objects from a predefined OS memory pool.
The current shape of the OS wrapper is not a good fit for work with operating systems that require manual memory management.
For example, if the child thread created in ns_test_helpers.c does a simple os_wrapper_thread_exit(), this does not give any opportunity for manually-managed thread resources to be freed; this leads to a memory leak.
Therefore os_wrapper_current_thread_suspend() and os_wrapper_thread_delete() are introduced to aid scenarios where manual memory management is required.
The removal of os_wrapper_thread_exit() is warranted as it encourages applications to avoid memory leak scenarios by requiring applications to remember to call os_wrapper_thread_terminate().
If we were to keep os_wrapper_thread_exit() around, this would impose undue cognitive overhead on wrapper users by making os_wrapper_thread_exit() do something other than exit the current thread (on platforms requiring manual memory management);
an os_wrapper_thread_exit() implementation could not actually exit a thread on a manual memory managed OS, as the thread must remain valid until clean up time, and exiting the thread would invalidate the OS's thread resource.
* https://review.trustedfirmware.org/c/trusted-firmware-m/+/3299
These changes reflect to avoid memory leaks on operating systems that use manually managed dynamic memory allocation but not from static memory/objects pools, allowing them to free after usage.
Remove "get_handle" because it's not possible to implement it efficiently on systems that require manual memory management.
The os-wrapper handle must be different from the actual underlying OS handle on a manually-memory-managed system in order to allow the resources be freed which are not managed by the OS.
For example:
struct {
os_handle;
external_to_os_resource;
};
The os-wrapper knows more about the resources being managed than the OS itself. It is supposed to return an OS Wrapper handle than OS handle, because implementations can't always create an os-wrapper handle from an OS handle.
In this case the os-wrapper handle could be a pointer to this struct, but could not be just the os_handle directly.
Further os_wrapper_current_thread_get_priority() is used to avoid confusion between the top and bottom layer handles, because the older implementation can refer to different object types when operating across multiple layers.
* https://review.trustedfirmware.org/c/trusted-firmware-m/+/3347 - Improves test efficiency.
Please review and share your thoughts.
Thanks & Best Regards,
Vikas Katariya
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Vikas Katariya via TF-M
Sent: Monday, January 27, 2020 15:52
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Subject: [TF-M] App: Changes to OS wrapper
Hi all,
I am proposing new changes to OS wrapper layer to help other RTOS use dynamic memory allocation.
OS wrapper layers help to create Mutex, Semaphores, and Thread on RTOS. The wrapper is designed to use static allocation of memory/objects
from predefined OS memory pool, which is not fully featured enough to allow dynamic memory allocation and freeing them after completion, if an RTOS
requires that kind of implementation.
For example, the child thread created in ns_test_helpers.c does a simple exit without passing a handle if the memory was dynamically allocated, which is a memory leak scenario.
Therefore os_wrapper_thread_suspend() and os_wrapper_thread_delete() are introduced to aid scenarios where dynamic memory allocation and freeing is required.
In the current patch we just suspend the child thread and terminate it from parent thread.
The patch is open for review here: https://review.trustedfirmware.org/c/trusted-firmware-m/+/3294
Thanks & Best Regards,
Vikas Katariya
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi TF-M maintainers,
It looks like there is no objection to the patchset https://review.trustedfirmware.org/c/trusted-firmware-m/+/3172. Can you please merge it?
Thanks,
Dev
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> on behalf of Devaraj Ranganna via TF-M <TF-M(a)lists.trustedfirmware.org>
Reply to: Devaraj Ranganna <Devaraj.Ranganna(a)arm.com>
Date: Tuesday, 4 February 2020 at 16:30
To: "TF-M(a)lists.trustedfirmware.org" <TF-M(a)lists.trustedfirmware.org>
Subject: [TF-M] Test: Update test framework API
Hi,
Currently the test framework which executes test suites doesn't return anything. Therefore it is not possible for application layer to know the status of test cases. The patchset https://review.trustedfirmware.org/c/trusted-firmware-m/+/3172 is intended to export the test case pass/fail status to application layer and beyond (if any test framework is used by Non-secure side).
If there are no objections then can the patchset be merged?
Thanks,
Dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi,
Now we just put code and RODATA into SFN so it works. And your concern really exists - for example, if a string is applied during SFN function, the string is actually out of SFN range and we need to put this string variable into SFN, too.
The SFN area should be simplified later one and SFN section should be removed (apply specific section to symbol should be restricted, only some special purposes like init-table usage are allowed). Instead, an overall library should be put in TFM_UNPRIV_CODE and with this: ?.lib (+RO) should work for your case - does IAR support this scenario with some a.lib (+RO) or it supports .o files only?
/Ken
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Thomas Törnblom via TF-M
Sent: Friday, February 7, 2020 7:11 PM
To: tf-m(a)lists.trustedfirmware.org
Subject: [TF-M] Linking issues with SFN section
I would like to discuss the use of the SFN section for the secure image.
During my port of tf-m to the IAR toolchain I ran into issues related to the SFN section. There are quite a few functions that are placed in the SFN section, which is then linked into the TFM_UNPRIV_CODE block.
I don't know how armclang or gcc handles this, but the IAR compiler may generate .rodata initializers, which does not end up in the SFN section, predominantly the in_vec and out_vec structs with debug builds. I've had to manually add the .rodata sections from these object files (tfm_*_secure_api.o) to the TFM_UNPRIV_CODE in the tfm_common linker script in order to work around MemManage_Handler traps.
I would like to suggest that the relevant files are added to the relevant� block in the tfm_common.* linker script instead of using the SFN section. That way one can specify that both the .text (ro code) and .rodata (const) goes into the same block.
Comments?
Thomas
--
Thomas T�rnblom, Product Engineer
IAR Systems AB
Box 23051, Strandbodgatan 1
SE-750 23 Uppsala, SWEDEN
Mobile: +46 76 180 17 80 Fax: +46 18 16 78 01
E-mail: thomas.tornblom(a)iar.com<mailto:thomas.tornblom@iar.com> Website: www.iar.com<http://www.iar.com>
Twitter: www.twitter.com/iarsystems<http://www.twitter.com/iarsystems>
Turns out the issue was with the timer interrupt routine clobbering a register when compiled with no optimization.
I have a workaround, but I’m also working with the compiler developers on what the proper behavior is.
Thomas
Thomas Törnblom, Product Engineer
IAR Systems AB
Box 23051, Strandbodgatan 1<x-apple-data-detectors://6/1>
SE-750 23 Uppsala, SWEDEN<x-apple-data-detectors://6/1>
Mobile: +46 76 180 17 80<tel:+46%2076%20180%2017%2080> Fax: +46 18 16 78 01<tel:+46%2018%2016%2078%2001>
E-mail: thomas.tornblom(a)iar.com<mailto:thomas.tornblom@iar.com> Website: www.iar.com<http://www.iar.com/>
Twitter: www.twitter.com/iarsystems<http://www.twitter.com/iarsystems>
6 feb. 2020 kl. 09:56 skrev Thomas Törnblom via TF-M <tf-m(a)lists.trustedfirmware.org>:
How is the IRQ_TEST_SCENARIO_4 supposed to work?
I suspect that there might be a lurking race condition somewhere in that test.
Some, not all, of the (M33/M23) targets gets stuck in that test when the ConfigRegression.cmake config is built with IAR in Debug mode. If I build it with RelWithDebInfo then the test runs OK for all applicable targets. No problems with Debug builds for the other configurations.
Occasionally the test will run successfully also for a normally problematic target if I run it in the debugger and stop execution at breakpoints, but it is very random, which is why I suspect there might be a race problem.
Thomas
--
Thomas T�rnblom, Product Engineer
IAR Systems AB
Box 23051, Strandbodgatan 1
SE-750 23 Uppsala, SWEDEN
Mobile: +46 76 180 17 80 Fax: +46 18 16 78 01
E-mail: thomas.tornblom(a)iar.com<mailto:thomas.tornblom@iar.com> Website: www.iar.com<http://www.iar.com>
Twitter: www.twitter.com/iarsystems<http://www.twitter.com/iarsystems>
--
TF-M mailing list
TF-M(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-m
Dear All,
As I mentioned on yesterday's call, there is a concern on user experience related to TF-M use.
To In order to understand and potentially improve it I am looking for a voice of partners who adopted TF-M project.
Please share your experience and thoughts on parts which are good or might be done better to simplify TF-M integration with your project.
You feedback will be very appreciated in any form - as a response to this mail or as a direct mail to me (anton.komlev(a)arm.com<mailto:anton.komlev@arm.com>) if it's more comfortable for you.
Thank you in advance,
Anton
Hi all,
Thanks for the e-mail.
I would like to highlight a few issues with the current API implementation.
* It isn't good for OS which relies on manual memory management, the current API doesn't give an opportunity to free any resources which were in use.
* The os-wrapper handle must be different from the actual underlying OS handle on a manually-memory-managed system in order to allow the resources to be freed which are not managed by the OS.
So a change in the API implementation is required to address the above issues.
* Use `os_wrapper_current_thread_suspend()` and `os_wrapper_thread_terminate(handle)` API, ensuring we suspend and terminate safely, enabling it to free allocated resources.
* Remove `os_wrapper_get_handle()` to make sure we differentiate between OS wrapper handle and OS handle. The os-wrapper knows more about the resources being managed than the OS itself. It is supposed to return an OS Wrapper handle than OS handle because implementations can't always create an OS-wrapper handle from an OS handle.
Comments towards the arguments:
* If we go with `os_wrapper_thread_exit()` to exit a child, then OS can allocate that resource to another purpose instantly and if we were to pass that handle to `os_wrapper_thread_delete(handle)`, we still risk corruption.
* If we go with `os_wrapper_thread_exit()` suspending the thread Or `os_wrapper_thread_delete(handle)` performing a NOP, it changes the semantics of what API intends to do and is not a natural way of moving forward. Also, it may confuse developers by making exit suspend instead of exit, or delete not delete anything.
* If we go with `os_wrapper_thread_exit()` performing a NOP and `os_wrapper_thread_delete(handle)` performing a termination of the thread, there are few things to consider here:
* It changes the semantics again for exit, but on some OS if the thread has finished its operations it will either exit or suspend itself as there is nothing to execute further.
* If the thread exits then os_wrapper_thread_delete(handle) will result in error.
* If we add a wrapper variable that captures info on if the thread has been exited to check deletion is safe or not, there few things to consider here:
* Adds an additional maintainability burden.
* Tracking if a thread has been exited or not adds a cost in RAM that would not be needed if the API had a good shape. Seeing as we all develop for embedded devices here, we should be very careful with RAM use.
* It can't figure out its identification because the OS-wrapper handle is not passed in `os_wrapper_thread_exit()`. Differentiating between the right handle is required to ensure we track the right thread information.
Depreciating the old API will ensure the following:
* Future applications are portable to any OS that needs manual memory management.
* It forces out-of-tree applications using the wrappers to know they need to make a change in order to be portable to operating systems that require manual memory management.
* The in-tree applications will be refactored as part of this API change.
Further, if there is a matter of handling deprecation of API, then I would like to know how that can be achieved in TF-M?
Thanks & Best Regards,
Vikas Katariya
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Anton Komlev via TF-M
Sent: Thursday, February 6, 2020 12:16
To: tf-m(a)lists.trustedfirmware.org
Cc: nd <nd(a)arm.com>
Subject: Re: [TF-M] Changes to OS wrapper
Hello,
Agree with Jamie seeing not enough arguments for extension of existing API. Looks like the required functionality can be hidden inside a specific os_wrapper, which is a main purpose of it.
@Vikas, could you explain a bit differently why you are blocked with the current API?
Cheers,
Anton
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Gyorgy Szing via TF-M
Sent: 06 February 2020 09:10
To: Vikas Katariya <Vikas.Katariya(a)arm.com<mailto:Vikas.Katariya@arm.com>>; Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: [TF-M] Changes to OS wrapper
Hi,
>From the architecture point of view, when an "owner" (sw entity) defines an API, the best is to do that based on it's own needs. This gives the most flexibility and makes the API best withstand future challenges. Sometimes this is not possible. An example for this is the TRIM functionality of SSD storage, where the file-system must give extra information to the storage. In this case the extension of the API is driven by the implementation and thus implementation details. This is risky as different implementations may have conflicting needs, and sometimes the API cannot fulfill both.
If your case, a possible solution is to implement the os_theread_delete() and an empty function. If that is not possible, then a wrapper can be added where a variable captures info on if the thread has been exited, and thus if deletion is safe or not. If it is, then os_thread_delete() can exit without doing anything.
The call sequence of suspend and the delete is specific to the OS you are using or to the way you use it. Do you think does here a strong reason exist to go for an API extension driven by the implementation? I don't have all details, but as far as I understand the specific cases you described I don't see an imminent need for the API change.
/George
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Vikas Katariya via TF-M
Sent: 06 February 2020 08:47
To: Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: [TF-M] Changes to OS wrapper
Hi Jamie,
Thanks for getting back.
It's because of os_wrapper_thread_exit() is used to exit the child thread in the SST test, which means the handle is no longer valid for os_wrapper_thread_delete() to operate on, resulting in an error.
The ideal way to do this properly is to use os_wrapper_thread_suspend() and then os_wrapper_thread_delete() from the parent thread.
Thanks & Best Regards,
Vikas Katariya
From: Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>
Sent: Wednesday, February 5, 2020 17:16
To: Vikas Katariya <Vikas.Katariya(a)arm.com<mailto:Vikas.Katariya@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: RE: Changes to OS wrapper
Hi Vikas,
I still do not really understand the rationale for these changes. If dynamic memory allocation inside the os_wrapper shim is really what you want to do, then what is stopping you from implementing the following?
os_wrapper_thread_new()
{
malloc(external_to_os_resource)
/* create thread */
}
os_wrapper_thread_delete()
{
free(external_to_os_resource)
}
Kind regards,
Jamie
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Vikas Katariya via TF-M
Sent: 05 February 2020 10:20
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Subject: Re: [TF-M] Changes to OS wrapper
Hi all,
The patch set has been updated with further changes:
* https://review.trustedfirmware.org/c/trusted-firmware-m/+/3294
OS wrapper layers help to create Mutex, Semaphores, and Thread on an OS. The wrapper was designed to be implemented on platforms that dynamically allocate memory or objects from a predefined OS memory pool.
The current shape of the OS wrapper is not a good fit for work with operating systems that require manual memory management.
For example, if the child thread created in ns_test_helpers.c does a simple os_wrapper_thread_exit(), this does not give any opportunity for manually-managed thread resources to be freed; this leads to a memory leak.
Therefore os_wrapper_current_thread_suspend() and os_wrapper_thread_delete() are introduced to aid scenarios where manual memory management is required.
The removal of os_wrapper_thread_exit() is warranted as it encourages applications to avoid memory leak scenarios by requiring applications to remember to call os_wrapper_thread_terminate().
If we were to keep os_wrapper_thread_exit() around, this would impose undue cognitive overhead on wrapper users by making os_wrapper_thread_exit() do something other than exit the current thread (on platforms requiring manual memory management);
an os_wrapper_thread_exit() implementation could not actually exit a thread on a manual memory managed OS, as the thread must remain valid until clean up time, and exiting the thread would invalidate the OS's thread resource.
* https://review.trustedfirmware.org/c/trusted-firmware-m/+/3299
These changes reflect to avoid memory leaks on operating systems that use manually managed dynamic memory allocation but not from static memory/objects pools, allowing them to free after usage.
Remove "get_handle" because it's not possible to implement it efficiently on systems that require manual memory management.
The os-wrapper handle must be different from the actual underlying OS handle on a manually-memory-managed system in order to allow the resources be freed which are not managed by the OS.
For example:
struct {
os_handle;
external_to_os_resource;
};
The os-wrapper knows more about the resources being managed than the OS itself. It is supposed to return an OS Wrapper handle than OS handle, because implementations can't always create an os-wrapper handle from an OS handle.
In this case the os-wrapper handle could be a pointer to this struct, but could not be just the os_handle directly.
Further os_wrapper_current_thread_get_priority() is used to avoid confusion between the top and bottom layer handles, because the older implementation can refer to different object types when operating across multiple layers.
* https://review.trustedfirmware.org/c/trusted-firmware-m/+/3347 - Improves test efficiency.
Please review and share your thoughts.
Thanks & Best Regards,
Vikas Katariya
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Vikas Katariya via TF-M
Sent: Monday, January 27, 2020 15:52
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Subject: [TF-M] App: Changes to OS wrapper
Hi all,
I am proposing new changes to OS wrapper layer to help other RTOS use dynamic memory allocation.
OS wrapper layers help to create Mutex, Semaphores, and Thread on RTOS. The wrapper is designed to use static allocation of memory/objects
from predefined OS memory pool, which is not fully featured enough to allow dynamic memory allocation and freeing them after completion, if an RTOS
requires that kind of implementation.
For example, the child thread created in ns_test_helpers.c does a simple exit without passing a handle if the memory was dynamically allocated, which is a memory leak scenario.
Therefore os_wrapper_thread_suspend() and os_wrapper_thread_delete() are introduced to aid scenarios where dynamic memory allocation and freeing is required.
In the current patch we just suspend the child thread and terminate it from parent thread.
The patch is open for review here: https://review.trustedfirmware.org/c/trusted-firmware-m/+/3294
Thanks & Best Regards,
Vikas Katariya
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
I would like to discuss the use of the SFN section for the secure image.
During my port of tf-m to the IAR toolchain I ran into issues related to
the SFN section. There are quite a few functions that are placed in the
SFN section, which is then linked into the TFM_UNPRIV_CODE block.
I don't know how armclang or gcc handles this, but the IAR compiler may
generate .rodata initializers, which does not end up in the SFN section,
predominantly the in_vec and out_vec structs with debug builds. I've had
to manually add the .rodata sections from these object files
(tfm_*_secure_api.o) to the TFM_UNPRIV_CODE in the tfm_common linker
script in order to work around MemManage_Handler traps.
I would like to suggest that the relevant files are added to the
relevant block in the tfm_common.* linker script instead of using the
SFN section. That way one can specify that both the .text (ro code) and
.rodata (const) goes into the same block.
Comments?
Thomas
--
*Thomas Törnblom*, /Product Engineer/
IAR Systems AB
Box 23051, Strandbodgatan 1
SE-750 23 Uppsala, SWEDEN
Mobile: +46 76 180 17 80 Fax: +46 18 16 78 01
E-mail: thomas.tornblom(a)iar.com <mailto:thomas.tornblom@iar.com>
Website: www.iar.com <http://www.iar.com>
Twitter: www.twitter.com/iarsystems <http://www.twitter.com/iarsystems>
So the patch does the wrong thing anyway - it effectively changes the watermark assertion so that it won't trigger if the TFM_RAM_CODE area goes beyond the end of RAM. That's just broken, because TFM_RAM_CODE is, obviously, supposed to be in RAM.
There may well be a problem here, but this is not the correct fix for it.
Chris
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Christopher Brand via TF-M
Sent: Thursday, February 6, 2020 3:58 PM
To: Jamie Fox <Jamie.Fox(a)arm.com>; tf-m(a)lists.trustedfirmware.org
Cc: Gabor Abonyi <Gabor.Abonyi(a)arm.com>; nd <nd(a)arm.com>
Subject: Re: [TF-M] PsoC64 build broken
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
I did an armclang build of the head of master and with this patch reverted, and there is a small difference between them - the TFM_RAM_CODE and SRAM_WATERMARK sections are swapped. That feels wrong, because the TFM_RAM_CODE is inside SRAM, so I'd expect the "end of SRAM watermark" to be after it, not before it. I'm not sure exactly how the watermark section is used, though. On the plus side, TFM_RAM_CODE does indeed still end up at the correct address, in SRAM.
I'm not sure exactly which sections you want me to try changing to set the LMA. The line numbers don't seem to match up...
Chris
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Christopher Brand via TF-M
Sent: Thursday, February 6, 2020 3:43 PM
To: Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>
Cc: Gabor Abonyi <Gabor.Abonyi(a)arm.com<mailto:Gabor.Abonyi@arm.com>>; nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: [TF-M] PsoC64 build broken
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
I'd really prefer to see the patch reverted first, and then we can figure out exactly what's going on and push a revised patch that achieves the objectives without breaking anything. It does seem to have some side-effects on the PSoC64 armclang build, too.
It's pretty standard open source process to revert a patch that breaks something like this...
Chris
From: Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>
Sent: Thursday, February 6, 2020 3:15 PM
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>; Christopher Brand <chris.brand(a)cypress.com<mailto:chris.brand@cypress.com>>
Cc: Gabor Abonyi <Gabor.Abonyi(a)arm.com<mailto:Gabor.Abonyi@arm.com>>; nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: PsoC64 build broken
Hi Chris,
Sorry about that. It shouldn't have changed the address of the region though because it's explicitly set to S_RAM_CODE_START.
I think this could be down to an "interesting" behaviour in GCC linker scripts where if the LMA is explicitly set with for a region with "AT>", then it won't revert back to being equal to the VMA for the next region in some cases. So before we revert the change, please can you try setting the LMA explicitly for the following region(s)? That is, try "> FLASH AT> FLASH" instead of just "> FLASH" on lines 527 and 563.
Best wishes,
Jamie
________________________________
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> on behalf of Christopher Brand via TF-M <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>>
Sent: 06 February 2020 20:11
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org> <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>>
Cc: Gabor Abonyi <Gabor.Abonyi(a)arm.com<mailto:Gabor.Abonyi@arm.com>>
Subject: [TF-M] PsoC64 build broken
Hi,
Commit 52182bc5e006752a4d28c3ccd909f93dafee0cf5 ("Build: Fix SRAM sanity check in common scatter file") seems to break the PSoc64 build. This is from https://review.trustedfirmware.org/c/trusted-firmware-m/+/3333
Building with gcc, I get:
/lhome/cbrand/work/trees_2/psoc6_atfm/trusted-firmware-m/build_gcc_psoc64/secure_fw/tfm_s.ld.i:352 cannot move location counter backwards (from 000000000802f578 to 0000000008000000)
collect2: error: ld returned 1 exit status
secure_fw/CMakeFiles/tfm_s.dir/build.make:210: recipe for target 'unit_test/tfm_s.axf' failed
I'm quite surprised that the comments on the review note that "I noticed that this define is currently only used for PSOC6 platform which I don't possess" and yet apparently a Musca B1-only build was considered sufficient to merge it.
I haven't dug into the details, but superficially it seems to move the .ramfunc code to before S_DATA_START, which means that it will no longer be in secure RAM.
Can we please revert this patch for now?
Chris
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.
I did an armclang build of the head of master and with this patch reverted, and there is a small difference between them - the TFM_RAM_CODE and SRAM_WATERMARK sections are swapped. That feels wrong, because the TFM_RAM_CODE is inside SRAM, so I'd expect the "end of SRAM watermark" to be after it, not before it. I'm not sure exactly how the watermark section is used, though. On the plus side, TFM_RAM_CODE does indeed still end up at the correct address, in SRAM.
I'm not sure exactly which sections you want me to try changing to set the LMA. The line numbers don't seem to match up...
Chris
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Christopher Brand via TF-M
Sent: Thursday, February 6, 2020 3:43 PM
To: Jamie Fox <Jamie.Fox(a)arm.com>; tf-m(a)lists.trustedfirmware.org
Cc: Gabor Abonyi <Gabor.Abonyi(a)arm.com>; nd <nd(a)arm.com>
Subject: Re: [TF-M] PsoC64 build broken
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
I'd really prefer to see the patch reverted first, and then we can figure out exactly what's going on and push a revised patch that achieves the objectives without breaking anything. It does seem to have some side-effects on the PSoC64 armclang build, too.
It's pretty standard open source process to revert a patch that breaks something like this...
Chris
From: Jamie Fox <Jamie.Fox(a)arm.com<mailto:Jamie.Fox@arm.com>>
Sent: Thursday, February 6, 2020 3:15 PM
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>; Christopher Brand <chris.brand(a)cypress.com<mailto:chris.brand@cypress.com>>
Cc: Gabor Abonyi <Gabor.Abonyi(a)arm.com<mailto:Gabor.Abonyi@arm.com>>; nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: PsoC64 build broken
Hi Chris,
Sorry about that. It shouldn't have changed the address of the region though because it's explicitly set to S_RAM_CODE_START.
I think this could be down to an "interesting" behaviour in GCC linker scripts where if the LMA is explicitly set with for a region with "AT>", then it won't revert back to being equal to the VMA for the next region in some cases. So before we revert the change, please can you try setting the LMA explicitly for the following region(s)? That is, try "> FLASH AT> FLASH" instead of just "> FLASH" on lines 527 and 563.
Best wishes,
Jamie
________________________________
From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> on behalf of Christopher Brand via TF-M <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>>
Sent: 06 February 2020 20:11
To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org> <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>>
Cc: Gabor Abonyi <Gabor.Abonyi(a)arm.com<mailto:Gabor.Abonyi@arm.com>>
Subject: [TF-M] PsoC64 build broken
Hi,
Commit 52182bc5e006752a4d28c3ccd909f93dafee0cf5 ("Build: Fix SRAM sanity check in common scatter file") seems to break the PSoc64 build. This is from https://review.trustedfirmware.org/c/trusted-firmware-m/+/3333
Building with gcc, I get:
/lhome/cbrand/work/trees_2/psoc6_atfm/trusted-firmware-m/build_gcc_psoc64/secure_fw/tfm_s.ld.i:352 cannot move location counter backwards (from 000000000802f578 to 0000000008000000)
collect2: error: ld returned 1 exit status
secure_fw/CMakeFiles/tfm_s.dir/build.make:210: recipe for target 'unit_test/tfm_s.axf' failed
I'm quite surprised that the comments on the review note that "I noticed that this define is currently only used for PSOC6 platform which I don't possess" and yet apparently a Musca B1-only build was considered sufficient to merge it.
I haven't dug into the details, but superficially it seems to move the .ramfunc code to before S_DATA_START, which means that it will no longer be in secure RAM.
Can we please revert this patch for now?
Chris
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.
This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.
