Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being "lazy"). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports "initial cache files" as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project's and external component's build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it's initial cache file [3].
Moreover, we might consider creating a tf.org specific "cmake library". This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
I agree that now many platforms are pulling in external libraries using the cmake mechanism, abstracting that code would be useful.
However I think at this time we don't need a solution such as LazyFetch - integrating it would require a fair bit of effort as it builds dependencies at a different stage to what we do currently, and it is also missing some features that we rely upon (such as patching). I don't believe that we have any particular performance issues with the fetching currently.
I'd suggest creating a file under the /cmake dir that abstracts the current template into a function (or macro if that is more appropriate) and then change everywhere in TF-M to use that.
Thanks for the suggestion Bohdan, I'll be happy to review such a change.
Raef
________________________________________ From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 12:46 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd Subject: [TF-M] Re: Library fetching improvements
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being “lazy”). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports “initial cache files” as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project’s and external component’s build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it’s initial cache file [3].
Moreover, we might consider creating a tf.org specific “cmake library”. This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
Hi,
All I am asking for is an investigation (few hours?) to understand the cost of both options. Can someone put a list of TF-M requirements on this component together please? Can we have an estimate for both options to allow having a proper comparison?
"- integrating it would require a fair bit of effort as it builds dependencies at a different stage" I assume most effort will be spent on testing locally and in the CI. That cost seems to be independent of which solution is chosen. I have a script to test the expected build behavior of external components for TS. It likely cannot be used to test TF-M, but LazyFetch was extensively tested. This might lower costs.
"it is also missing some features that we rely upon (such as patching" Patching is supported (it seems I managed to hide this info in my previous reply better than intended 😊 ).
/George
-----Original Message----- From: Raef Coles Raef.Coles@arm.com Sent: 12 April 2022 13:55 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com Cc: nd nd@arm.com Subject: Re: Library fetching improvements
I agree that now many platforms are pulling in external libraries using the cmake mechanism, abstracting that code would be useful.
However I think at this time we don't need a solution such as LazyFetch - integrating it would require a fair bit of effort as it builds dependencies at a different stage to what we do currently, and it is also missing some features that we rely upon (such as patching). I don't believe that we have any particular performance issues with the fetching currently.
I'd suggest creating a file under the /cmake dir that abstracts the current template into a function (or macro if that is more appropriate) and then change everywhere in TF-M to use that.
Thanks for the suggestion Bohdan, I'll be happy to review such a change.
Raef
________________________________________ From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 12:46 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd Subject: [TF-M] Re: Library fetching improvements
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being “lazy”). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports “initial cache files” as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project’s and external component’s build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it’s initial cache file [3].
Moreover, we might consider creating a tf.org specific “cmake library”. This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
Hi,
1. Let me remind the support of static libraries to avoid fetching on every new build via MBEDCRYPTO_PATH, MCUBOOT_PATH TFM_TEST_REPO_PATH variables (used in CI). Setting those variables signals to the build system to skip fetching. Moreover, theses and other config variables can be gathered in a config_file and provided to Cmake via single TFM_EXTRA_CONFIG_PAT=config_files option, which is very handy.
2. Wrapping the repetitive code into a function or macro looks as a low hanging fruit and good way to go, while the LazyFetch can be left to further improvement.
In general, I will vote for simplicity ahead of extra functionality.
Cheers, Anton
-----Original Message----- From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 2:46 PM To: Raef Coles Raef.Coles@arm.com; Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
All I am asking for is an investigation (few hours?) to understand the cost of both options. Can someone put a list of TF-M requirements on this component together please? Can we have an estimate for both options to allow having a proper comparison?
"- integrating it would require a fair bit of effort as it builds dependencies at a different stage" I assume most effort will be spent on testing locally and in the CI. That cost seems to be independent of which solution is chosen. I have a script to test the expected build behavior of external components for TS. It likely cannot be used to test TF-M, but LazyFetch was extensively tested. This might lower costs.
"it is also missing some features that we rely upon (such as patching" Patching is supported (it seems I managed to hide this info in my previous reply better than intended 😊 ).
/George
-----Original Message----- From: Raef Coles Raef.Coles@arm.com Sent: 12 April 2022 13:55 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com Cc: nd nd@arm.com Subject: Re: Library fetching improvements
I agree that now many platforms are pulling in external libraries using the cmake mechanism, abstracting that code would be useful.
However I think at this time we don't need a solution such as LazyFetch - integrating it would require a fair bit of effort as it builds dependencies at a different stage to what we do currently, and it is also missing some features that we rely upon (such as patching). I don't believe that we have any particular performance issues with the fetching currently.
I'd suggest creating a file under the /cmake dir that abstracts the current template into a function (or macro if that is more appropriate) and then change everywhere in TF-M to use that.
Thanks for the suggestion Bohdan, I'll be happy to review such a change.
Raef
________________________________________ From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 12:46 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd Subject: [TF-M] Re: Library fetching improvements
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being “lazy”). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports “initial cache files” as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project’s and external component’s build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it’s initial cache file [3].
Moreover, we might consider creating a tf.org specific “cmake library”. This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
-- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org
Hi,
I'd like to see the updated library fetching method, as there is always room for updating the integration system. Maybe we can see the introduction in the next tech forum?
Besides this, I am curious about those libraries that need a copy-and-modify. Could you help to tell me, is the modification happen in C sources? Wondered if it is still the situation I knew - I guess platform packages and crypto libraries are on the to-be-modified list.
Ideally, the code structure should be updated to make "direct" use of 3rd party libraries possible. For example, using the HAL adapter layer to connect the platform packages, and using a Secure Partition glue layer to use mbed-crypto directly.
Cheers.
/Ken
-----Original Message----- From: Anton Komlev via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 10:34 PM To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
1. Let me remind the support of static libraries to avoid fetching on every new build via MBEDCRYPTO_PATH, MCUBOOT_PATH TFM_TEST_REPO_PATH variables (used in CI). Setting those variables signals to the build system to skip fetching. Moreover, theses and other config variables can be gathered in a config_file and provided to Cmake via single TFM_EXTRA_CONFIG_PAT=config_files option, which is very handy.
2. Wrapping the repetitive code into a function or macro looks as a low hanging fruit and good way to go, while the LazyFetch can be left to further improvement.
In general, I will vote for simplicity ahead of extra functionality.
Cheers, Anton
-----Original Message----- From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 2:46 PM To: Raef Coles Raef.Coles@arm.com; Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
All I am asking for is an investigation (few hours?) to understand the cost of both options. Can someone put a list of TF-M requirements on this component together please? Can we have an estimate for both options to allow having a proper comparison?
"- integrating it would require a fair bit of effort as it builds dependencies at a different stage" I assume most effort will be spent on testing locally and in the CI. That cost seems to be independent of which solution is chosen. I have a script to test the expected build behavior of external components for TS. It likely cannot be used to test TF-M, but LazyFetch was extensively tested. This might lower costs.
"it is also missing some features that we rely upon (such as patching" Patching is supported (it seems I managed to hide this info in my previous reply better than intended 😊 ).
/George
-----Original Message----- From: Raef Coles Raef.Coles@arm.com Sent: 12 April 2022 13:55 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com Cc: nd nd@arm.com Subject: Re: Library fetching improvements
I agree that now many platforms are pulling in external libraries using the cmake mechanism, abstracting that code would be useful.
However I think at this time we don't need a solution such as LazyFetch - integrating it would require a fair bit of effort as it builds dependencies at a different stage to what we do currently, and it is also missing some features that we rely upon (such as patching). I don't believe that we have any particular performance issues with the fetching currently.
I'd suggest creating a file under the /cmake dir that abstracts the current template into a function (or macro if that is more appropriate) and then change everywhere in TF-M to use that.
Thanks for the suggestion Bohdan, I'll be happy to review such a change.
Raef
________________________________________ From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 12:46 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd Subject: [TF-M] Re: Library fetching improvements
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being “lazy”). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports “initial cache files” as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project’s and external component’s build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it’s initial cache file [3].
Moreover, we might consider creating a tf.org specific “cmake library”. This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
-- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org
Hi everyone,
Patch for library fetching improvements was created and is under review process. Patch link: https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/15811
Regards, Bohdan Hunko Cypress Semiconductor Ukraine Engineer
CSUKR CSS ICW SW FW
Mobile: +38099 50 19 714 HYPERLINK "mailto:Bohdan.Hunko@infineon.com" Bohdan.Hunko@infineon.com -----Original Message----- From: Ken Liu via TF-M tf-m@lists.trustedfirmware.org Sent: Wednesday, 13 April 2022 11:04 To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi,
I'd like to see the updated library fetching method, as there is always room for updating the integration system. Maybe we can see the introduction in the next tech forum?
Besides this, I am curious about those libraries that need a copy-and-modify. Could you help to tell me, is the modification happen in C sources? Wondered if it is still the situation I knew - I guess platform packages and crypto libraries are on the to-be-modified list.
Ideally, the code structure should be updated to make "direct" use of 3rd party libraries possible. For example, using the HAL adapter layer to connect the platform packages, and using a Secure Partition glue layer to use mbed-crypto directly.
Cheers.
/Ken
-----Original Message----- From: Anton Komlev via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 10:34 PM To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
1. Let me remind the support of static libraries to avoid fetching on every new build via MBEDCRYPTO_PATH, MCUBOOT_PATH TFM_TEST_REPO_PATH variables (used in CI). Setting those variables signals to the build system to skip fetching. Moreover, theses and other config variables can be gathered in a config_file and provided to Cmake via single TFM_EXTRA_CONFIG_PAT=config_files option, which is very handy.
2. Wrapping the repetitive code into a function or macro looks as a low hanging fruit and good way to go, while the LazyFetch can be left to further improvement.
In general, I will vote for simplicity ahead of extra functionality.
Cheers, Anton
-----Original Message----- From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 2:46 PM To: Raef Coles Raef.Coles@arm.com; Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
All I am asking for is an investigation (few hours?) to understand the cost of both options. Can someone put a list of TF-M requirements on this component together please? Can we have an estimate for both options to allow having a proper comparison?
"- integrating it would require a fair bit of effort as it builds dependencies at a different stage" I assume most effort will be spent on testing locally and in the CI. That cost seems to be independent of which solution is chosen. I have a script to test the expected build behavior of external components for TS. It likely cannot be used to test TF-M, but LazyFetch was extensively tested. This might lower costs.
"it is also missing some features that we rely upon (such as patching" Patching is supported (it seems I managed to hide this info in my previous reply better than intended 😊 ).
/George
-----Original Message----- From: Raef Coles Raef.Coles@arm.com Sent: 12 April 2022 13:55 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com Cc: nd nd@arm.com Subject: Re: Library fetching improvements
I agree that now many platforms are pulling in external libraries using the cmake mechanism, abstracting that code would be useful.
However I think at this time we don't need a solution such as LazyFetch - integrating it would require a fair bit of effort as it builds dependencies at a different stage to what we do currently, and it is also missing some features that we rely upon (such as patching). I don't believe that we have any particular performance issues with the fetching currently.
I'd suggest creating a file under the /cmake dir that abstracts the current template into a function (or macro if that is more appropriate) and then change everywhere in TF-M to use that.
Thanks for the suggestion Bohdan, I'll be happy to review such a change.
Raef
________________________________________ From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 12:46 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd Subject: [TF-M] Re: Library fetching improvements
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being “lazy”). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports “initial cache files” as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project’s and external component’s build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it’s initial cache file [3].
Moreover, we might consider creating a tf.org specific “cmake library”. This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
-- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org
Hi Bohdan,
Thanks for the good patch. Personally, like it as it makes TF-M scripts cleaner more structural and easier to read. Have few comments but will put it into Gerrit.
Would you like to introduce and discuss it on the forum tomorrow?
Best regards, Anton
-----Original Message----- From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: Wednesday, July 6, 2022 4:25 PM To: Ken Liu Ken.Liu@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi everyone,
Patch for library fetching improvements was created and is under review process. Patch link: https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/15811
Regards, Bohdan Hunko Cypress Semiconductor Ukraine Engineer
CSUKR CSS ICW SW FW
Mobile: +38099 50 19 714 HYPERLINK "mailto:Bohdan.Hunko@infineon.com" Bohdan.Hunko@infineon.com -----Original Message----- From: Ken Liu via TF-M tf-m@lists.trustedfirmware.org Sent: Wednesday, 13 April 2022 11:04 To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi,
I'd like to see the updated library fetching method, as there is always room for updating the integration system. Maybe we can see the introduction in the next tech forum?
Besides this, I am curious about those libraries that need a copy-and-modify. Could you help to tell me, is the modification happen in C sources? Wondered if it is still the situation I knew - I guess platform packages and crypto libraries are on the to-be-modified list.
Ideally, the code structure should be updated to make "direct" use of 3rd party libraries possible. For example, using the HAL adapter layer to connect the platform packages, and using a Secure Partition glue layer to use mbed-crypto directly.
Cheers.
/Ken
-----Original Message----- From: Anton Komlev via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 10:34 PM To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
1. Let me remind the support of static libraries to avoid fetching on every new build via MBEDCRYPTO_PATH, MCUBOOT_PATH TFM_TEST_REPO_PATH variables (used in CI). Setting those variables signals to the build system to skip fetching. Moreover, theses and other config variables can be gathered in a config_file and provided to Cmake via single TFM_EXTRA_CONFIG_PAT=config_files option, which is very handy.
2. Wrapping the repetitive code into a function or macro looks as a low hanging fruit and good way to go, while the LazyFetch can be left to further improvement.
In general, I will vote for simplicity ahead of extra functionality.
Cheers, Anton
-----Original Message----- From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 2:46 PM To: Raef Coles Raef.Coles@arm.com; Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
All I am asking for is an investigation (few hours?) to understand the cost of both options. Can someone put a list of TF-M requirements on this component together please? Can we have an estimate for both options to allow having a proper comparison?
"- integrating it would require a fair bit of effort as it builds dependencies at a different stage" I assume most effort will be spent on testing locally and in the CI. That cost seems to be independent of which solution is chosen. I have a script to test the expected build behavior of external components for TS. It likely cannot be used to test TF-M, but LazyFetch was extensively tested. This might lower costs.
"it is also missing some features that we rely upon (such as patching" Patching is supported (it seems I managed to hide this info in my previous reply better than intended 😊 ).
/George
-----Original Message----- From: Raef Coles Raef.Coles@arm.com Sent: 12 April 2022 13:55 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com Cc: nd nd@arm.com Subject: Re: Library fetching improvements
I agree that now many platforms are pulling in external libraries using the cmake mechanism, abstracting that code would be useful.
However I think at this time we don't need a solution such as LazyFetch - integrating it would require a fair bit of effort as it builds dependencies at a different stage to what we do currently, and it is also missing some features that we rely upon (such as patching). I don't believe that we have any particular performance issues with the fetching currently.
I'd suggest creating a file under the /cmake dir that abstracts the current template into a function (or macro if that is more appropriate) and then change everywhere in TF-M to use that.
Thanks for the suggestion Bohdan, I'll be happy to review such a change.
Raef
________________________________________ From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 12:46 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd Subject: [TF-M] Re: Library fetching improvements
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being “lazy”). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports “initial cache files” as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project’s and external component’s build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it’s initial cache file [3].
Moreover, we might consider creating a tf.org specific “cmake library”. This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
-- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org
Hi Anton,
I am not the person who was writing the code but I have talked with patch creator (Roman.Mazurak@infineon.com), so he will present this patch tomorrow on tech forum.
Regards, Bohdan Hunko Cypress Semiconductor Ukraine Engineer
CSUKR CSS ICW SW FW
Mobile: +38099 50 19 714 HYPERLINK "mailto:Bohdan.Hunko@infineon.com" Bohdan.Hunko@infineon.com -----Original Message----- From: Anton Komlev via TF-M tf-m@lists.trustedfirmware.org Sent: Wednesday, 6 July 2022 19:40 To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi Bohdan,
Thanks for the good patch. Personally, like it as it makes TF-M scripts cleaner more structural and easier to read. Have few comments but will put it into Gerrit.
Would you like to introduce and discuss it on the forum tomorrow?
Best regards, Anton
-----Original Message----- From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: Wednesday, July 6, 2022 4:25 PM To: Ken Liu Ken.Liu@arm.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi everyone,
Patch for library fetching improvements was created and is under review process. Patch link: https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/15811
Regards, Bohdan Hunko
Cypress Semiconductor Ukraine Engineer
CSUKR CSS ICW SW FW
Mobile: +38099 50 19 714 HYPERLINK "mailto:Bohdan.Hunko@infineon.com" Bohdan.Hunko@infineon.com
-----Original Message----- From: Ken Liu via TF-M tf-m@lists.trustedfirmware.org Sent: Wednesday, 13 April 2022 11:04 To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safehttps://intranet-content.infineon.com/explore/aboutinfineon/rules/informationsecurity/ug/SocialEngineering/Pages/SocialEngineeringElements_en.aspx.
Hi,
I'd like to see the updated library fetching method, as there is always room for updating the integration system. Maybe we can see the introduction in the next tech forum?
Besides this, I am curious about those libraries that need a copy-and-modify. Could you help to tell me, is the modification happen in C sources? Wondered if it is still the situation I knew - I guess platform packages and crypto libraries are on the to-be-modified list.
Ideally, the code structure should be updated to make "direct" use of 3rd party libraries possible. For example, using the HAL adapter layer to connect the platform packages, and using a Secure Partition glue layer to use mbed-crypto directly.
Cheers.
/Ken
-----Original Message----- From: Anton Komlev via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 10:34 PM To: tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
1. Let me remind the support of static libraries to avoid fetching on every new build via MBEDCRYPTO_PATH, MCUBOOT_PATH TFM_TEST_REPO_PATH variables (used in CI). Setting those variables signals to the build system to skip fetching. Moreover, theses and other config variables can be gathered in a config_file and provided to Cmake via single TFM_EXTRA_CONFIG_PAT=config_files option, which is very handy.
2. Wrapping the repetitive code into a function or macro looks as a low hanging fruit and good way to go, while the LazyFetch can be left to further improvement.
In general, I will vote for simplicity ahead of extra functionality.
Cheers, Anton
-----Original Message----- From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: Tuesday, April 12, 2022 2:46 PM To: Raef Coles Raef.Coles@arm.com; Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: [TF-M] Re: Library fetching improvements
Hi,
All I am asking for is an investigation (few hours?) to understand the cost of both options. Can someone put a list of TF-M requirements on this component together please? Can we have an estimate for both options to allow having a proper comparison?
"- integrating it would require a fair bit of effort as it builds dependencies at a different stage" I assume most effort will be spent on testing locally and in the CI. That cost seems to be independent of which solution is chosen. I have a script to test the expected build behavior of external components for TS. It likely cannot be used to test TF-M, but LazyFetch was extensively tested. This might lower costs.
"it is also missing some features that we rely upon (such as patching" Patching is supported (it seems I managed to hide this info in my previous reply better than intended 😊 ).
/George
-----Original Message----- From: Raef Coles Raef.Coles@arm.com Sent: 12 April 2022 13:55 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com Cc: nd nd@arm.com Subject: Re: Library fetching improvements
I agree that now many platforms are pulling in external libraries using the cmake mechanism, abstracting that code would be useful.
However I think at this time we don't need a solution such as LazyFetch - integrating it would require a fair bit of effort as it builds dependencies at a different stage to what we do currently, and it is also missing some features that we rely upon (such as patching). I don't believe that we have any particular performance issues with the fetching currently.
I'd suggest creating a file under the /cmake dir that abstracts the current template into a function (or macro if that is more appropriate) and then change everywhere in TF-M to use that.
Thanks for the suggestion Bohdan, I'll be happy to review such a change.
Raef
________________________________________ From: Gyorgy Szing via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 12:46 To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org Cc: nd Subject: [TF-M] Re: Library fetching improvements
Hi,
Dear team, please consider adopting LazyFetch [1]. Features:
* Optimizes on build speed (hence being “lazy”). If the binary is available will skip building, if the source is available will skip fetching. If you need to ensure the component is up-to-date, a clean build is needed. * It integrates well with cmake projects, but can support other build systems too. Note: if an extra step is needed between the fetch and the build phase, that is not supported yet (except for applying a patch). * It supports “initial cache files” as a convenient way to configure external cmake projects. Initial cache files avoid parameters being passed through multiple interpreting layers (cmake, shell, makefile, shell again). * It builds the external component during the config phase. This allows communication between the host project’s and external component’s build systems as the output of the external component build will be available during config phase of host. * It works as an extra layer on top of FetchContent.cmake.
For an example how it is used see here: qcbor [2] and it’s initial cache file [3].
Moreover, we might consider creating a tf.org specific “cmake library”. This would contain scripts and would help sharing cmake code between tf.org projects. All external components in TS would be a good candidate, but some scripts form the tf-a cmake repo are worth being considered too.
[1]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/tools/cmake/com... [2]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/... [3]: https://git.trustedfirmware.org/TS/trusted-services.git/tree/external/qcbor/...
From: Bohdan.Hunko--- via TF-M tf-m@lists.trustedfirmware.org Sent: 12 April 2022 13:19 To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Library fetching improvements
Hi, Recently I have been adding some new libraries to my TFM project and what I always end up doing is: go to some existing file which fetches the library, copy code from there, paste it to my file, change few links, versions and names.
It is a bit annoying to copy-paste that code each time, also it is hard to maintain (if pattern for fetching libraries changes) and also copy pasting might lead to some code not being updated.
My proposal is to have a function that can be used to fetch a library. This way it will be easier to add new libraries and this change will make code cleaner.
Please let me know your thoughts on this proposal.
Regards Bohdan Hunko
Cypress Semiconductor Ukraine Engineer CSUKR CSS ICW SW FW Mobile: +38099 50 19 714 Bohdan.Hunko@infineon.commailto:Bohdan.Hunko@infineon.com
-- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org -- TF-M mailing list -- tf-m@lists.trustedfirmware.org To unsubscribe send an email to tf-m-leave@lists.trustedfirmware.org
tf-m@lists.trustedfirmware.org
-
Anton Komlev -
Bohdan.Hunko＠infineon.com -
Gyorgy Szing -
Ken Liu -
Raef Coles