I’d like to take this opportunity to share my ideas to see if it makes sense.

 

The ideal solution should be:

  1. All the C source files are generated during build time while the CMake files are generated separately at config time so that the duplication of generation issue mentioned in below mail thread should be gone. (advanced, optional)

Currently the manifest tool generates both at the same time (at CMake config step).

  1. The source files building (target_add_sources) should be done in the same CMake file that generates them (tools/CMakeLists.txt) so that the CMake can correctly generate the files needed (otherwise it’s complicated for CMake to be aware where the files are coming from), and dependencies can be easily set up.

This requires a “centralized” building of partition intermedia and load info (putting them into two dedicated libraries).

This brings another benefit that the Secure Partitions does not need to take care of the generated files anymore.

 

The detailed engineering could be:

  1. Add a new Jinja2 template CMake file which builds (target_add_sources) all the generated source files.
    1. Two libraries, one for partition load info and one for partition intermedia
  2. The partition intermedia (stacks for now) should be put into Secure Partition libraries by putting them to specific sections within the Secure Partition libraries in linker scripts
    1. An existing utility can help this - TFM_LINK_SET_ZI_IN_PARTITION_SECTION
  3. Add a custom CMake target (add_custom_target) of the manifest tool command and set up the dependencies between generate files and the libraries build them (https://samthursfield.wordpress.com/2015/11/21/cmake-dependencies-between-targets-and-files-and-custom-commands/)
  4. The tools/CMakeLists.txt includes the generated CMake file in step 1.

These should solve the dependency issue, but the duplication of generation remains.

 

As I’m not available for implementing these, contributions are welcome 😊.

 

-Kevin

 

From: Kevin Peng via TF-M <tf-m@lists.trustedfirmware.org>
Sent: Friday, June 2, 2023 10:23 AM
To: Bohdan.Hunko@infineon.com; Joakim.Andersson@nordicsemi.no; Anton Komlev <Anton.Komlev@arm.com>; tf-m@lists.trustedfirmware.org
Subject: [TF-M] Re: Dependency on generated files is broken

 

Hi Bohdan,

 

Unfortunately, that patch didn’t fully work as expected.

I dig deeper and found out the root cause and it is more complicated than I thought to make the dependencies fully work.

I could make a quick fix but that would make the code hard to maintain and understand.

So, I’d like to apply some a comprehensive solution.

It requires more efforts though.

And I have stopped working on it because of other priorities.

 

-Kevin

 

From: Bohdan.Hunko@infineon.com <Bohdan.Hunko@infineon.com>
Sent: Thursday, June 1, 2023 9:58 PM
To: Kevin Peng <Kevin.Peng@arm.com>; Joakim.Andersson@nordicsemi.no; Anton Komlev <Anton.Komlev@arm.com>; tf-m@lists.trustedfirmware.org
Subject: RE: Dependency on generated files is broken

 

Hi,

 

Any updates on this?

 

 

Regards,

Bohdan Hunko

 

Cypress Semiconductor Ukraine

Engineer

CSUKR CSS ICW SW FW

Mobile: +38099 50 19 714
Bohdan.Hunko@infineon.com

 

 

From: Kevin Peng <Kevin.Peng@arm.com>
Sent: 6 May 2023 10:33
To: Andersson, Joakim <Joakim.Andersson@nordicsemi.no>; Hunko Bohdan (CSUKR CSS ICW SW FW 3) <Bohdan.Hunko@infineon.com>; Anton Komlev <Anton.Komlev@arm.com>; tf-m@lists.trustedfirmware.org
Subject: RE: Dependency on generated files is broken

 

Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safe.

 

Thanks for the information. It helps.

I’ve created a patch to add the dependencies back, whilst the duplication of the generating files also comes back.

They once happen in CMake config step and the other happen in the build time.

It happens only for Ninja build, not for make.

I’m not able to fix this for the time being. But I guess we can live with it.

 

Thanks,

Kevin

 

From: Andersson, Joakim <Joakim.Andersson@nordicsemi.no>
Sent: Thursday, April 27, 2023 3:27 PM
To: Kevin Peng <Kevin.Peng@arm.com>; Bohdan.Hunko@infineon.com; Anton Komlev <Anton.Komlev@arm.com>; tf-m@lists.trustedfirmware.org
Subject: RE: Dependency on generated files is broken

 

Cmake and generated files from custom commands can be quite difficult to get right.
I have had problems with this before and found this to be a great help in solving the problem:
https://samthursfield.wordpress.com/2015/11/21/cmake-dependencies-between-targets-and-files-and-custom-commands/

 

-Joakim

 

From: Kevin Peng <Kevin.Peng@arm.com>
Sent: torsdag 27. april 2023 04:58
To: Bohdan.Hunko@infineon.com; Andersson, Joakim <Joakim.Andersson@nordicsemi.no>; Anton Komlev <Anton.Komlev@arm.com>; tf-m@lists.trustedfirmware.org
Subject: RE: Dependency on generated files is broken

 

Caution: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

 

Hi Bohdan,

 

The commit you mentioned was to remove the duplication of generating files – the manifest tool was running twice.

It might also accidently remove the dependency.

However, I reverted the commit and dependency still doesn’t work.

I guess it might haven’t worked reliably in the past.

Anyway, we could try to work it out again.

 

For the time being, you need to do a clean build to get correct results after changing any templates or any other files the manifest tool consumes.

 

Thanks,

Kevin

 

From: Bohdan.Hunko--- via TF-M <tf-m@lists.trustedfirmware.org>
Sent: Thursday, April 27, 2023 4:23 AM
To: Joakim.Andersson@nordicsemi.no; Anton Komlev <Anton.Komlev@arm.com>; tf-m@lists.trustedfirmware.org
Subject: [TF-M] Re: Dependency on generated files is broken

 

I am talking about .template files in tfm repo (e.g.: tools/templates/manifestfilename.template)

 

  • @Anton.Komlev@arm.com: I afraid that dependencies on templates were missed all the time and shall be added manually.

Hmm, I seem to remember that it worked for me. Also there is a code in tools/CMakeLists.txt that collects a list of generated files into ${TEMPLATE_FILES} variable and then sets (DEPENDS ${TEMPLATE_FILES}), but 1ce59292a47b1316e5d8b4d28bcaf9d8e2bdc0a5 removed the code that does DEPENDS part, so now list is collected but never used.

Yeah, sure, this problem is not stopping us in any way 😊

 

 

Regards,

Bohdan Hunko

 

Cypress Semiconductor Ukraine

Engineer

CSUKR CSS ICW SW FW

Mobile: +38099 50 19 714
Bohdan.Hunko@infineon.com

 

 

From: Andersson, Joakim <Joakim.Andersson@nordicsemi.no>
Sent: 26 April 2023 18:45
To: Anton Komlev <Anton.Komlev@arm.com>; Hunko Bohdan (CSUKR CSS ICW SW FW 3) <Bohdan.Hunko@infineon.com>; tf-m@lists.trustedfirmware.org
Subject: RE: Dependency on generated files is broken

 

Caution: This e-mail originated outside Infineon Technologies. Do not click on links or open attachments unless you validate it is safe.

 

Which .template files?
There are also .template files in the psa-arch-tests repository.
And as I’ve figured out when reporting this issue: https://github.com/ARM-software/psa-arch-tests/issues/344 there is not a BUILD_ALWAYS on external projects.
Which means that all of psa-arch-tests would not get rebuilt on edit AFAICT.

 

-Joakim

 

From: Anton Komlev via TF-M <tf-m@lists.trustedfirmware.org>
Sent: onsdag 26. april 2023 17:06
To: Bohdan.Hunko@infineon.com; tf-m@lists.trustedfirmware.org
Subject: [TF-M] Re: Dependency on generated files is broken

 

Caution: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

 

Hi,

 

Thanks, Bohdan for finding that.

You may be right but I afraid that dependencies on templates were missed all the time and shall be added manually.

Let’s wait for Kevin and David opinion.

 

Cheers,

Anton

 

From: Bohdan.Hunko--- via TF-M <tf-m@lists.trustedfirmware.org>
Sent: Wednesday, April 26, 2023 3:20 PM
To: tf-m@lists.trustedfirmware.org
Subject: [TF-M] Dependency on generated files is broken

 

Hi all,

 

Seems like dependency on generated files is broken.

Steps to reproduce:

  1. Build any platform at any mode
  2. Change any .template file
  3.  

Expected result:

  1. New file is generated from the updated .template file

 

Actual result:

  1. Generated files step is skipped.

 

My best guess will be that 1ce59292a47b1316e5d8b4d28bcaf9d8e2bdc0a5 broke it.

 

Could this be fixed?

 

Regards,

Bohdan Hunko

 

Cypress Semiconductor Ukraine

Engineer

CSUKR CSS ICW SW FW

Mobile: +38099 50 19 714
Bohdan.Hunko@infineon.com