Hello,
when building TF-M you get the same result no matter what computer you build from assuming all the tooling is the same version.
The builds are reproducible in other words.
But when building TF-M together with the tests, e.g. by building the regression test suite you will get a different binary depending on where in the filesystem the tf-m-tests repository has been installed.
This is because the TEST_FAIL macro uses the gcc macro _FILE_ which injects the absolute paths of source files into strings in the binary.
This is a big problem for people using CI systems as a different sized binary can affect race conditions, flash region usage overflows, etc..
I think we should make builds reproducible by default, possibly with a configuration for adding absolute paths into the binary for those that prefer so.
I would like to write a patch to this affect, but would prefer to get some feedback now so I don't have to re-write it in review.
The simplest solution is to remove __FILE__ from the TEST_FAIL macro,but this would remove useful information and may also fail in the future if someone else uses __FILE__.
The more common way of solving this problem is to add a flag to the compiler that instructs it to replace any occurrences of the absolute path with something else during macro expansion. For instance replace "C:/repos/tf-m-tests/source_file.c" with "TFM_TESTS/source_file.c". For gcc the flag is -fmacro-prefix-map=<old_string>=<new_string>. I don't know what the flag is for the other supported compilers or if they support it at all.
Would it be OK to only support reproducible builds for the compilers that support an equivalent of macro-prefix-map?
Hi Sebastian,
Thanks a lot for bringing this up and sorry for the trouble. It is an interesting topic. The `file-prefix-map` workaround looks nice. Whilst I’m wondering if TF-M main repo also consists of __FILE__ macros, can the file paths (tf-m and tf-m-tests) be distinguished with `file-prefix-map`?
Best regards, Hu Ziji
From: Bøe, Sebastian via TF-M tf-m@lists.trustedfirmware.org Sent: Friday, September 2, 2022 6:43 PM To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Reproducible test builds
Hello,
when building TF-M you get the same result no matter what computer you build from assuming all the tooling is the same version.
The builds are reproducible in other words.
But when building TF-M together with the tests, e.g. by building the regression test suite you will get a different binary depending on where in the filesystem the tf-m-tests repository has been installed.
This is because the TEST_FAIL macro uses the gcc macro _FILE_ which injects the absolute paths of source files into strings in the binary.
This is a big problem for people using CI systems as a different sized binary can affect race conditions, flash region usage overflows, etc..
I think we should make builds reproducible by default, possibly with a configuration for adding absolute paths into the binary for those that prefer so.
I would like to write a patch to this affect, but would prefer to get some feedback now so I don't have to re-write it in review.
The simplest solution is to remove __FILE__ from the TEST_FAIL macro,but this would remove useful information and may also fail in the future if someone else uses __FILE__.
The more common way of solving this problem is to add a flag to the compiler that instructs it to replace any occurrences of the absolute path with something else during macro expansion. For instance replace "C:/repos/tf-m-tests/source_file.c" with "TFM_TESTS/source_file.c". For gcc the flag is -fmacro-prefix-map=<old_string>=<new_string>. I don't know what the flag is for the other supported compilers or if they support it at all.
Would it be OK to only support reproducible builds for the compilers that support an equivalent of macro-prefix-map?
Hi,
good to get positive feedback.
I have not seen the __FILE__ macro be expanded in any source file from tf-m yet (just tf-m-tests). But to be future proof we can use the gcc flag multiple times and make sure the absolute path of every repo is truncated. ________________________________ From: David Hu David.Hu@arm.com Sent: Tuesday, September 6, 2022 9:41 AM To: Bøe, Sebastian Sebastian.Boe@nordicsemi.no; tf-m@lists.trustedfirmware.org tf-m@lists.trustedfirmware.org Subject: RE: Reproducible test builds
Hi Sebastian,
Thanks a lot for bringing this up and sorry for the trouble.
It is an interesting topic. The `file-prefix-map` workaround looks nice.
Whilst I’m wondering if TF-M main repo also consists of __FILE__ macros, can the file paths (tf-m and tf-m-tests) be distinguished with `file-prefix-map`?
Best regards,
Hu Ziji
From: Bøe, Sebastian via TF-M tf-m@lists.trustedfirmware.org Sent: Friday, September 2, 2022 6:43 PM To: tf-m@lists.trustedfirmware.org Subject: [TF-M] Reproducible test builds
Hello,
when building TF-M you get the same result no matter what
computer you build from assuming all the tooling is the same version.
The builds are reproducible in other words.
But when building TF-M together with the tests, e.g. by building the regression test suite
you will get a different binary depending on where in the filesystem the tf-m-tests repository
has been installed.
This is because the TEST_FAIL macro uses the gcc macro _FILE_ which injects the absolute paths of source files
into strings in the binary.
This is a big problem for people using CI systems as a different sized binary can affect race conditions,
flash region usage overflows, etc..
I think we should make builds reproducible by default, possibly with a configuration for adding absolute paths
into the binary for those that prefer so.
I would like to write a patch to this affect, but would prefer to get some feedback now so I don't have to re-write it in review.
The simplest solution is to remove __FILE__ from the TEST_FAIL macro,but this would remove useful information and may also fail in the future if someone else uses __FILE__.
The more common way of solving this problem is to add a flag to the compiler that instructs it to replace any occurrences of the absolute path with something else during macro expansion. For instance replace "C:/repos/tf-m-tests/source_file.c" with "TFM_TESTS/source_file.c". For gcc the flag is -fmacro-prefix-map=<old_string>=<new_string>. I don't know what the flag is for the other supported compilers or if they support it at all.
Would it be OK to only support reproducible builds for the compilers that support an equivalent of macro-prefix-map?
Hi Sebastian,
IIUC, currently TF-M builds of all components share the compiling flags, unless you especially configure it for each targets. Probably it does work although it might not look elegant. 😊
It looks like we agree to the approach. Do you mind uploading the change so we can see the details? Besides, openCI can help verify the change in diverse cases.
Best regards Hu Ziji
From: Bøe, Sebastian Sebastian.Boe@nordicsemi.no Sent: Tuesday, September 6, 2022 4:10 PM To: David Hu David.Hu@arm.com; tf-m@lists.trustedfirmware.org Subject: Re: Reproducible test builds
Hi,
good to get positive feedback.
I have not seen the __FILE__ macro be expanded in any source file from tf-m yet (just tf-m-tests). But to be future proof we can use the gcc flag multiple times and make sure the absolute path of every repo is truncated. ________________________________ From: David Hu <David.Hu@arm.commailto:David.Hu@arm.com> Sent: Tuesday, September 6, 2022 9:41 AM To: Bøe, Sebastian <Sebastian.Boe@nordicsemi.nomailto:Sebastian.Boe@nordicsemi.no>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Reproducible test builds
Hi Sebastian,
Thanks a lot for bringing this up and sorry for the trouble.
It is an interesting topic. The `file-prefix-map` workaround looks nice.
Whilst I’m wondering if TF-M main repo also consists of __FILE__ macros, can the file paths (tf-m and tf-m-tests) be distinguished with `file-prefix-map`?
Best regards,
Hu Ziji
From: Bøe, Sebastian via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Sent: Friday, September 2, 2022 6:43 PM To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org Subject: [TF-M] Reproducible test builds
Hello,
when building TF-M you get the same result no matter what
computer you build from assuming all the tooling is the same version.
The builds are reproducible in other words.
But when building TF-M together with the tests, e.g. by building the regression test suite
you will get a different binary depending on where in the filesystem the tf-m-tests repository
has been installed.
This is because the TEST_FAIL macro uses the gcc macro _FILE_ which injects the absolute paths of source files
into strings in the binary.
This is a big problem for people using CI systems as a different sized binary can affect race conditions,
flash region usage overflows, etc..
I think we should make builds reproducible by default, possibly with a configuration for adding absolute paths
into the binary for those that prefer so.
I would like to write a patch to this affect, but would prefer to get some feedback now so I don't have to re-write it in review.
The simplest solution is to remove __FILE__ from the TEST_FAIL macro,but this would remove useful information and may also fail in the future if someone else uses __FILE__.
The more common way of solving this problem is to add a flag to the compiler that instructs it to replace any occurrences of the absolute path with something else during macro expansion. For instance replace "C:/repos/tf-m-tests/source_file.c" with "TFM_TESTS/source_file.c". For gcc the flag is -fmacro-prefix-map=<old_string>=<new_string>. I don't know what the flag is for the other supported compilers or if they support it at all.
Would it be OK to only support reproducible builds for the compilers that support an equivalent of macro-prefix-map?
I will upload it in due time, thank you for the feedback. ________________________________ From: David Hu David.Hu@arm.com Sent: Tuesday, September 6, 2022 10:49 AM To: Bøe, Sebastian Sebastian.Boe@nordicsemi.no; tf-m@lists.trustedfirmware.org tf-m@lists.trustedfirmware.org Subject: RE: Reproducible test builds
Hi Sebastian,
IIUC, currently TF-M builds of all components share the compiling flags, unless you especially configure it for each targets. Probably it does work although it might not look elegant. 😊
It looks like we agree to the approach. Do you mind uploading the change so we can see the details?
Besides, openCI can help verify the change in diverse cases.
Best regards
Hu Ziji
From: Bøe, Sebastian Sebastian.Boe@nordicsemi.no Sent: Tuesday, September 6, 2022 4:10 PM To: David Hu David.Hu@arm.com; tf-m@lists.trustedfirmware.org Subject: Re: Reproducible test builds
Hi,
good to get positive feedback.
I have not seen the __FILE__ macro be expanded in any source file from tf-m yet (just tf-m-tests). But to be future proof
we can use the gcc flag multiple times and make sure the absolute path of every repo is truncated.
________________________________
From: David Hu <David.Hu@arm.commailto:David.Hu@arm.com> Sent: Tuesday, September 6, 2022 9:41 AM To: Bøe, Sebastian <Sebastian.Boe@nordicsemi.nomailto:Sebastian.Boe@nordicsemi.no>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Reproducible test builds
Hi Sebastian,
Thanks a lot for bringing this up and sorry for the trouble.
It is an interesting topic. The `file-prefix-map` workaround looks nice.
Whilst I’m wondering if TF-M main repo also consists of __FILE__ macros, can the file paths (tf-m and tf-m-tests) be distinguished with `file-prefix-map`?
Best regards,
Hu Ziji
From: Bøe, Sebastian via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Sent: Friday, September 2, 2022 6:43 PM To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org Subject: [TF-M] Reproducible test builds
Hello,
when building TF-M you get the same result no matter what
computer you build from assuming all the tooling is the same version.
The builds are reproducible in other words.
But when building TF-M together with the tests, e.g. by building the regression test suite
you will get a different binary depending on where in the filesystem the tf-m-tests repository
has been installed.
This is because the TEST_FAIL macro uses the gcc macro _FILE_ which injects the absolute paths of source files
into strings in the binary.
This is a big problem for people using CI systems as a different sized binary can affect race conditions,
flash region usage overflows, etc..
I think we should make builds reproducible by default, possibly with a configuration for adding absolute paths
into the binary for those that prefer so.
I would like to write a patch to this affect, but would prefer to get some feedback now so I don't have to re-write it in review.
The simplest solution is to remove __FILE__ from the TEST_FAIL macro,but this would remove useful information and may also fail in the future if someone else uses __FILE__.
The more common way of solving this problem is to add a flag to the compiler that instructs it to replace any occurrences of the absolute path with something else during macro expansion. For instance replace "C:/repos/tf-m-tests/source_file.c" with "TFM_TESTS/source_file.c". For gcc the flag is -fmacro-prefix-map=<old_string>=<new_string>. I don't know what the flag is for the other supported compilers or if they support it at all.
Would it be OK to only support reproducible builds for the compilers that support an equivalent of macro-prefix-map?
tf-m@lists.trustedfirmware.org
-
Bøe, Sebastian -
David Hu