- TF-M - lists.trustedfirmware.org

Re: [TF-M] [EXTERNAL] Re: out of tree build

by Kevin Peng

Hi, I'm planning to settle down the first three patches for tools change of this topic, before Christmas. Because personally think they are very mature and close to merge. (And I've still working on the reset patches). So if there are no more new review comments, I'll go with the current reviews and try to merge it. Best Regards, Kevin -----Original Message----- From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Kevin Peng (Arm Technology China) via TF-M Sent: Friday, December 6, 2019 2:41 PM To: DeMars, Alan <ademars(a)ti.com>; 'tf-m(a)lists.trustedfirmware.org' <tf-m(a)lists.trustedfirmware.org> Cc: nd <nd(a)arm.com> Subject: Re: [TF-M] [EXTERNAL] Re: out of tree build I've created some patches for supporting this out of tree build: https://review.trustedfirmware.org/q/topic:%22out_of_tree_build%22+(status:… I firstly modified the tools/tfm_parse_manifest_list.py so that it can take customized secure partition manifest and list of files to generate and output the files to a specified directory. You can even "append" your manifests or files to generate to the default lists under tools. The format of the yamls are the same. The tfm_parse_manifest_list.py script will also create a CMakeLists.inc which contains all the paths of generated header. And the paths are added to include search path. Other CMake projects can include this file to find generated headers. And then the generated files are moved to a dedicated folder to solve the issue mentioned below (generated headers cannot be included). Finally, the build system is able to call the parse tool to generate files in build time(can be disabled) and use the new generated files to build. You can set the optional arguments for the parse tool by setting CMake arguments: cmake -DTFM_MANIFEST(_APPEND)=some_manifest -DTFM_GEN_LIST(_APPEND)=some_generated_file -DTFM_GEN_DIR=... -G'Unix Makefiles' -DTARGET_PLATFORM=AN521 -D COMPILER=ARMCLANG -DPROJ_CONFIG=`readlink -f ConfigXXX.cmake` TFM_ROOT cmake –build Or you can pre-generate the files, disable build time generation and tell cmake the output directory only. Comments and discussions are welcome. Best Regards, Kevin -----Original Message----- From: DeMars, Alan <ademars(a)ti.com> Sent: Saturday, November 23, 2019 12:33 AM To: Kevin Peng (Arm Technology China) <Kevin.Peng(a)arm.com> Cc: 'tf-m(a)lists.trustedfirmware.org' <tf-m(a)lists.trustedfirmware.org> Subject: RE: [TF-M] [EXTERNAL] Re: out of tree build In our use case, we added complete paths to the build area's corresponding directory to the embedded include paths and removed the default generated content from our git repo. Consequently only the newly generated content is found, wherever it happens to be placed. Alan -----Original Message----- From: TF-M [mailto:tf-m-bounces@lists.trustedfirmware.org] On Behalf Of Kevin Peng (Arm Technology China) via TF-M Sent: Friday, November 22, 2019 2:15 AM To: tf-m(a)lists.trustedfirmware.org Cc: nd Subject: Re: [TF-M] [EXTERNAL] Re: out of tree build Hi all, I'm having issues on supporting the out of tree build. So I'd like to propose some additional changes. What does the out of tree build currently support is that: The user uses the tfm_parse_manifest_list.py with the custom manifests to generate the customized files to a directory out of TF-M. And the TF-M build system uses the customized files rather than the pre-generated ones in the TF-M for building. The problem is that some of the source files include the generated headers using the relative path to where the source itself is. And the directory of the source file is always searched first. This makes it impossible to use the generated files outside TF-M even the custom output directory is added to searching list. So I suggest to put the generated files to a dedicated directory in TF-M. Then for the default build, add the directory and its subdirectories to the search list for headers. For out of tree build, use the specified directory and its subdirectories instead. Then there would only one header file with the same name in all the directories for searching headers. Any concerns, thoughts or suggestions? Best Regards, Kevin -----Original Message----- From: Kevin Peng (Arm Technology China) Sent: Friday, November 22, 2019 3:45 PM To: DeMars, Alan <ademars(a)ti.com>; tf-m(a)lists.trustedfirmware.org Cc: nd <nd(a)arm.com> Subject: RE: [EXTERNAL] Re: [TF-M] out of tree build Hi Alan, I've created a patch for the tfm_parse_manifest_list.py as the first step for the out of tree build support: https://review.trustedfirmware.org/c/trusted-firmware-m/+/2606 The changes for build system is ongoing. The functionality changes for the tfm_parse_manifest_list.py is the same as your version. Please have a review and help verify it if possible. Comments from anyone else are also welcome. Best Regards, Kevin -----Original Message----- From: Kevin Peng (Arm Technology China) Sent: Thursday, November 14, 2019 3:31 PM To: DeMars, Alan <ademars(a)ti.com> Cc: tf-m(a)lists.trustedfirmware.org; nd <nd(a)arm.com> Subject: RE: [EXTERNAL] Re: [TF-M] out of tree build OK. It's much clearer for me now. Thanks. Best Regards, Kevin -----Original Message----- From: DeMars, Alan <ademars(a)ti.com> Sent: Thursday, November 14, 2019 10:34 AM To: Kevin Peng (Arm Technology China) <Kevin.Peng(a)arm.com> Cc: tf-m(a)lists.trustedfirmware.org; nd <nd(a)arm.com> Subject: Re: [EXTERNAL] Re: [TF-M] out of tree build 1) I verified the tfm_parse_manifest.py script changes using the master branch’s version of the tfm_manifest_list.yaml and tfm_generated_file_list.yaml files. 2) Yes, I’m building using the unmodified master branch’s cmake build system. 3) Just as all other build artifacts are placed in the user’s build directory and NOT in the source tree, the generated files should also. Otherwise, the user is required to update the source tree’s tfm_manifest_list.yaml and tfm_generated_file_list.yaml files to build a custom SPE, with the generated files also ending up in the source tree. The files necessary to create a custom SPE should not be kept in the source tree. Nor should the generated content be inserted in the source tree. This creates headaches for maintaining the original source tree. Alan > On Nov 13, 2019, at 1:57 AM, Kevin Peng (Arm Technology China) via TF-M <tf-m(a)lists.trustedfirmware.org> wrote: > > Hi Alan, > > I checked your modified tfm_parse_manifest_list.py. > It basically uses the input tfm_manifest_list.yaml and tfm_generated_file_list.yaml to generate the TF-M auto generated files to the third input. > > I'm sorry I was not in the workshop. So I have some questions. > 1. Do you have your own tfm_manifest_list.yaml and tfm_generated_file_list.yaml that is not upstreamed in TF-M? > 2. Are you using the TF-M provided CMake build system? > 3. Do you only need the files generated by tfm_parse_manifest_list.py to be out of the TF-M source directory? > > Could you provide some details about your requirements. Thanks. > > Best Regards, > Kevin > > -----Original Message----- > From: Kevin Peng (Arm Technology China) > Sent: Wednesday, November 13, 2019 10:27 AM > To: tf-m(a)lists.trustedfirmware.org > Cc: nd <nd(a)arm.com> > Subject: RE: out of tree build > > Hi Alan, > > I'm following on this and will get you updated on any progress. > > Best Regards, > Kevin > > -----Original Message----- > From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of DeMars, Alan via TF-M > Sent: Thursday, November 7, 2019 12:42 PM > To: Abhishek Pandit <Abhishek.Pandit(a)arm.com> > Cc: 'tf-m(a)lists.trustedfirmware.org' <tf-m(a)lists.trustedfirmware.org> > Subject: Re: [TF-M] out of tree build > > Trying again after renaming tfm_parse_manifest_list.py to tfm_parse_manifest_list.txt so it wouldn't get deleted. > --- > > With the attached modified secure_fw/CMakeLists.txt and tools/tfm_parse_manifest_list.py files, I am able to redirect the generated files using the following command line: > > python3 <full_path_to_your>/tfm_parse_manifest_list.py <full_path_to_your>/tfm_manifest_list.yaml <full_path_to_your>/tfm_generated_file_list.yaml <full_path_to_your>/<build_dir> > > And then build the ConfigCoreIPC artifacts within <your_build_dir> using the following cmake line: > > cmake -G"Unix Makefiles" -DPROJ_CONFIG=`readlink -f ../configs/ConfigCoreIPC.cmake` -DREMOTE_GEN_DIR=<full_path_to_your><build_dir> -DTARGET_PLATFORM=AN521 -DCOMPILER=GNUARM ../ > > The changes to tfm_parse_manifest_list.py are backward compatible with the standard usage. > > Alan > > > -----Original Message----- > From: TF-M [mailto:tf-m-bounces@lists.trustedfirmware.org] On Behalf Of DeMars, Alan via TF-M > Sent: Monday, November 4, 2019 4:36 PM > To: Abhishek Pandit; tf-m(a)lists.trustedfirmware.org > Subject: [EXTERNAL] Re: [TF-M] out of tree build > > Abishek, > > Yes, we have modified tfm_gen.py and tfm_parse_manifest_list.py to support redirecting the destination of the generated template files into a command line provided destination build directory. > A corresponding change to our platform's platform/ext/xyz.cmake file was also required to add the path to the root of the build directory to the embedded_include_directories() list as well as the paths to the generated linker command files. > > I was not planning to provide these changes as a patch for review as I am very unsure of the applicability of this to other platforms. Also, I was fairly certain that given my very poor understanding of the CMake build system, my approach to the problem was not utilizing features present in CMake that would make the job simpler and more extensible. > > Since the topic came up at the conference and you already seemed willing to address the out-of-tree build problem that the templates lead to, I assumed you folks would find a simple and elegant solution that would save me the embarrassment of exposing my lack of CMake expertise. > > Alan > > -----Original Message----- > From: Abhishek Pandit [mailto:Abhishek.Pandit@arm.com] > Sent: Monday, November 4, 2019 3:08 PM > To: DeMars, Alan; tf-m(a)lists.trustedfirmware.org > Subject: [EXTERNAL] RE: out of tree build > > Hi Alan, > Not sure if I remember the exact detail from the workshop last week, but did you mention that you have a prototype for this? If so, are you planning to push a patch for review? > Thanks, > Abhishek > > -----Original Message----- > From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of DeMars, Alan via TF-M > Sent: 01 November 2019 16:29 > To: 'tf-m(a)lists.trustedfirmware.org' <tf-m(a)lists.trustedfirmware.org> > Subject: [TF-M] out of tree build > > Please modify the template generators to support out of tree build, and modify the CMake files to add the necessary include paths so that the files that include the template-generated files can find them. > > Alan > -- > TF-M mailing list > TF-M(a)lists.trustedfirmware.org > https://lists.trustedfirmware.org/mailman/listinfo/tf-m > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > -- > TF-M mailing list > TF-M(a)lists.trustedfirmware.org > https://lists.trustedfirmware.org/mailman/listinfo/tf-m > -- > TF-M mailing list > TF-M(a)lists.trustedfirmware.org > https://lists.trustedfirmware.org/mailman/listinfo/tf-m -- TF-M mailing list TF-M(a)lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-m -- TF-M mailing list TF-M(a)lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-m

4 years, 8 months

1
1
0 0

Move dual-cpu design document into code repo

by David Hu

Hi all, We are moving dual-cpu design documents from trustedfirmware.org wiki pages into TF-M code repo. We convert the documents to rst format, fix some typos, update them to align with latest implementation. The patches have been reviewed for several rounds in https://review.trustedfirmware.org/q/topic:%22dualcpu-docs%22+(status:open%…. The documents patches will be merged soon by this week. Please comment on the patches if there is any serious issue in the design. Any suggestion or improvement is still welcome after the patches are merged! Best regards, Hu Ziji

4 years, 8 months

1
0
0 0

Re: [TF-M] [Call for cygwin volunteers] Remove the mbed-crypto building workaround

by Ken Liu

From the feedbacks till now, no fault case is reported, let merge this patch. If someone met problem while building please reply in the mailing list. /Ken From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Ken Liu via TF-M Sent: Thursday, January 9, 2020 9:32 AM To: tf-m(a)lists.trustedfirmware.org Cc: nd <nd(a)arm.com> Subject: Re: [TF-M] [Call for cygwin volunteers] Remove the mbed-crypto building workaround Got 3 feedbacks till now all reports no problem. I am going to merge this one before end of today. /Ken From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Ken Liu via TF-M Sent: Monday, January 6, 2020 1:56 PM To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org> Cc: nd <nd(a)arm.com<mailto:nd@arm.com>> Subject: Re: [TF-M] [Call for cygwin volunteers] Remove the mbed-crypto building workaround Thanks for the feedback, let me take a note. /Ken From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Qixiang Xu via TF-M Sent: Monday, January 6, 2020 1:49 PM To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org> Subject: Re: [TF-M] [Call for cygwin volunteers] Remove the mbed-crypto building workaround Ken, I have cherry picked the patch and tested it on Cygwin: $ cmake --version cmake version 3.11.1 CMake suite maintained and supported by Kitware (kitware.com/cmake). The patch works and no issue found. Best Regards, Qixiang Xu From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Ken Liu via TF-M Sent: Monday, January 6, 2020 11:00 AM To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org> Cc: nd <nd(a)arm.com<mailto:nd@arm.com>> Subject: [TF-M] [Call for cygwin volunteers] Remove the mbed-crypto building workaround Hi, I create a patch for removing the workaround for mbed-crypto building: https://review.trustedfirmware.org/c/trusted-firmware-m/+/3022 We tried on cmake 3.7 and 3.10 with cygwin and it works; can some Cygwin/mingw user pick this patch and test if it could work in your side? Thanks for your contribution 😊 /Ken IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

4 years, 8 months

1
0
0 0

Invitation: TF-M Tech Forum @ Thu 23 Jan 2020 17:00 - 18:00 (GMT) (tf-m@lists.trustedfirmware.org)

by Bill Fletcher

You have been invited to the following event. Title: TF-M Tech Forum This is an open forum for anyone to participate and it is not restricted to Trusted Firmware project members. It will operate under the guidance of the TF TSC. Feel free to forward it to colleagues. ────────── Bill Fletcher is inviting you to a scheduled Zoom meeting. Join Zoom Meeting https://zoom.us/j/5810428000 Meeting ID: 581 042 8000 One tap mobile +16465588656,,5810428000# US (New York) +16699009128,,5810428000# US (San Jose) Dial by your location +1 646 558 8656 US (New York) +1 669 900 9128 US (San Jose) 877 853 5247 US Toll-free 888 788 0099 US Toll-free Meeting ID: 581 042 8000 Find your local number: https://zoom.us/u/aerUYXPhSL ────────── When: Thu 23 Jan 2020 17:00 – 18:00 United Kingdom Time Where: https://zoom.us/j/5810428000 Calendar: tf-m(a)lists.trustedfirmware.org Who: (Guest list has been hidden at organiser's request) Event details: https://www.google.com/calendar/event?action=VIEW&eid=MzB0czdyMXVlMXBpaGY5M… Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account tf-m(a)lists.trustedfirmware.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively, you can sign up for a Google Account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to send a response to the organiser and be added to the guest list, invite others regardless of their own invitation status or to modify your RSVP. Learn more at https://support.google.com/calendar/answer/37135#forwarding

4 years, 8 months

1
0
0 0

Re: [TF-M] T398: Update to IAR support

by Karl Zhang

Hi Thomas Thanks for the update, I go through your new patches initially and give some first review comments. Besides, I am trying to have a build by IAR myself, and do a more detail review later. Please be aware, the concept for review now is to make sure the new compiler can work on specified platform and won't block the current TFM process. Since the new year holiday coming soon in China, my response may slow before 1st Feb. Sorry about that. Thanks Karl From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Thomas T?rnblom via TF-M Sent: Tuesday, January 14, 2020 16:38 To: tf-m(a)lists.trustedfirmware.org Subject: [TF-M] T398: Update to IAR support I have just pushed the latest commits for the source cleanup and IAR integration. https://review.trustedfirmware.org/c/trusted-firmware-m/+/3127/1 Please review. Thanks, Thomas -- Thomas Tï¿½rnblom, Product Engineer IAR Systems AB Box 23051, Strandbodgatan 1 SE-750 23 Uppsala, SWEDEN Mobile: +46 76 180 17 80 Fax: +46 18 16 78 01 E-mail: thomas.tornblom(a)iar.com<mailto:thomas.tornblom@iar.com> Website: www.iar.com<http://www.iar.com> Twitter: www.twitter.com/iarsystems<http://www.twitter.com/iarsystems>

4 years, 8 months

1
0
0 0

Jan 23rd call details

by Mohammed Billoo

Hello, May I get the invite for the call next week? Thank you -- Mohammed A Billoo Founder MAB Labs, LLC www.mab-labs.com 201-201-8088

4 years, 9 months

1
0
0 0

T398: Update to IAR support

by Thomas Törnblom

I have just pushed the latest commits for the source cleanup and IAR integration. https://review.trustedfirmware.org/c/trusted-firmware-m/+/3127/1 Please review. Thanks, Thomas -- *Thomas Törnblom*, /Product Engineer/ IAR Systems AB Box 23051, Strandbodgatan 1 SE-750 23 Uppsala, SWEDEN Mobile: +46 76 180 17 80 Fax: +46 18 16 78 01 E-mail: thomas.tornblom(a)iar.com <mailto:thomas.tornblom@iar.com> Website: www.iar.com <http://www.iar.com> Twitter: www.twitter.com/iarsystems <http://www.twitter.com/iarsystems>

4 years, 9 months

1
0
0 0

Re: [TF-M] Code Protection between secure services

by Andrew Thoelke

PSA-FF does not require isolation of code. Section 3.1 describes the isolation architecture (pages 21-25) and has three mandatory rules: 1. Only "code" is executable 2. Only "private data" is writable 3. "private data" is protected from untrusted "protection domains". (E.g. secure side "private data" cannot be accessed by non-secure) [quoted items are defined terms in that section of the document] Protecting "code" and "constant data" from other "protection domains" is provided in optional isolation rules 4, 5 and 6 with increasing levels of isolation (and system cost). The security rationale for isolating code is (a) confidentiality of the code and associated read-only data and (b) reduction of the number of ROP/JOP gadgets available to an attacker. The level of isolation and implementation of the optional rules is intended to be a framework and/or product decision in order to match the product security requirements. Regards, Andrew From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Ken Liu via TF-M Sent: 09 January 2020 09:01 To: tf-m(a)lists.trustedfirmware.org Cc: nd <nd(a)arm.com> Subject: Re: [TF-M] Code Protection between secure services Hi, I assume the main purpose of isolation would be protect the code been seen by the AppRoT. Let's check with the FF author for detailed answers. The building instructions now is just create separate libraries and finally combine them together - since vendors can create Secure Partitions, these modularized building can't be avoided. /Ken From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Reinhard Keil via TF-M Sent: Thursday, January 9, 2020 4:00 PM To: tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Code Protection between secure services I suggest we review the requirement of code isolation on the secure side. R/W data and R/O data should definitely be isolated, but code isolation has implications: * Code cannot be share between services (i.e. no linker optimization to reduce memory footprint) * Sharing library code * Overall the build instructions of the system are more complicated * Adding device specific driver code (i.e. to crypto) can become tricky Reinhard IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

4 years, 9 months

1
0
0 0

Re: [TF-M] TF-M Technical Forum call - January 23

by Anton Komlev

Hello, Just noted (thanks to David) that January 23 is a day ahead of Lunar New Year so we may expect less interest to the forum from Asia. This is an opportunity to make the forum time US friendly on the next session. Preliminary suggest to have it Thursday, January 23rd at 17:00-18:00 UTC. Which a morning time in US. Again, please send your topics in respond to this mail. Experts and developers could be invited to answer specific questions asked in advance. Best regards, Anton Komlev From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Anton Komlev via TF-M Sent: 09 January 2020 11:22 To: TF-M(a)lists.trustedfirmware.org Cc: nd <nd(a)arm.com> Subject: [TF-M] TF-M Technical Forum call - January 23 Dear All, The next session of the Technical Forum is planned in 2 weeks, Thursday, January 23rd at 7:00-8:00 UTC. Please treat this email and an early invitation for agenda topic collection. Any questions, proposals, concerns are all valid points for our open discussion so do not hesitate to share it. A big or complicated topics are worth to preliminary discussed over a mailing list. Best regards, Anton Komlev

4 years, 9 months

1
0
0 0

Re: [TF-M] Create another branch for feature development

by Edison Ai

Thanks Minos for comment. I agree that there is still a small window as you describe in the 2nd case. Maybe this only can be solved by a super maintain who can merged patches to the master branch. I am not sure if Gerrit can be enhanced to do this: it can trigger CI to do the test when it detects one merged patch are not on the TOP when two dev branches are merged at the same time. So that, we still can catch the problem soon even patches are merged to the master branch. I also agree your most of the assumptions except this: "There is no mitigation for scenarios of having individually tested patches merged in a small window, introducing a bug when combined, or having a patch-chain merged, and some of the intermediate patches breaking master but the final patch passing tests." It is why we are discussing if we need to import the development or integration branch to TF-M to help to solve these problems partly. I agree the newly added branch cannot confirm the master branch is stable 100%. But it is significant if it can help improve stability. There are many things we can and maybe we need to do to help to increase the quality of merged patches and help the users to use the master branch more convenient, such as policy, CI, code review, self-test, etc. The new branch is just only one part. Thanks, Edison From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org> On Behalf Of Minos Galanakis via TF-M Sent: Friday, January 10, 2020 6:18 PM To: Edison Ai via TF-M <tf-m(a)lists.trustedfirmware.org> Cc: nd <nd(a)arm.com> Subject: Re: [TF-M] Create another branch for feature development Hi. I am a bit confused on how would the following be addressed by a having a dedicated development branch. Would it be possible to elaborate please? There is another way maybe help to solve this problem, we can define a merge policy like this: The patches cannot be merged to master branch if the patch is not based on the TOP HEAD. 1. If you have dev branches A and B, with common ancestry the HEAD_M , they both have been tested, and are merged with 2 minutes difference, then branch B will be based on A and will not have tested in that way, but will be present in master and assumed stable. 2. If you have dev branches A and B , and we decide as policy that B needs to constantly be based on A ( i.e the newest patch should be based on the last patch under review ). There will always be a still a small window that you can make a change in A, merge it, and B is not updated. Not to account for the significant overhead this will have to development. 3. If we adopt a generic dev branch and merge it to master overnight then this problem does not exist to begin with but the overall development flow will be delayed. The TF-M Merge strategy has the following components: * Master as the common development branch. * Release Tags for stable releases, based on master * Feature branches for development of big changes without affecting the flow of master To my understanding the following assumptions apply: * RC tags are always stable and extensively tested. * Feature branches usually are based on RC tags, and re-based on top of master right before merging them. * Master should be stable, but is not guaranteed to to. * There is no mitigation for scenarios of having individually tested patches merged in a small window, introducing a bug when combined, or having a patch-chain merged, and some of the intermediate patches breaking master but the final patch passing tests. * It is impossible to test everything all the time, while keeping the merge bandwidth at maximum. The CI will have to test a significant amount of platforms, but relies on the developer on having a look on weather his change may affect anything outside of this scope. Since with the current process flow, there is no roll-back strategy, or a merge windows, there is a significant time gain in the time that each patch is held back when in review. Then the assumption is that if it breaks something which has not been tested, the developer will have to commit some extra time to address it. If that trade-off is unacceptable it can easily be addressed with a policy change. In community projects the most commonly accepted solution that seems to mitigate a scenario of patches A, B being merged in a small time delta, is using "merge windows". But before we adopt any new policy we should be evaluating our needs. To that end would it be possible to go back to start and decide on the requirements? What are we trying to achieve? What are we trying to address? What would the acceptable cost in development time be? Regards, Minos ________________________________ From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> on behalf of Edison Ai via TF-M <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>> Sent: 10 January 2020 07:40 To: 'tf-m(a)lists.trustedfirmware.org' <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>> Cc: nd <nd(a)arm.com<mailto:nd@arm.com>> Subject: Re: [TF-M] Create another branch for feature development Hi, Let's continue to discuss this topic. I agree with this, the CI is just a tool, we should not only rely on it. Thanks Gyorgy to point out the quality policy, we need to think about them when we discuss the version control. In the current status, there is only one master branch for most of the development work in TF-M. Of course, we can add a warning to say that the master is constantly changing, there may be some problems in building or test running, and let the user use the release tag. But it is not convenience and impossible if they are upstreaming patches. And even to TF-M developers, we had met several times of the master branch is broken. There is another way maybe help to solve this problem, we can define a merge policy like this: The patches cannot be merged to master branch if the patch is not based on the TOP HEAD. But this is very difficult to follow because there may be several maintainers could and need to merged patches at the same time. They need to align with each other to confirm their patches are on the TOP. But now, we often meet this case, while one patch rebases to TOP and waiting for the CI result, another patch is merged into master. The patch needs to rebase again. I do not want to give more examples here. Of course, it is a big change to involve a new branch, there must be many documents that need to be updated, and some policies need to be changed. And even this needs someone to maintain the alignment with 2 branches. But I think it is more useful and helpful for all users. Thanks, Edison -----Original Message----- From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Gyorgy Szing via TF-M Sent: Friday, December 13, 2019 10:01 PM To: Minos Galanakis <Minos.Galanakis(a)arm.com<mailto:Minos.Galanakis@arm.com>>; Soby Mathew <Soby.Mathew(a)arm.com<mailto:Soby.Mathew@arm.com>>; tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org> Cc: nd <nd(a)arm.com<mailto:nd@arm.com>> Subject: Re: [TF-M] Create another branch for feature development Hi, Hi, I agree, the CI shall not dictate how we use the version control system. It shall adapt. Regarding your suggestions, I think the main problem is we are mixing stuff, this time quality with version control. Before we make decisions we shall understand where we are. The current quality policy is that we only make releases for communication purposes. To give a clean interface for tf-m users and to allow planning their work. Releases allow them to execute their tf-m integration process less frequently. Only for each release or specific releases and not for each commit. The current quality policy identifies a single quality level only, and says any patch we publish is "golden quality", it matches the highest quality standard we can achieve (with sane constraints). Also to make our life easy we decided to use the master branch to hold these patches. At the same time we use the master branch for development. Any change we make is made against master. This means each pull request and thus each review targets master. For review purposes the best is to have a chain of small modifications, otherwise the review content becomes too large to follow. The TF-A "branching strategy" tries to address this issue by introducing an integration branch used for development. This allows master to be more release specific. I suggest to take the following approach (details to be discussed): - introduce more quality levels i.e.: - none: content of a topic branch, or content pushed to review. - bronze: content passed code review and patch specific testing. - silver: content passed a more though daily testing. - gold: a release. A pack of source-code, feature state document (release notes), reviewed documentation (user manual, reference manual), test evidence, documentation of test efforts to allow repeatability. The version control system can be used to store content, and to provide identification info (i.e. tagging), but most likely the release will need other kind of storage to be used (i.e. documentation). - platina: reaching extra quality level trough passing PSA or some FUSA qualification. Or we may simply use extra release for this. Naming the quality levels allows us to have a cleaner definition of what can be expected at a specific level (set of quality measures, i.e. static analysis, code review, test configuration). It would also allow us cleaner communication and to find how we use the version system for quality purposes. I also expect this discussion to help defining how the version system is used for development purposes. The current state works ok, but is a sort of naturally grown. We might have reached the point when more pragmatic approach may be needed. /George -----Original Message----- From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> On Behalf Of Minos Galanakis via TF-M Sent: 13 December 2019 12:23 To: Edison Ai (Arm Technology China) via TF-M <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>>; Soby Mathew <Soby.Mathew(a)arm.com<mailto:Soby.Mathew@arm.com>> Cc: nd <nd(a)arm.com<mailto:nd@arm.com>> Subject: Re: [TF-M] Create another branch for feature development Hi all. My personal comments on this. I would like to point out that the CI is a tool, not the core project. I do not believe we should be changing our development strategy based on what the tool is doing. We should instead adjust the tool to fit our requirements. * No patches should be/ are merged to master when CI fails. If master breaks it should most commonly be because of something we are not testing for. Using an integration branch would not change that. * As a developer I find it more convoluted to work with projects who use different integration strategies. The most common assumption in open source projects is that you have a master branch which is the bleeding edge, but can contain untested bugs, and the release immutable git tags for versioning. Using branch merges as versioning is a design for the pull request model which is not quite compatible with Gerrit. * Most of the CI downtime has nothing to do with the merge strategy, they are more of a chicken and the egg philosophical problem. If your patch or branch introduced a change which affects the tests outputs, how will you test it if the CI expects the old output? An integration branch would not solve the merge freeze periods, would just affect a different branch from master. * I believe feature branches are quite useful, since changes to master do not disrupt the development flow of a big change, and even though they will require some maintenance to re-sync before the final patch , it will be handled by an engineer who knows the feature, and full understands the regression vectors. If I were to suggest some changes for stability purposes, I would start smaller: * Update documentation to instruct users to check out from release tags, warning then that master is constantly changing. * Adjust the CI to detect an Allow-CI flag from every branch. That way developers can test any patch from any feature branch. The logic for that is already present in the code, but requires Gerrit to be configured accordingly. * Add an undo process. This would be the only case for an integration branch. All patches are merged to a temporary branch, after confirming they have passed testing individually. On the once per day nightly test, the group of different patches, will be tested against an extensive job, in models and hardware, and only if successful it will fast forward master to that state. Regards Minos ________________________________ From: TF-M <tf-m-bounces(a)lists.trustedfirmware.org<mailto:tf-m-bounces@lists.trustedfirmware.org>> on behalf of Edison Ai (Arm Technology China) via TF-M <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>> Sent: 13 December 2019 08:55 To: Soby Mathew <Soby.Mathew(a)arm.com<mailto:Soby.Mathew@arm.com>>; 'tf-m(a)lists.trustedfirmware.org' <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>> Cc: nd <nd(a)arm.com<mailto:nd@arm.com>> Subject: Re: [TF-M] Create another branch for feature development Hi Soby, Thanks for your detail description. > Integration is a temporary merge branch to merge several patches and run the CI against. Usually once CI passes, the master will be fast forwarded to integration within a day. > This helps us to test integration of patches and detect any failure before master is updated. This means the master will pass CI at any given merge point. I think it's a good method like this so that we can double confirm the "master" branch is stable. And this also can fix one case even the CI can work normally: one patch is ready to merge, and it is not based on the latest HEAD, but there is no conflict. We can merge the patch directly and let gerrit do rebase by itself. But we cannot confirm the CI test can pass. Any comment for this from others? For multiple feature branches, I think we can stop to discuss about it now until we have some strong demands for it. It is indeed a big change for us now. Thanks, Edison -----Original Message----- From: Soby Mathew <Soby.Mathew(a)arm.com<mailto:Soby.Mathew@arm.com>> Sent: Friday, December 13, 2019 5:14 AM To: Edison Ai (Arm Technology China) <Edison.Ai(a)arm.com<mailto:Edison.Ai@arm.com>>; 'tf-m(a)lists.trustedfirmware.org' <tf-m(a)lists.trustedfirmware.org<mailto:tf-m@lists.trustedfirmware.org>> Cc: nd <nd(a)arm.com<mailto:nd@arm.com>> Subject: Re: [TF-M] Create another branch for feature development On 11/12/2019 09:05, Edison Ai (Arm Technology China) via TF-M wrote: > Hi Gyorgy, > > Thanks to point it out. I agree with you that it will be better if we can align these two projects in this. I had a quick check the branches from TF-A: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/. > There are three branches in TF-A: > - "integration" branch, should be used for new features. > - "master" branch, which is behind of "integration" branch. But I am nor sure what is the strategy to update it. Hi Edison, Integration is a temporary merge branch to merge several patches and run the CI against. Usually once CI passes, the master will be fast forwarded to integration within a day. This helps us to test integration of patches and detect any failure before master is updated. This means the master will pass CI at any given merge point. > - "topics/epic_beta0_spmd", I thinks it should like a feature branch for big feature. > @Soby Mathew Could you help to share more information about it? Thanks very much. We usually do not have feature branches in TF-A. The topics/epic_beta0_spmd is a prototyping branch where we wanted to share code with collaborators outside TF-A. The patches on this branch are not visible in gerrit review and no patches in gerrit review will be merged to this branch. Once the prototyping is complete, then patches on this branch will be reworked and pushed to gerrit review and finally get merged to integration and this branch will be deleted. Our experience have been, long running development branches are generally a maintenance overhead. Merging these development branches before a release may also be risky as some of the changes may have unknown interactions and may become difficult to resolve. The "topic" in gerrit review is effectively a branch. For example, this review: https://review.trustedfirmware.org/#/q/topic:od/debugfs+(status:open+OR+sta… is a set of patches on topic "od/debugfs" and can be treated as development branch. This branch is alive as long as patches are not merged. We need to understand the motivations for the change. Broken CI is an argument but development branches will only exacerbate that problem since we don't know the stability of each of those branches. Also merge conflict will not reduce due to development branches. Its just delaying the merge conflict to a later point. There may be other reasons, but generally it is better to merge sensible patches (+2ed) within a feature even before the feature is complete as it will reduce merge conflicts (we have to ensure testing/build coverage for the patch). These are my thoughts on this. Best Regards Soby Mathew -- TF-M mailing list TF-M(a)lists.trustedfirmware.org<mailto:TF-M@lists.trustedfirmware.org> https://lists.trustedfirmware.org/mailman/listinfo/tf-m -- TF-M mailing list TF-M(a)lists.trustedfirmware.org<mailto:TF-M@lists.trustedfirmware.org> https://lists.trustedfirmware.org/mailman/listinfo/tf-m -- TF-M mailing list TF-M(a)lists.trustedfirmware.org<mailto:TF-M@lists.trustedfirmware.org> https://lists.trustedfirmware.org/mailman/listinfo/tf-m -- TF-M mailing list TF-M(a)lists.trustedfirmware.org<mailto:TF-M@lists.trustedfirmware.org> https://lists.trustedfirmware.org/mailman/listinfo/tf-m

4 years, 9 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

TF-M