New subject: Python script to generate tf-m dependencies in JSON format

23 Oct 2019


      Hi,
Thanks for your inputs.
The original intention of the script was to not to modify the origin of the information i.e rst file as Minos quoted and also to avoid duplication of information.  Considering all the new information, I think we should try to separate build and post-build dependencies (ex: srec_cat). This separation will allow us to handle all the build dependencies using "cmake" and then we can explore the possible option on how to handle post-build dependencies.
@Minos,
I believe all sw dependencies defined in https://ci.trustedfirmware.org/job/tf-m-build-test-nightly/lastSuccessfulBui... are already validated in cmake. Adding other library dependencies into it will make it complete. This way we don’t need mention library dependency in document anymore and still achieve single source of information.
@George,
I think first 3 question can be answered if we decide to go ahead with handling build dependencies with cmake.
Thanks,
Dev
On 21/10/2019, 15:56, "TF-M on behalf of Gyorgy Szing via TF-M" <tf-m-bounces@lists.trustedfirmware.org on behalf of tf-m@lists.trustedfirmware.org> wrote:
Hi,
We need to keep an eye on some factors which the current prototype ignores. Some that come to my mind:
1. which component shall be the owner of sw dependency info? (Documentation, cmake, something else?).
    2. how does the solution scale? (i.e. be able to handle platform specific dependencies). 
    3. how we handle build configuration specific dependencies? (i.e. if I don't build a service then some dependency is not needed).
    4. As Minos mentioned some environment verification is already done by CMake. Is it worth to extract all dependency info (tooling + SW) into a dedicated place?
/George
-----Original Message-----
    From: TF-M tf-m-bounces@lists.trustedfirmware.org On Behalf Of Minos Galanakis via TF-M
    Sent: 21 October 2019 16:47
    To: tf-m@lists.trustedfirmware.org
    Cc: nd nd@arm.com
    Subject: Re: [TF-M] Python script to generate tf-m dependencies in JSON format
Hi,
Making CMAKE able to output the dependencies it is expecting is the quickest path, but it is only able to resolve a subset of the requirements captured by the documenation.  .https://ci.trustedfirmware.org/job/tf-m-build-test-nightly/lastSuccessfulBuild/artifact/build-docs/tf-m_documents/install/doc/user_guide/html/docs/user_guides/tfm_sw_requirement.html For example the cmake version itself, or tools like srec_cat used to produce MUSCA binaries. There are certain dependencies that are provided by other means ( package managers ) .
So while I agree that it looks quite odd, it is hierarchically the origin of the information we are trying to capture.
Minos
    ________________________________
    From: TF-M tf-m-bounces@lists.trustedfirmware.org on behalf of Kumar Gala via TF-M tf-m@lists.trustedfirmware.org
    Sent: 21 October 2019 15:25
    To: Devaraj Ranganna Devaraj.Ranganna@arm.com
    Cc: tf-m@lists.trustedfirmware.org tf-m@lists.trustedfirmware.org
    Subject: Re: [TF-M] Python script to generate tf-m dependencies in JSON format
This feels backwards, parsing rst to determine build steps, dependancies doesn’t feel correct.  I think it would be better to codify the build steps in scripts of CMake, etc than reference those in the .rst instead.
- k
> On Oct 21, 2019, at 9:03 AM, Devaraj Ranganna via TF-M tf-m@lists.trustedfirmware.org wrote:
    >
    > Hi,
    >
    > Currently tf-m dependencies are only listed in documentation file (docs/user_guides/tfm_build_instruction.rst) and there is no option to programmatically retrieve it. In order to integrate tf-m into non-secure side RTOSes, it is very essential to be able build tf-m automatically without manual intervention (may be as part of CI).
    >
    > The tf-m CI build system uses fixed values for dependencies and it is the responsibility of developers who update the dependencies to update the documentation and the CI build system.
    >
    > If non-secure side RTOSes use fix values for tf-m dependencies then either build or regression tests may fail when tf-m updates dependencies. In order avoid this issue, I’m proposing the following
    >
    >
    >  *   Use a reStructuredText grid table to define dependencies
    >  *   Python script to produce a json file in the root directory when invoked as standalone script or to return a json string when imported into another python script
    >
    > I’ve created a patch set (https://review.trustedfirmware.org/c/trusted-firmware-m/+/2333) with above changes. I’d appreciate your inputs/feedback on this proposal.
    >
    > This will also ensure that dependencies are captured in one place (docs/user_guides/tfm_build_instruction.rst).
    >
    > Additionally, have another python script as “post-checkout” git hook which can parse dependencies in JSON format and clone them automatically when tf-m repository is cloned or when switching from master to feature branch and vice versa.
    >
    > Thanks,
    > Dev
    > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
    > --
    > TF-M mailing list
    > TF-M@lists.trustedfirmware.org
    > https://lists.trustedfirmware.org/mailman/listinfo/tf-m
--
    TF-M mailing list
    TF-M@lists.trustedfirmware.org
    https://lists.trustedfirmware.org/mailman/listinfo/tf-m
    -- 
    TF-M mailing list
    TF-M@lists.trustedfirmware.org
    https://lists.trustedfirmware.org/mailman/listinfo/tf-m
    -- 
    TF-M mailing list
    TF-M@lists.trustedfirmware.org
    https://lists.trustedfirmware.org/mailman/listinfo/tf-m