Hi Chris, +David

I think it would be good to make a call on it. Because then we can cleanup at least our platform. If you say it is doxygen I expect there is any tool which can be used for checking. If it is something else that's fine too.

I just want to open this topic to be able to fix our code in a way how it should be.

Thanks, Michal

On 4/3/23 14:13, Chris Kay wrote:

Sorry, I sent my response just as Sandrine did so I'll re-submit it here for the sake of thread continuity:

Hi Michal,

TF-A doesn't currently employ any source-level documentation tool so the documentation style varies across the code-base and we don't enforce any particular style. Doxygen is something that we might want to consider - particularly because it can be integrated directly into the existing Sphinx documentation via Breathe (https://breathe.readthedocs.io/en/latest/ https://breathe.readthedocs.io/en/latest/) - but I think we could only reasonably enforce it for new code.

Chris

---

*From:* Sandrine Bailleux via TF-A tf-a@lists.trustedfirmware.org *Sent:* 03 April 2023 13:11 *To:* Michal Simek monstr@monstr.eu; tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org *Subject:* [TF-A] Re: TF-A code documentation Hi Michal,

Thanks for starting this thread. I'll give my own opinion on this topic.

I think there are areas that could benefit from such documentation generation techniques. A prime example would be the porting guide [1]. I would be favorable to the adoption of some Doxygen-like tool (exact tool TBD) for documenting platform interfaces and auto-generate the porting guide from that.

Maybe generic drivers (GPT, console, I/O, ...) would be good candidates too.

But I would not generalize Doxygen-like comments across the entire code base. This sounds too heavyweight on developers to me with no obvious gain.

Also I think this would not replace design documents [2], only complement them. Not saying you're suggesting to replace them! but I just wanted to point it out.

Regards, Sandrine

[1] https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/porting-g... https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/porting-guide.html [2] https://trustedfirmware-a.readthedocs.io/en/latest/components/index.html https://trustedfirmware-a.readthedocs.io/en/latest/components/index.html

On 4/3/23 13:37, Michal Simek via TF-A wrote:

...

Hi,

I have talked to a couple of people to figure out what TF-A project is using for code documentation. Because I see at least in our platform that our documentation is somewhere between doxygen and kernel-doc but actually with a lot of mismatches. Sanbrine mentioned sending an email to the mailing list to start to have discussion about it.

That's why I want to know the official code documentation format and how we should be checking that everything matches to make sure that documentation is not out of sync from code itself. When this is clear I will ask my team to fix all these issues.

Thanks, Michal

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

Doxygen embeds the documentation structure into the source files and views the full documentation as a single "book". Inline comments must define groups and similar structure. This does not scale well to situations where files are part of multiple projects and hence multiple documentations need to be covered. As an example, CMSIS headers forked into TF-M completely ruined the documentation structure by adding CMSIS groupings. If the files are changed to fix the documentation, then syncing new versions from the original repository become difficult. If files are filtered with scripts during documentation build, script maintenance becomes an issue (cost depends on frequency of updates to filtered source files). If Doxygen is generating XML then further processing may solve the issue, but implementing the machinery might be an expensive task. Not sure if Breathe can address this issue.

I don't think we need to concern ourselves with multiple projects - aside from the fact that we don't vendor any dependencies that use Doxygen, we could simply exclude them anyway.

Doxygen supports markdown formatting which is different compared to reST. Writing documentation becomes more complex as the writer will need to use different syntax for inline and Sphinx documents.

Breathe supports the `@rst`/`@endrst` block for snippets of RST, but I think most in-line source documentation generally does not need the additional complexity of RST in the first place, so the simplicity of Markdown may be of benefit here.

Not sure how Breathe manages merging the formatting. Will it remove all Doxygen specific formatting, or will it translate to reST? Or is there a Sphinx plugin which enables rendering Doxygen markdown and using that solves the issue?

Breathe operates on the generated Doxygen XML (which includes formatting information) and is translated into RST directly.

It would be possible to use Sphinx and Doxygen independently, and to host the two documents independent. The two document could still link to each other although such inks would have to be sparse as keeping the links active would be a manual task.

What would the benefit be of maintaining the documentation outside of ReadTheDocs? Integrating it directly via Breathe would allow us to use the Sphinx reference syntax directly from the architectural documentation, e.g.

:c:func:`bl31_main()`

As an example, check the "moderncmakedomain" plugin [1]. This allows using inline reST comments in cmake scripts. Something similar could be used for C/C++.

This is effectively what Breathe does, only you write your documentation using Doxygen's semantics and they are translated into the Sphinx C domain.

---

I think one place where we might run into trouble with any autogen tool is with preprocessing, as we have a huge amount of conditionally-defined code, a lack of a clear header structure (e.g. think `<platform_def.h>`), and some function prototypes differ depending on preprocessor definitions.

Doxygen allows us to disable preprocessing entirely, but then I believe it would be unable to document preprocessor definitions (I’m unsure of that – you might still be able to do it explicitly). Still, that might be a worthwhile tradeoff.

Certainly I think we want to avoid any substantial maintenance burdens here, including any ad-hoc scripts or infrastructure. I’m hoping that prior art (i.e. vanilla Doxygen + Breathe + Exhale?) can do the heavy lifting for us, even if it only gets us 75% of the way there.

Chris

From: Gyorgy Szing Gyorgy.Szing@arm.com Date: Tuesday, 4 April 2023 at 11:11 To: Michal Simek monstr@monstr.eu, Chris Kay Chris.Kay@arm.com, tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org, Sandrine Bailleux Sandrine.Bailleux@arm.com, david.brown@linaro.org david.brown@linaro.org Cc: nd nd@arm.com Subject: RE: [TF-A] Re: TF-A code documentation Hi,

Sorry for jumping in while not actively working on TF-A. I have some experience with the topic in relation to TF-M (and other projects).

There are a few problems with Doxygen, although my knowledge is a bit lacking and perhaps outdated: - Doxygen embeds the documentation structure into the source files and views the full documentation as a single "book". Inline comments must define groups and similar structure. This does not scale well to situations where files are part of multiple projects and hence multiple documentations need to be covered. As an example, CMSIS headers forked into TF-M completely ruined the documentation structure by adding CMSIS groupings. If the files are changed to fix the documentation, then syncing new versions from the original repository become difficult. If files are filtered with scripts during documentation build, script maintenance becomes an issue (cost depends on frequency of updates to filtered source files). If Doxygen is generating XML then further processing may solve the issue, but implementing the machinery might be an expensive task. Not sure if Breathe can address this issue. - Doxygen supports markdown formatting which is different compared to reST. Writing documentation becomes more complex as the writer will need to use different syntax for inline and Sphinx documents. Not sure how Breathe manages merging the formatting. Will it remove all Doxygen specific formatting, or will it translate to reST? Or is there a Sphinx plugin which enables rendering Doxygen markdown and using that solves the issue? - It would be possible to use Sphinx and Doxygen independently, and to host the two documents independent. The two document could still link to each other although such inks would have to be sparse as keeping the links active would be a manual task.

Last time I was looking at the documentation topic (a few years back) I planned to stick to Sphinx and use the C/C++ domain. (See "domains" documentation here [2]). This solution would not "understand" source code as Doxygen does. It would not be able to identify functions, macro definitions, etc.. automatically, and the writer would have to add the proper reST constructs manually. Keeping function signatures, etc... in sync between documentation and source code would be a manual task. (With a clang backend developing tools understanding C/C++ code is much more easy nowadays, and this could be solved longer term.) Undocumented stuff would be missing from the output. Documentation could be made inline by using reST comments in source files. As an example, check the "moderncmakedomain" plugin [1]. This allows using inline reST comments in cmake scripts. Something similar could be used for C/C++. The way of implementation is a question: - Could the "moderncmakedomain" plugin be "misused" for C/C++ files? If it does nothing more than to filter out inline comments and passing it to Sphinx, it could work. - Should a new plugin be developed? - Or is a python/perl/awk/sed filtering script plus extra documentation build steps the solution? (The filtered content would be saved to temporary files, which could be added to the Sphinx document tree.)

As I mentioned my knowledge is a bit dated and I never actually tries Breathe and Most of my points above might be outdated or simply plain wrong.

Overall, the task to enable inline documentation is not simple. It needs careful investigation, design and implementation.

/George

1: https://github.com/scikit-build/moderncmakedomain 2: http://www.pythondoc.com/sphinx/domains.html

-----Original Message----- From: Michal Simek via TF-A tf-a@lists.trustedfirmware.org Sent: Tuesday, April 4, 2023 8:19 AM To: Chris Kay Chris.Kay@arm.com; tf-a@lists.trustedfirmware.org; Sandrine Bailleux Sandrine.Bailleux@arm.com; david.brown@linaro.org Subject: [TF-A] Re: TF-A code documentation

Hi Chris, +David

I think it would be good to make a call on it. Because then we can cleanup at least our platform. If you say it is doxygen I expect there is any tool which can be used for checking. If it is something else that's fine too.

I just want to open this topic to be able to fix our code in a way how it should be.

Thanks, Michal

On 4/3/23 14:13, Chris Kay wrote:

Sorry, I sent my response just as Sandrine did so I'll re-submit it here for the sake of thread continuity:

Hi Michal,

TF-A doesn't currently employ any source-level documentation tool so the documentation style varies across the code-base and we don't enforce any particular style. Doxygen is something that we might want to consider - particularly because it can be integrated directly into the existing Sphinx documentation via Breathe (https://breathe.readthedocs.io/en/latest/ https://breathe.readthedocs.io/en/latest/) - but I think we could only reasonably enforce it for new code.

Chris

---

---

*From:* Sandrine Bailleux via TF-A tf-a@lists.trustedfirmware.org *Sent:* 03 April 2023 13:11 *To:* Michal Simek monstr@monstr.eu; tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org *Subject:* [TF-A] Re: TF-A code documentation Hi Michal,

Thanks for starting this thread. I'll give my own opinion on this topic.

I think there are areas that could benefit from such documentation generation techniques. A prime example would be the porting guide [1]. I would be favorable to the adoption of some Doxygen-like tool (exact tool TBD) for documenting platform interfaces and auto-generate the porting guide from that.

Maybe generic drivers (GPT, console, I/O, ...) would be good candidates too.

But I would not generalize Doxygen-like comments across the entire code base. This sounds too heavyweight on developers to me with no obvious gain.

Also I think this would not replace design documents [2], only complement them. Not saying you're suggesting to replace them! but I just wanted to point it out.

Regards, Sandrine

[1] https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/por ting-guide.html https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/po rting-guide.html [2] https://trustedfirmware-a.readthedocs.io/en/latest/components/index.ht ml https://trustedfirmware-a.readthedocs.io/en/latest/components/index.h tml

On 4/3/23 13:37, Michal Simek via TF-A wrote:

...

Hi,

I have talked to a couple of people to figure out what TF-A project is using for code documentation. Because I see at least in our platform that our documentation is somewhere between doxygen and kernel-doc but actually with a lot of mismatches. Sanbrine mentioned sending an email to the mailing list to start to have discussion about it.

That's why I want to know the official code documentation format and how we should be checking that everything matches to make sure that documentation is not out of sync from code itself. When this is clear I will ask my team to fix all these issues.

Thanks, Michal

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

-- Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.euhttp://www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Xilinx Microblaze Maintainer of Linux kernel - Xilinx Zynq ARM and ZynqMP/Versal ARM64 SoCs U-Boot custodian - Xilinx Microblaze/Zynq/ZynqMP/Versal/Versal NET SoCs TF-A maintainer - Xilinx ZynqMP/Versal/Versal NET SoCs -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

Hi,

it is definitely process to correct it to the right form which takes time. Right now it is pretty much wild west and without knowing what specification is none can really start to correct it properly. I think that it should be discussed by any project committee (or any project group) and do any decision about it.

Thanks, Michal

On 4/5/23 09:52, Joanna Farley wrote:

From a conceptional perspective I’ve no objections. It seems a sensible idea and I could imagine some CI support could be useful. From the comments from Georgy we would need to think carefully if the tooling becomes difficult to manage.

I would not think for common code we would not be able to update existing code in the short term but I could imagine as we refactor or extend areas we would be able to take this into account and incrementally support.

So what specific proposals do we have so we can discuss the practicalities of putting something in place?

Joanna

*From: *Gyorgy Szing via TF-A tf-a@lists.trustedfirmware.org *Date: *Tuesday, 4 April 2023 at 11:12 *To: *Michal Simek monstr@monstr.eu, Chris Kay Chris.Kay@arm.com, tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org, Sandrine Bailleux Sandrine.Bailleux@arm.com, david.brown@linaro.org david.brown@linaro.org *Cc: *nd nd@arm.com *Subject: *[TF-A] Re: TF-A code documentation

Hi,

Sorry for jumping in while not actively working on TF-A. I have some experience with the topic in relation to TF-M (and other projects).

There are a few problems with Doxygen, although my knowledge is a bit lacking and perhaps outdated:   - Doxygen embeds the documentation structure into the source files and views     the full documentation as a single "book". Inline comments must define     groups and similar structure. This does not scale well to situations where     files are part of multiple projects and hence multiple documentations need     to be covered. As an example, CMSIS headers forked into TF-M completely     ruined the documentation structure by adding CMSIS groupings. If the files     are changed to fix the documentation, then syncing new versions from the     original repository become difficult. If files are filtered with scripts     during documentation build, script maintenance becomes an issue (cost     depends on frequency of updates to filtered source files). If Doxygen is     generating XML then further processing may solve the issue, but implementing     the machinery might be an expensive task. Not sure if Breathe can address     this issue.   - Doxygen supports markdown formatting which is different compared to reST.     Writing documentation becomes more complex as the writer will need to use     different syntax for inline and Sphinx documents. Not sure how Breathe     manages merging the formatting. Will it remove all Doxygen specific     formatting, or will it translate to reST? Or is there a Sphinx plugin which     enables rendering Doxygen markdown and using that solves the issue?   - It would be possible to use Sphinx and Doxygen independently, and to host     the two documents independent. The two document could still link to each     other although such inks would have to be sparse as keeping the links active     would be a manual task.

Last time I was looking at the documentation topic (a few years back) I planned to stick to Sphinx and use the C/C++ domain. (See "domains" documentation here [2]). This solution would not "understand" source code as Doxygen does. It would not be able to identify functions, macro definitions, etc.. automatically, and the writer would have to add the proper reST constructs manually. Keeping function signatures, etc... in sync between documentation and source code would be a manual task. (With a clang backend developing tools understanding C/C++ code is much more easy nowadays, and this could be solved longer term.) Undocumented stuff would be missing from the output. Documentation could be made inline by using reST comments in source files. As an example, check the "moderncmakedomain" plugin [1]. This allows using inline reST comments in cmake scripts. Something similar could be used for C/C++. The way of implementation is a question:   - Could the "moderncmakedomain" plugin be "misused" for C/C++ files? If it     does nothing more than to filter out inline comments and passing it to     Sphinx, it could work.   - Should a new plugin be developed?   - Or is a python/perl/awk/sed filtering script plus extra documentation build     steps the solution? (The filtered content would be saved to temporary files,     which could be added to the Sphinx document tree.)

As I mentioned my knowledge is a bit dated and I never actually tries Breathe and Most of my points above might be outdated or simply plain wrong.

Overall, the task to enable inline documentation is not simple. It needs careful investigation, design and implementation.

/George

1: https://github.com/scikit-build/moderncmakedomain https://github.com/scikit-build/moderncmakedomain 2: http://www.pythondoc.com/sphinx/domains.html http://www.pythondoc.com/sphinx/domains.html

-----Original Message----- From: Michal Simek via TF-A tf-a@lists.trustedfirmware.org Sent: Tuesday, April 4, 2023 8:19 AM To: Chris Kay Chris.Kay@arm.com; tf-a@lists.trustedfirmware.org; Sandrine Bailleux Sandrine.Bailleux@arm.com; david.brown@linaro.org Subject: [TF-A] Re: TF-A code documentation

Hi Chris, +David

I think it would be good to make a call on it. Because then we can cleanup at least our platform. If you say it is doxygen I expect there is any tool which can be used for checking. If it is something else that's fine too.

I just want to open this topic to be able to fix our code in a way how it should be.

Thanks, Michal

On 4/3/23 14:13, Chris Kay wrote:

...

Sorry, I sent my response just as Sandrine did so I'll re-submit it here for the sake of thread continuity:

Hi Michal,

TF-A doesn't currently employ any source-level documentation tool so the documentation style varies across the code-base and we don't enforce any particular style. Doxygen is something that we might want to consider - particularly because it can be integrated directly into the existing Sphinx documentation via Breathe (https://breathe.readthedocs.io/en/latest/ <https://breathe.readthedocs.io/en/latest/

https://breathe.readthedocs.io/en/latest/>) - but I think we could

...

only reasonably enforce it for new code.

Chris

---

---

*From:* Sandrine Bailleux via TF-A tf-a@lists.trustedfirmware.org *Sent:* 03 April 2023 13:11 *To:* Michal Simek monstr@monstr.eu; tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org *Subject:* [TF-A] Re: TF-A code documentation Hi Michal,

Thanks for starting this thread. I'll give my own opinion on this topic.

I think there are areas that could benefit from such documentation generation techniques. A prime example would be the porting guide [1]. I would be favorable to the adoption of some Doxygen-like tool (exact tool TBD) for documenting platform interfaces and auto-generate the porting guide from that.

Maybe generic drivers (GPT, console, I/O, ...) would be good candidates too.

But I would not generalize Doxygen-like comments across the entire code base. This sounds too heavyweight on developers to me with no obvious gain.

Also I think this would not replace design documents [2], only complement them. Not saying you're suggesting to replace them! but I just wanted to point it out.

Regards, Sandrine

[1] https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/por

https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/por

...

ting-guide.html https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/po rting-guide.html [2] https://trustedfirmware-a.readthedocs.io/en/latest/components/index.ht

https://trustedfirmware-a.readthedocs.io/en/latest/components/index.ht

...

ml https://trustedfirmware-a.readthedocs.io/en/latest/components/index.h tml

On 4/3/23 13:37, Michal Simek via TF-A wrote:

...

Hi,

I have talked to a couple of people to figure out what TF-A project is using for code documentation. Because I see at least in our platform that our documentation is somewhere between doxygen and kernel-doc but actually with a lot of mismatches. Sanbrine mentioned sending an email to the mailing list to start to have discussion about it.

That's why I want to know the official code documentation format and how we should be checking that everything matches to make sure that documentation is not out of sync from code itself. When this is clear I will ask my team to fix all these issues.

Thanks, Michal

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

-- Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu http://www.monstr.eup: +42-0-721842854 Maintainer of Linux kernel - Xilinx Microblaze Maintainer of Linux kernel - Xilinx Zynq ARM and ZynqMP/Versal ARM64 SoCs U-Boot custodian - Xilinx Microblaze/Zynq/ZynqMP/Versal/Versal NET SoCs TF-A maintainer - Xilinx ZynqMP/Versal/Versal NET SoCs -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org