Hi Pali

See inline [OD]

Regards, Olivier.

________________________________________ From: Pali Rohár pali@kernel.org Sent: 09 July 2021 14:28 To: Varun Wadekar Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org Subject: Re: [TF-A] Missing CPU workaround warning message

Hello!

It seems that ARM erratas 855873 and 1530924 affect all revisions of A53 CPU, which is in Marvell Armada 3720 platform.

So setting compile time macros

ERRATA_A53_855873 := 1 ERRATA_A53_1530924 := 1

in plat/marvell/armada/a3k/common/a3700_common.mk should be enough?

[OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change.

And why this warning is printed only in debug builds? Does it mean that it affects only when TF-A is build in debug mode? Or warning reporting in currently TF-A is broken and it should be reported also in release build?

[OD] (see earlier answer in this thread, I'm re-formulating here) It's the way it is designed and expected. Errata affect both debug and release builds. It's not printed in release builds as this is considered sensitive information and a possible flaw/attack vector for a product in the field. A maintainer spots the missing errata during development cycle while building in debug mode. When a product is released it is hopefully delivered using release builds and hence the warning not reported for the above reason (even if the errata misses).

On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote:

Hi Pali,

My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported.

I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile.

Hope this helps.

-Varun

-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Pali Rohár via TF-A Sent: Wednesday, July 7, 2021 9:11 PM To: Olivier Deprez Olivier.Deprez@arm.com; Bipin Ravi Bipin.Ravi@arm.com; tf-a@lists.trustedfirmware.org Cc: Konstantin Porotchkin kostap@marvell.com; Marek Behún marek.behun@nic.cz Subject: Re: [TF-A] Missing CPU workaround warning message

External email: Use caution opening links or attachments

Hello! Could somebody from TF-A helps with these two topics? I would really need to know if "missing errata warnings" debug message is some critical and needs to be fixed (and how?) or it is just a debug message and therefore should not be a warning...

On Monday 28 June 2021 17:11:18 Pali Rohár wrote:

...

On Monday 28 June 2021 14:03:06 Olivier Deprez wrote:

...

Hi,

Is the question strictly related to this platform not implementing the mentioned errata (for which a platform change can be emitted)?

Hello! The first question is if this is an issue that CPU workaround is missing. And if yes (which seems to be) how big issue it is? And how to resolve it?

...

Or is it more generally that those "missing errata warnings" are not printed in release mode? Assuming the latter, it looks to me it is the integrator mistake to not include the appropriate mitigations at development phase (hence while using debug mode for building TF-A). Then when the device is deployed (hence most often built for release mode), if this message is printed it is an indication for a malicious agent that such attack vector through mis-implemented errata is possible. So the consequence is possibly even worst than just "missing" to include the errata.

Other TF-Aers (Bipin?) may have other opinions?

And this is a second question. If missing CPU workaround is an issue, should not be it printed also in release build?

Also I see that in release builds are omitted not only messages about missing CPU workarounds, but basically _all_ warning messages. But notice messages are _not_ omitted. Which seems strange as in most cases notice message has lower priority than warning message.

...

Regards, Olivier.

---

From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Pali Rohár via TF-A tf-a@lists.trustedfirmware.org Sent: 28 June 2021 15:36 To: tf-a@lists.trustedfirmware.org Cc: Konstantin Porotchkin; Marek Behún Subject: [TF-A] Missing CPU workaround warning message

Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug mode then at runtime it prints following warning messages:

WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing! WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing!

These lines are not printed in non-debug mode. It is an issue?

TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli sts.trustedfirmware.org%2Fmailman%2Flistinfo%2Ftf-a&data=04%7C01 %7Cvwadekar%40nvidia.com%7Cb3605175f552468740e708d941836783%7C43083d 15727340c1b7db39efd9ccc17a%7C0%7C0%7C637612854914595696%7CUnknown%7C TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJX VCI6Mn0%3D%7C1000&sdata=%2FW6HuFPYQCD5ECIA%2FZZxhm5ti5HYILNlsWTz moJ7L8E%3D&reserved=0

-- TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...

Hi,

See inline [OD]

Regards, Olivier.

________________________________________ From: Pali Rohár pali@kernel.org Sent: 09 July 2021 14:53 To: Olivier Deprez Cc: Varun Wadekar; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org Subject: Re: [TF-A] Missing CPU workaround warning message

Hello!

On Friday 09 July 2021 12:43:33 Olivier Deprez wrote:

Hi Pali

See inline [OD]

Regards, Olivier.

---

From: Pali Rohár pali@kernel.org Sent: 09 July 2021 14:28 To: Varun Wadekar Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org Subject: Re: [TF-A] Missing CPU workaround warning message

Hello!

It seems that ARM erratas 855873 and 1530924 affect all revisions of A53 CPU, which is in Marvell Armada 3720 platform.

So setting compile time macros

ERRATA_A53_855873 := 1
ERRATA_A53_1530924 := 1

in plat/marvell/armada/a3k/common/a3700_common.mk should be enough?

[OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change.

Konstantin (already in email loop) is platform maintainer.

And why this warning is printed only in debug builds? Does it mean that it affects only when TF-A is build in debug mode? Or warning reporting in currently TF-A is broken and it should be reported also in release build?

[OD] (see earlier answer in this thread, I'm re-formulating here) It's the way it is designed and expected. Errata affect both debug and release builds. It's not printed in release builds as this is considered sensitive information and a possible flaw/attack vector for a product in the field. A maintainer spots the missing errata during development cycle while building in debug mode. When a product is released it is hopefully delivered using release builds and hence the warning not reported for the above reason (even if the errata misses).

Ok, thanks for explanation, now I understood it. For me it looks quite strange that it is masked in this way as I have not seen other software to mask similar information but seems that you expect this development cycle... so I have just to accept it.

Anyway, it means that existing end users are not aware of this issue as they do not read this list...

[OD] see https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.ht...

"In a debug build of TF-A, on a CPU that comes out of reset, both BL1 and the runtime firmware (BL31 in AArch64, and BL32 in AArch32) will invoke errata status reporting function, if one exists, for that type of CPU."

On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote:

...

Hi Pali,

My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported.

I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile.

Hope this helps.

-Varun

-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Pali Rohár via TF-A Sent: Wednesday, July 7, 2021 9:11 PM To: Olivier Deprez Olivier.Deprez@arm.com; Bipin Ravi Bipin.Ravi@arm.com; tf-a@lists.trustedfirmware.org Cc: Konstantin Porotchkin kostap@marvell.com; Marek Behún marek.behun@nic.cz Subject: Re: [TF-A] Missing CPU workaround warning message

External email: Use caution opening links or attachments

Hello! Could somebody from TF-A helps with these two topics? I would really need to know if "missing errata warnings" debug message is some critical and needs to be fixed (and how?) or it is just a debug message and therefore should not be a warning...

On Monday 28 June 2021 17:11:18 Pali Rohár wrote:

...

On Monday 28 June 2021 14:03:06 Olivier Deprez wrote:

...

Hi,

Is the question strictly related to this platform not implementing the mentioned errata (for which a platform change can be emitted)?

Hello! The first question is if this is an issue that CPU workaround is missing. And if yes (which seems to be) how big issue it is? And how to resolve it?

...

Or is it more generally that those "missing errata warnings" are not printed in release mode? Assuming the latter, it looks to me it is the integrator mistake to not include the appropriate mitigations at development phase (hence while using debug mode for building TF-A). Then when the device is deployed (hence most often built for release mode), if this message is printed it is an indication for a malicious agent that such attack vector through mis-implemented errata is possible. So the consequence is possibly even worst than just "missing" to include the errata.

Other TF-Aers (Bipin?) may have other opinions?

And this is a second question. If missing CPU workaround is an issue, should not be it printed also in release build?

Also I see that in release builds are omitted not only messages about missing CPU workarounds, but basically _all_ warning messages. But notice messages are _not_ omitted. Which seems strange as in most cases notice message has lower priority than warning message.

...

Regards, Olivier.

---

From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Pali Rohár via TF-A tf-a@lists.trustedfirmware.org Sent: 28 June 2021 15:36 To: tf-a@lists.trustedfirmware.org Cc: Konstantin Porotchkin; Marek Behún Subject: [TF-A] Missing CPU workaround warning message

Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug mode then at runtime it prints following warning messages:

WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing! WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing!

These lines are not printed in non-debug mode. It is an issue?

TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli sts.trustedfirmware.org%2Fmailman%2Flistinfo%2Ftf-a&data=04%7C01 %7Cvwadekar%40nvidia.com%7Cb3605175f552468740e708d941836783%7C43083d 15727340c1b7db39efd9ccc17a%7C0%7C0%7C637612854914595696%7CUnknown%7C TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJX VCI6Mn0%3D%7C1000&sdata=%2FW6HuFPYQCD5ECIA%2FZZxhm5ti5HYILNlsWTz moJ7L8E%3D&reserved=0

-- TF-A mailing list TF-A@lists.trustedfirmware.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.trus...

Hello! Could you help with this issue?

On Friday 27 August 2021 12:52:25 Pali Rohár wrote:

Hello!

On Friday 09 July 2021 14:53:57 Pali Rohár wrote:

...

Hello!

...

From: Pali Rohár pali@kernel.org Sent: 09 July 2021 14:28 To: Varun Wadekar Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org Subject: Re: [TF-A] Missing CPU workaround warning message

Hello!

It seems that ARM erratas 855873 and 1530924 affect all revisions of A53 CPU, which is in Marvell Armada 3720 platform.

So setting compile time macros

ERRATA_A53_855873 := 1
ERRATA_A53_1530924 := 1

in plat/marvell/armada/a3k/common/a3700_common.mk should be enough?

[OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change.

Konstantin (already in email loop) is platform maintainer.

Could somebody with ARM knowledge helps with A53 errata 855873?

According to this ARM document (A53 Software Developers Errata Notice) https://documentation-service.arm.com/static/5fa29fddb209f547eebd361d errata 855873 affects A53 if CPU is connected to some interconnect with system cache or a snoop filter.

Marvell Armada 3720 uses interconnect CCI-400 which according to ARM information does not have a snoop filter. And A53 does not have L3 cache, it has only L2 cache.

What is that "system cache" which is described in above ARM document for errata 855873? L2 or L3 cache? Or something else? And how to check if Marvell Armada 3720 platform has this "system cache"?

Konstantin, do you know something about this A53 errata 855873? And if it affects Armada 3720 platform?

...

...

On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote:

...

Hi Pali,

My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported.

I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile.

...

...

From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Pali Rohár via TF-A tf-a@lists.trustedfirmware.org Sent: 28 June 2021 15:36 To: tf-a@lists.trustedfirmware.org Cc: Konstantin Porotchkin; Marek Behún Subject: [TF-A] Missing CPU workaround warning message

Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug mode then at runtime it prints following warning messages:

WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing! WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing!

These lines are not printed in non-debug mode. It is an issue?

Hi Pali,

Konstantin needs to respond for platform specific questions as the Marcell Platform owner and I see he is copied directly here while Bipin can follow up if you need any more generic Errata guidance. I think for the latter all questions have been answered?

Joanna

On 26/09/2021, 15:04, "TF-A on behalf of Pali Rohár via TF-A" <tf-a-bounces@lists.trustedfirmware.org on behalf of tf-a@lists.trustedfirmware.org> wrote:

Hello! Gentle reminder, could you help us with this issue?

On Monday 06 September 2021 21:32:26 Pali Rohár wrote: > Hello! Could you help with this issue? > > On Friday 27 August 2021 12:52:25 Pali Rohár wrote: > > Hello! > > > > On Friday 09 July 2021 14:53:57 Pali Rohár wrote: > > > Hello! > > > > > > > From: Pali Rohár pali@kernel.org > > > > Sent: 09 July 2021 14:28 > > > > To: Varun Wadekar > > > > Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org > > > > Subject: Re: [TF-A] Missing CPU workaround warning message > > > > > > > > Hello! > > > > > > > > It seems that ARM erratas 855873 and 1530924 affect all revisions of A53 > > > > CPU, which is in Marvell Armada 3720 platform. > > > > > > > > So setting compile time macros > > > > > > > > ERRATA_A53_855873 := 1 > > > > ERRATA_A53_1530924 := 1 > > > > > > > > in plat/marvell/armada/a3k/common/a3700_common.mk should be enough? > > > > > > > > > > > > [OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change. > > > > > > Konstantin (already in email loop) is platform maintainer. > > > > Could somebody with ARM knowledge helps with A53 errata 855873? > > > > According to this ARM document (A53 Software Developers Errata Notice) > > https://documentation-service.arm.com/static/5fa29fddb209f547eebd361d > > errata 855873 affects A53 if CPU is connected to some interconnect with > > system cache or a snoop filter. > > > > Marvell Armada 3720 uses interconnect CCI-400 which according to ARM > > information does not have a snoop filter. And A53 does not have L3 > > cache, it has only L2 cache. > > > > What is that "system cache" which is described in above ARM document for > > errata 855873? L2 or L3 cache? Or something else? And how to check if > > Marvell Armada 3720 platform has this "system cache"? > > > > Konstantin, do you know something about this A53 errata 855873? And if > > it affects Armada 3720 platform? > > > > > > On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote: > > > > > Hi Pali, > > > > > > > > > > My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported. > > > > > > > > > > I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile. > > > > > > > > > > > > From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of > > > > > > > Pali Rohár via TF-A tf-a@lists.trustedfirmware.org > > > > > > > Sent: 28 June 2021 15:36 > > > > > > > To: tf-a@lists.trustedfirmware.org > > > > > > > Cc: Konstantin Porotchkin; Marek Behún > > > > > > > Subject: [TF-A] Missing CPU workaround warning message > > > > > > > > > > > > > > Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug > > > > > > > mode then at runtime it prints following warning messages: > > > > > > > > > > > > > > WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing! > > > > > > > WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing! > > > > > > > > > > > > > > These lines are not printed in non-debug mode. It is an issue? -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a

Hi Pali,

Please find my comments inline below.

--Bipin

-----Original Message----- From: Pali Rohár pali@kernel.org Sent: Sunday, September 26, 2021 9:57 AM To: Joanna Farley Joanna.Farley@arm.com; Konstantin Porotchkin kostap@marvell.com; Bipin Ravi Bipin.Ravi@arm.com Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Marek Behún marek.behun@nic.cz Subject: Re: [TF-A] A53 errata 855873 on Marvell Armada 3720 (Was: Re: Missing CPU workaround warning message)

Hello Joanna! Thank you for response.

Konstantin, could you check if A3720 is affected by that ARM errata 855873?

Bipin or Joanna, in ARM errata 855873 is described "system cache". What kind of cache it is? It is L3 or L2 or some other? Because I do not understand to which cache that errata description refers.

<Bipin Ravi> Cortex A53 implements the optional integrated L2 configurable caches with option for sizes being 128KB, 256KB, 512KB, 1MB, and 2MB. So, L2 is integrated and part of the Cluster if it's included. System Cache in Cortex A53 context is any cache like an L3 cache implemented outside of the Cluster.

On Sunday 26 September 2021 14:49:19 Joanna Farley wrote:

Hi Pali,

Konstantin needs to respond for platform specific questions as the Marcell Platform owner and I see he is copied directly here while Bipin can follow up if you need any more generic Errata guidance. I think for the latter all questions have been answered?

Joanna

On 26/09/2021, 15:04, "TF-A on behalf of Pali Rohár via TF-A" <tf-a-bounces@lists.trustedfirmware.org on behalf of tf-a@lists.trustedfirmware.org> wrote:

Hello! Gentle reminder, could you help us with this issue?

On Monday 06 September 2021 21:32:26 Pali Rohár wrote:
> Hello! Could you help with this issue?
> 
> On Friday 27 August 2021 12:52:25 Pali Rohár wrote:
> > Hello!
> > 
> > On Friday 09 July 2021 14:53:57 Pali Rohár wrote:
> > > Hello!
> > > 
> > > > From: Pali Rohár <pali@kernel.org>
> > > > Sent: 09 July 2021 14:28
> > > > To: Varun Wadekar
> > > > Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org
> > > > Subject: Re: [TF-A] Missing CPU workaround warning message
> > > > 
> > > > Hello!
> > > > 
> > > > It seems that ARM erratas 855873 and 1530924 affect all revisions of A53
> > > > CPU, which is in Marvell Armada 3720 platform.
> > > > 
> > > > So setting compile time macros
> > > > 
> > > >     ERRATA_A53_855873 := 1
> > > >     ERRATA_A53_1530924 := 1
> > > > 
> > > > in plat/marvell/armada/a3k/common/a3700_common.mk should be enough?
> > > > 
> > > > 
> > > > [OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change.
> > > 
> > > Konstantin (already in email loop) is platform maintainer.
> > 
> > Could somebody with ARM knowledge helps with A53 errata 855873?
> > 
> > According to this ARM document (A53 Software Developers Errata Notice)
> > https://documentation-service.arm.com/static/5fa29fddb209f547eebd361d
> > errata 855873 affects A53 if CPU is connected to some interconnect with
> > system cache or a snoop filter.
> > 
> > Marvell Armada 3720 uses interconnect CCI-400 which according to ARM
> > information does not have a snoop filter. And A53 does not have L3
> > cache, it has only L2 cache.
> > 
> > What is that "system cache" which is described in above ARM document for
> > errata 855873? L2 or L3 cache? Or something else? And how to check if
> > Marvell Armada 3720 platform has this "system cache"?
> > 
> > Konstantin, do you know something about this A53 errata 855873? And if
> > it affects Armada 3720 platform?
> > 
> > > > On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote:
> > > > > Hi Pali,
> > > > >
> > > > > My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported.
> > > > >
> > > > > I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile.
> > > > >
> > > > > > > From: TF-A <tf-a-bounces@lists.trustedfirmware.org> on behalf of
> > > > > > > Pali Rohár via TF-A <tf-a@lists.trustedfirmware.org>
> > > > > > > Sent: 28 June 2021 15:36
> > > > > > > To: tf-a@lists.trustedfirmware.org
> > > > > > > Cc: Konstantin Porotchkin; Marek Behún
> > > > > > > Subject: [TF-A] Missing CPU workaround warning message
> > > > > > >
> > > > > > > Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug
> > > > > > > mode then at runtime it prints following warning messages:
> > > > > > >
> > > > > > >   WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing!
> > > > > > >   WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing!
> > > > > > >
> > > > > > > These lines are not printed in non-debug mode. It is an issue?
-- 
TF-A mailing list
TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a

Hello Bipin! Thank you for reply. I have one more question, see below.

On Tuesday 28 September 2021 06:10:47 Bipin Ravi wrote:

Hi Pali,

Please find my comments inline below.

--Bipin

-----Original Message----- From: Pali Rohár pali@kernel.org Sent: Sunday, September 26, 2021 9:57 AM To: Joanna Farley Joanna.Farley@arm.com; Konstantin Porotchkin kostap@marvell.com; Bipin Ravi Bipin.Ravi@arm.com Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Marek Behún marek.behun@nic.cz Subject: Re: [TF-A] A53 errata 855873 on Marvell Armada 3720 (Was: Re: Missing CPU workaround warning message)

Hello Joanna! Thank you for response.

Konstantin, could you check if A3720 is affected by that ARM errata 855873?

Bipin or Joanna, in ARM errata 855873 is described "system cache". What kind of cache it is? It is L3 or L2 or some other? Because I do not understand to which cache that errata description refers.

<Bipin Ravi> Cortex A53 implements the optional integrated L2 configurable caches with option for sizes being 128KB, 256KB, 512KB, 1MB, and 2MB. So, L2 is integrated and part of the Cluster if it's included. System Cache in Cortex A53 context is any cache like an L3 cache implemented outside of the Cluster.

In errata 855873 is written:

"Implications If the processor is connected to an interconnect that has a system cache or a snoop filter then this erratum might cause data corruption."

Does it mean if A53 processor does not have snoop filter and it also does not have system cache then it is not affected by this errata?

I'm asking because we have CCI-400 interconnect which IIRC does not have snoop filter. And we do not have L3 cache and based on your description we should not have any "system cache".

So is then this platform affected by errata 855873?

On Sunday 26 September 2021 14:49:19 Joanna Farley wrote:

...

Hi Pali,

Konstantin needs to respond for platform specific questions as the Marcell Platform owner and I see he is copied directly here while Bipin can follow up if you need any more generic Errata guidance. I think for the latter all questions have been answered?

Joanna

On 26/09/2021, 15:04, "TF-A on behalf of Pali Rohár via TF-A" <tf-a-bounces@lists.trustedfirmware.org on behalf of tf-a@lists.trustedfirmware.org> wrote:

Hello! Gentle reminder, could you help us with this issue?

On Monday 06 September 2021 21:32:26 Pali Rohár wrote:
> Hello! Could you help with this issue?
> 
> On Friday 27 August 2021 12:52:25 Pali Rohár wrote:
> > Hello!
> > 
> > On Friday 09 July 2021 14:53:57 Pali Rohár wrote:
> > > Hello!
> > > 
> > > > From: Pali Rohár <pali@kernel.org>
> > > > Sent: 09 July 2021 14:28
> > > > To: Varun Wadekar
> > > > Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org
> > > > Subject: Re: [TF-A] Missing CPU workaround warning message
> > > > 
> > > > Hello!
> > > > 
> > > > It seems that ARM erratas 855873 and 1530924 affect all revisions of A53
> > > > CPU, which is in Marvell Armada 3720 platform.
> > > > 
> > > > So setting compile time macros
> > > > 
> > > >     ERRATA_A53_855873 := 1
> > > >     ERRATA_A53_1530924 := 1
> > > > 
> > > > in plat/marvell/armada/a3k/common/a3700_common.mk should be enough?
> > > > 
> > > > 
> > > > [OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change.
> > > 
> > > Konstantin (already in email loop) is platform maintainer.
> > 
> > Could somebody with ARM knowledge helps with A53 errata 855873?
> > 
> > According to this ARM document (A53 Software Developers Errata Notice)
> > https://documentation-service.arm.com/static/5fa29fddb209f547eebd361d
> > errata 855873 affects A53 if CPU is connected to some interconnect with
> > system cache or a snoop filter.
> > 
> > Marvell Armada 3720 uses interconnect CCI-400 which according to ARM
> > information does not have a snoop filter. And A53 does not have L3
> > cache, it has only L2 cache.
> > 
> > What is that "system cache" which is described in above ARM document for
> > errata 855873? L2 or L3 cache? Or something else? And how to check if
> > Marvell Armada 3720 platform has this "system cache"?
> > 
> > Konstantin, do you know something about this A53 errata 855873? And if
> > it affects Armada 3720 platform?
> > 
> > > > On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote:
> > > > > Hi Pali,
> > > > >
> > > > > My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported.
> > > > >
> > > > > I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile.
> > > > >
> > > > > > > From: TF-A <tf-a-bounces@lists.trustedfirmware.org> on behalf of
> > > > > > > Pali Rohár via TF-A <tf-a@lists.trustedfirmware.org>
> > > > > > > Sent: 28 June 2021 15:36
> > > > > > > To: tf-a@lists.trustedfirmware.org
> > > > > > > Cc: Konstantin Porotchkin; Marek Behún
> > > > > > > Subject: [TF-A] Missing CPU workaround warning message
> > > > > > >
> > > > > > > Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug
> > > > > > > mode then at runtime it prints following warning messages:
> > > > > > >
> > > > > > >   WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing!
> > > > > > >   WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing!
> > > > > > >
> > > > > > > These lines are not printed in non-debug mode. It is an issue?
-- 
TF-A mailing list
TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a

Hi Pali,

Bipin who looks after our Errata work is OoO at the moment back on the 7th November, however I've included John who also implements errata mitigations who may be able to help. I guess how the summary of the errata is written and your description of the platform it may be not applicable. However our folks who work on the errata mitigations consult with our CPU team who define them and the mitigations so can take a closer look.

Cheers

Joanna

On 30/10/2021, 12:58, "Pali Rohár" pali@kernel.org wrote:

Hello Bipin! Hello Joanna! Could you look at the question from previous email about 855873?

On Tuesday 05 October 2021 16:24:10 Pali Rohár wrote: > Hello Bipin! Thank you for reply. I have one more question, see below. > > On Tuesday 28 September 2021 06:10:47 Bipin Ravi wrote: > > Hi Pali, > > > > Please find my comments inline below. > > > > --Bipin > > > > -----Original Message----- > > From: Pali Rohár pali@kernel.org > > Sent: Sunday, September 26, 2021 9:57 AM > > To: Joanna Farley Joanna.Farley@arm.com; Konstantin Porotchkin kostap@marvell.com; Bipin Ravi Bipin.Ravi@arm.com > > Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Marek Behún marek.behun@nic.cz > > Subject: Re: [TF-A] A53 errata 855873 on Marvell Armada 3720 (Was: Re: Missing CPU workaround warning message) > > > > Hello Joanna! Thank you for response. > > > > Konstantin, could you check if A3720 is affected by that ARM errata 855873? > > > > Bipin or Joanna, in ARM errata 855873 is described "system cache". What kind of cache it is? It is L3 or L2 or some other? Because I do not understand to which cache that errata description refers. > > > > <Bipin Ravi> Cortex A53 implements the optional integrated L2 configurable caches with option for sizes being 128KB, 256KB, 512KB, 1MB, and 2MB. So, L2 is integrated and part of the Cluster if it's included. System Cache in Cortex A53 context is any cache like an L3 cache implemented outside of the Cluster. > > In errata 855873 is written: > > "Implications If the processor is connected to an interconnect that has a system cache or a snoop filter then this erratum might cause data corruption." > > Does it mean if A53 processor does not have snoop filter and it also > does not have system cache then it is not affected by this errata? > > I'm asking because we have CCI-400 interconnect which IIRC does not > have snoop filter. And we do not have L3 cache and based on your > description we should not have any "system cache". > > So is then this platform affected by errata 855873? > > > On Sunday 26 September 2021 14:49:19 Joanna Farley wrote: > > > Hi Pali, > > > > > > Konstantin needs to respond for platform specific questions as the Marcell Platform owner and I see he is copied directly here while Bipin can follow up if you need any more generic Errata guidance. I think for the latter all questions have been answered? > > > > > > Joanna > > > > > > On 26/09/2021, 15:04, "TF-A on behalf of Pali Rohár via TF-A" <tf-a-bounces@lists.trustedfirmware.org on behalf of tf-a@lists.trustedfirmware.org> wrote: > > > > > > Hello! Gentle reminder, could you help us with this issue? > > > > > > On Monday 06 September 2021 21:32:26 Pali Rohár wrote: > > > > Hello! Could you help with this issue? > > > > > > > > On Friday 27 August 2021 12:52:25 Pali Rohár wrote: > > > > > Hello! > > > > > > > > > > On Friday 09 July 2021 14:53:57 Pali Rohár wrote: > > > > > > Hello! > > > > > > > > > > > > > From: Pali Rohár pali@kernel.org > > > > > > > Sent: 09 July 2021 14:28 > > > > > > > To: Varun Wadekar > > > > > > > Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org > > > > > > > Subject: Re: [TF-A] Missing CPU workaround warning message > > > > > > > > > > > > > > Hello! > > > > > > > > > > > > > > It seems that ARM erratas 855873 and 1530924 affect all revisions of A53 > > > > > > > CPU, which is in Marvell Armada 3720 platform. > > > > > > > > > > > > > > So setting compile time macros > > > > > > > > > > > > > > ERRATA_A53_855873 := 1 > > > > > > > ERRATA_A53_1530924 := 1 > > > > > > > > > > > > > > in plat/marvell/armada/a3k/common/a3700_common.mk should be enough? > > > > > > > > > > > > > > > > > > > > > [OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change. > > > > > > > > > > > > Konstantin (already in email loop) is platform maintainer. > > > > > > > > > > Could somebody with ARM knowledge helps with A53 errata 855873? > > > > > > > > > > According to this ARM document (A53 Software Developers Errata Notice) > > > > > https://documentation-service.arm.com/static/5fa29fddb209f547eebd361d > > > > > errata 855873 affects A53 if CPU is connected to some interconnect with > > > > > system cache or a snoop filter. > > > > > > > > > > Marvell Armada 3720 uses interconnect CCI-400 which according to ARM > > > > > information does not have a snoop filter. And A53 does not have L3 > > > > > cache, it has only L2 cache. > > > > > > > > > > What is that "system cache" which is described in above ARM document for > > > > > errata 855873? L2 or L3 cache? Or something else? And how to check if > > > > > Marvell Armada 3720 platform has this "system cache"? > > > > > > > > > > Konstantin, do you know something about this A53 errata 855873? And if > > > > > it affects Armada 3720 platform? > > > > > > > > > > > > On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote: > > > > > > > > Hi Pali, > > > > > > > > > > > > > > > > My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported. > > > > > > > > > > > > > > > > I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile. > > > > > > > > > > > > > > > > > > From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of > > > > > > > > > > Pali Rohár via TF-A tf-a@lists.trustedfirmware.org > > > > > > > > > > Sent: 28 June 2021 15:36 > > > > > > > > > > To: tf-a@lists.trustedfirmware.org > > > > > > > > > > Cc: Konstantin Porotchkin; Marek Behún > > > > > > > > > > Subject: [TF-A] Missing CPU workaround warning message > > > > > > > > > > > > > > > > > > > > Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug > > > > > > > > > > mode then at runtime it prints following warning messages: > > > > > > > > > > > > > > > > > > > > WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing! > > > > > > > > > > WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing! > > > > > > > > > > > > > > > > > > > > These lines are not printed in non-debug mode. It is an issue? > > > -- > > > TF-A mailing list > > > TF-A@lists.trustedfirmware.org > > > https://lists.trustedfirmware.org/mailman/listinfo/tf-a > > >

Hi Pali,

Please find the reply inline below.

--Bipin

-----Original Message----- From: Pali Rohár pali@kernel.org Sent: Tuesday, October 5, 2021 9:24 AM To: Bipin Ravi Bipin.Ravi@arm.com Cc: Joanna Farley Joanna.Farley@arm.com; Konstantin Porotchkin kostap@marvell.com; Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Marek Behún marek.behun@nic.cz Subject: Re: [TF-A] A53 errata 855873 on Marvell Armada 3720 (Was: Re: Missing CPU workaround warning message)

Hello Bipin! Thank you for reply. I have one more question, see below.

On Tuesday 28 September 2021 06:10:47 Bipin Ravi wrote:

Hi Pali,

Please find my comments inline below.

--Bipin

-----Original Message----- From: Pali Rohár pali@kernel.org Sent: Sunday, September 26, 2021 9:57 AM To: Joanna Farley Joanna.Farley@arm.com; Konstantin Porotchkin kostap@marvell.com; Bipin Ravi Bipin.Ravi@arm.com Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Marek Behún marek.behun@nic.cz Subject: Re: [TF-A] A53 errata 855873 on Marvell Armada 3720 (Was: Re: Missing CPU workaround warning message)

Hello Joanna! Thank you for response.

Konstantin, could you check if A3720 is affected by that ARM errata 855873?

Bipin or Joanna, in ARM errata 855873 is described "system cache". What kind of cache it is? It is L3 or L2 or some other? Because I do not understand to which cache that errata description refers.

<Bipin Ravi> Cortex A53 implements the optional integrated L2 configurable caches with option for sizes being 128KB, 256KB, 512KB, 1MB, and 2MB. So, L2 is integrated and part of the Cluster if it's included. System Cache in Cortex A53 context is any cache like an L3 cache implemented outside of the Cluster.

In errata 855873 is written:

"Implications If the processor is connected to an interconnect that has a system cache or a snoop filter then this erratum might cause data corruption."

Does it mean if A53 processor does not have snoop filter and it also does not have system cache then it is not affected by this errata?

I'm asking because we have CCI-400 interconnect which IIRC does not have snoop filter. And we do not have L3 cache and based on your description we should not have any "system cache".

So is then this platform affected by errata 855873?

<Bipin Ravi> The implications for the errata says " If the processor is connected to an interconnect that has a system cache or a snoop filter then this erratum might cause data corruption."

The emphasis is on the interconnect having a system cache OR snoop filter. I talked to the support team and they mentioned there is no SF(Snoop Filter) or SLC(System Level Cache) in CCI-400. There is one in CCI-550(and maybe CCI-500), both doesn’t apply for this case.

So, I would say this errata doesn't apply for the above said platform configuration. Hope it clarifies.

On Sunday 26 September 2021 14:49:19 Joanna Farley wrote:

...

Hi Pali,

Konstantin needs to respond for platform specific questions as the Marcell Platform owner and I see he is copied directly here while Bipin can follow up if you need any more generic Errata guidance. I think for the latter all questions have been answered?

Joanna

On 26/09/2021, 15:04, "TF-A on behalf of Pali Rohár via TF-A" <tf-a-bounces@lists.trustedfirmware.org on behalf of tf-a@lists.trustedfirmware.org> wrote:

Hello! Gentle reminder, could you help us with this issue?

On Monday 06 September 2021 21:32:26 Pali Rohár wrote:
> Hello! Could you help with this issue?
> 
> On Friday 27 August 2021 12:52:25 Pali Rohár wrote:
> > Hello!
> > 
> > On Friday 09 July 2021 14:53:57 Pali Rohár wrote:
> > > Hello!
> > > 
> > > > From: Pali Rohár <pali@kernel.org>
> > > > Sent: 09 July 2021 14:28
> > > > To: Varun Wadekar
> > > > Cc: Olivier Deprez; Bipin Ravi; Konstantin Porotchkin; Marek Behún; tf-a@lists.trustedfirmware.org
> > > > Subject: Re: [TF-A] Missing CPU workaround warning message
> > > > 
> > > > Hello!
> > > > 
> > > > It seems that ARM erratas 855873 and 1530924 affect all revisions of A53
> > > > CPU, which is in Marvell Armada 3720 platform.
> > > > 
> > > > So setting compile time macros
> > > > 
> > > >     ERRATA_A53_855873 := 1
> > > >     ERRATA_A53_1530924 := 1
> > > > 
> > > > in plat/marvell/armada/a3k/common/a3700_common.mk should be enough?
> > > > 
> > > > 
> > > > [OD] In general it's a platform maintainer duty to analyse applicable errata from the software developer notice (SDEN) and judge. Yes it looks reasonable to enable those but make sure it doesn't affect other SW stacks (e.g. kernel) by enabling those in the firmware, for any platform which uses this chipset. You can poll through the ML when doing this change.
> > > 
> > > Konstantin (already in email loop) is platform maintainer.
> > 
> > Could somebody with ARM knowledge helps with A53 errata 855873?
> > 
> > According to this ARM document (A53 Software Developers Errata Notice)
> > https://documentation-service.arm.com/static/5fa29fddb209f547eebd361d
> > errata 855873 affects A53 if CPU is connected to some interconnect with
> > system cache or a snoop filter.
> > 
> > Marvell Armada 3720 uses interconnect CCI-400 which according to ARM
> > information does not have a snoop filter. And A53 does not have L3
> > cache, it has only L2 cache.
> > 
> > What is that "system cache" which is described in above ARM document for
> > errata 855873? L2 or L3 cache? Or something else? And how to check if
> > Marvell Armada 3720 platform has this "system cache"?
> > 
> > Konstantin, do you know something about this A53 errata 855873? And if
> > it affects Armada 3720 platform?
> > 
> > > > On Wednesday 07 July 2021 21:59:25 Varun Wadekar wrote:
> > > > > Hi Pali,
> > > > >
> > > > > My understanding of the errata reporting mechanism is that some erratas are always checked during CPU boot. If the corresponding MACRO (ERRATA_A53_*) is disabled, then the ERRATA_MISSING code is reported.
> > > > >
> > > > > I would be concerned if the CPU is affected by the errata. If the errata needs to be enabled, the fix would be to enable the ERRATA_A53_* from the platform makefile.
> > > > >
> > > > > > > From: TF-A <tf-a-bounces@lists.trustedfirmware.org> on behalf of
> > > > > > > Pali Rohár via TF-A <tf-a@lists.trustedfirmware.org>
> > > > > > > Sent: 28 June 2021 15:36
> > > > > > > To: tf-a@lists.trustedfirmware.org
> > > > > > > Cc: Konstantin Porotchkin; Marek Behún
> > > > > > > Subject: [TF-A] Missing CPU workaround warning message
> > > > > > >
> > > > > > > Hello! If TF-A for Marvell Armada 3720 platform is compiled in debug
> > > > > > > mode then at runtime it prints following warning messages:
> > > > > > >
> > > > > > >   WARNING: BL1: cortex_a53: CPU workaround for 855873 was missing!
> > > > > > >   WARNING: BL1: cortex_a53: CPU workaround for 1530924 was missing!
> > > > > > >
> > > > > > > These lines are not printed in non-debug mode. It is an issue?
-- 
TF-A mailing list
TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a