Thanks for the info. We tried booting on few different platforms (ACPI-Linux/DT-Linux/TFTF) from our end with Suspend and LTO enabled and boot seems to go through without any problems - so we still can’t reproduce the issue on our end. Could I request the
following info from your end to see if we can spot something?
From: Kummari, Prasad <Prasad.Kummari@amd.com>
Date: Tuesday, 6 January 2026 at 06:34
To: Rohit Mathew <Rohit.Mathew@arm.com>, Sammit Joshi <Sammit.Joshi@arm.com>, scan-admin--- via TF-A <tf-a@lists.trustedfirmware.org>
Cc: Belsare, Akshay <akshay.belsare@amd.com>, Bollapalli, Maheedhar Sai <MaheedharSai.Bollapalli@amd.com>, Simek, Michal <michal.simek@amd.com>
Subject: RE: ZynqMP regression with NUMA_AWARE_PER_CPU changes with ENABLED_LTO : Linux runtime hang due to EL3 re-entry
[AMD Official Use Only - AMD Internal Distribution Only]
Hi Mathew,
Yes, if the CPU_IDLE–related configurations listed below are removed while compiling the kernel image, the hang is no longer observed.
Configs:
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_MULTIPLE_DRIVERS=y
CONFIG_CPU_IDLE_GOV_MENU=y
CONFIG_DT_IDLE_STATES=y
Observed Behavior
- System later triggers an Unhandled Exception in EL3 CPUs enter psci_cpu_suspend(). Debug prints confirm execution reaches the suspend path.
- The affected CPUs enter Reset Catch at EL3 and hang at the bl31_warm_entrypoint()->el3_entrypoint_common pc.
Regards,
Prasad.
From: Rohit Mathew <Rohit.Mathew@arm.com>
Sent: Tuesday, January 6, 2026 4:37 AM
To: Kummari, Prasad <Prasad.Kummari@amd.com>; Sammit Joshi <Sammit.Joshi@arm.com>; scan-admin--- via TF-A <tf-a@lists.trustedfirmware.org>
Cc: Belsare, Akshay <akshay.belsare@amd.com>; Bollapalli, Maheedhar Sai <MaheedharSai.Bollapalli@amd.com>; Simek, Michal <michal.simek@amd.com>
Subject: Re: ZynqMP regression with NUMA_AWARE_PER_CPU changes with ENABLED_LTO : Linux runtime hang due to EL3 re-entry
|
|
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
|
Hi Prasad,
Thanks for your mail. I’m trying to reproduce this at our end. Could I check if you see the boot progressing to shell without the CPU idle dependencies?
Regards,
Rohit
[AMD Official Use Only - AMD Internal Distribution Only]
Hello Maintainers,
We are observing a reproducible runtime regression on the ZynqMP (Cortex-A53) platform after enabling LTO (ENABLED_LTO=1) and merging the changes from the topic NUMA_AWARE_PER_CPU into our integration
branch (https://github.com/ARM-software/arm-trusted-firmware/commit/7303319b3823e9e33748d963e9173f3678aba4da).
Summary of the issue
- Baseline behavior
- Platform: ZynqMP (Cortex-A53)
- Configuration: ENABLED_LTO=1
- Without NUMA_AWARE_PER_CPU: Linux boots and runs stably
- After merging NUMA_AWARE_PER_CPU, Linux boots but hangs during runtime
- During the hang, CPUs are observed to unexpectedly re-enter EL3
- Reverting the NUMA_AWARE_PER_CPU changes restores stable Linux execution
- The issue is reproducible only when NUMA_AWARE_PER_CPU is present
- This clearly identifies NUMA_AWARE_PER_CPU as the regression source
- Suspect with LTO
- With NUMA_AWARE_PER_CPU enabled, LTO breaks the per-CPU base calculation
- BL31 contains hand-written assembly that relies on linker-script symbols (e.g., per-CPU section boundaries)
- Under LTO, symbol placement and retention are no longer guaranteed in the same way, leading to incorrect
per-CPU base computation
- This results in corrupted per-CPU data and subsequent erroneous PSCI suspend behavior (EL3 re-entry)
- CPU idle dependency
- The following kernel configuration options are enabled:
- CONFIG_CPU_IDLE_MULTIPLE_DRIVERS=y
- CONFIG_CPU_IDLE_GOV_MENU=y
- This further suggests the issue is triggered during CPU idle / suspend-resume paths, where correct
per-CPU state handling is critical
Based on the above:
- This is specific to NUMA_AWARE_PER_CPU combined with LTO
- The failure mode points to per-CPU base calculation and PSCI state corruption
- Reverting NUMA_AWARE_PER_CPU fully restores stability on ZynqMP
We wanted to report this issue upstream and seek guidance on:
- Whether NUMA_AWARE_PER_CPU is expected to be LTO-safe on platforms relying on linker-defined per-CPU
sections
- Or if additional constraints / fixes are required for platforms like ZynqMP
We are happy to provide further logs, configuration details, or help to fixes.
Regards,
Prasad Kummari