[AMD Official Use Only - AMD Internal Distribution Only]
Hi Mathew,
Yes, if the CPU_IDLE-related configurations listed below are removed while compiling the kernel image, the hang is no longer observed.
Configs: CONFIG_CPU_IDLE=y CONFIG_CPU_IDLE_MULTIPLE_DRIVERS=y CONFIG_CPU_IDLE_GOV_MENU=y CONFIG_DT_IDLE_STATES=y
Observed Behavior
* System later triggers an Unhandled Exception in EL3 CPUs enter psci_cpu_suspend(). Debug prints confirm execution reaches the suspend path. * The affected CPUs enter Reset Catch at EL3 and hang at the bl31_warm_entrypoint()->el3_entrypoint_common pc.
Regards, Prasad.
From: Rohit Mathew Rohit.Mathew@arm.com Sent: Tuesday, January 6, 2026 4:37 AM To: Kummari, Prasad Prasad.Kummari@amd.com; Sammit Joshi Sammit.Joshi@arm.com; scan-admin--- via TF-A tf-a@lists.trustedfirmware.org Cc: Belsare, Akshay akshay.belsare@amd.com; Bollapalli, Maheedhar Sai MaheedharSai.Bollapalli@amd.com; Simek, Michal michal.simek@amd.com Subject: Re: ZynqMP regression with NUMA_AWARE_PER_CPU changes with ENABLED_LTO : Linux runtime hang due to EL3 re-entry
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Hi Prasad,
Thanks for your mail. I'm trying to reproduce this at our end. Could I check if you see the boot progressing to shell without the CPU idle dependencies?
Regards, Rohit
From: Kummari, Prasad <Prasad.Kummari@amd.commailto:Prasad.Kummari@amd.com> Date: Monday, 5 January 2026 at 18:04 To: Sammit Joshi <Sammit.Joshi@arm.commailto:Sammit.Joshi@arm.com>, Rohit Mathew <Rohit.Mathew@arm.commailto:Rohit.Mathew@arm.com>, scan-admin--- via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> Cc: Belsare, Akshay <akshay.belsare@amd.commailto:akshay.belsare@amd.com>, Bollapalli, Maheedhar Sai <MaheedharSai.Bollapalli@amd.commailto:MaheedharSai.Bollapalli@amd.com>, Simek, Michal <michal.simek@amd.commailto:michal.simek@amd.com> Subject: ZynqMP regression with NUMA_AWARE_PER_CPU changes with ENABLED_LTO : Linux runtime hang due to EL3 re-entry
[AMD Official Use Only - AMD Internal Distribution Only]
Hello Maintainers,
We are observing a reproducible runtime regression on the ZynqMP (Cortex-A53) platform after enabling LTO (ENABLED_LTO=1) and merging the changes from the topic NUMA_AWARE_PER_CPU into our integration branch (https://github.com/ARM-software/arm-trusted-firmware/commit/7303319b3823e9e3...).
Summary of the issue
1. Baseline behavior
* Platform: ZynqMP (Cortex-A53) * Configuration: ENABLED_LTO=1 * Without NUMA_AWARE_PER_CPU: Linux boots and runs stably * After merging NUMA_AWARE_PER_CPU, Linux boots but hangs during runtime * During the hang, CPUs are observed to unexpectedly re-enter EL3 * Re-entry into EL3 should not occur during normal Linux runtime execution and strongly suggests corruption or mismanagement of PSCI and/or per-CPU state(arm-trusted-firmware/lib/per_cpu/aarch64/per_cpu_asm.S at master * ARM-software/arm-trusted-firmwarehttps://github.com/ARM-software/arm-trusted-firmware/blob/master/lib/per_cpu/aarch64/per_cpu_asm.S#L28) * Reverting the NUMA_AWARE_PER_CPU changes restores stable Linux execution * The issue is reproducible only when NUMA_AWARE_PER_CPU is present * This clearly identifies NUMA_AWARE_PER_CPU as the regression source
1. Suspect with LTO
* With NUMA_AWARE_PER_CPU enabled, LTO breaks the per-CPU base calculation * BL31 contains hand-written assembly that relies on linker-script symbols (e.g., per-CPU section boundaries) * Under LTO, symbol placement and retention are no longer guaranteed in the same way, leading to incorrect per-CPU base computation * This results in corrupted per-CPU data and subsequent erroneous PSCI suspend behavior (EL3 re-entry)
1. CPU idle dependency
* The following kernel configuration options are enabled: * CONFIG_CPU_IDLE=y * CONFIG_CPU_IDLE_MULTIPLE_DRIVERS=y * CONFIG_CPU_IDLE_GOV_MENU=y * CONFIG_DT_IDLE_STATES=y * This further suggests the issue is triggered during CPU idle / suspend-resume paths, where correct per-CPU state handling is critical
Based on the above:
* This is specific to NUMA_AWARE_PER_CPU combined with LTO * The failure mode points to per-CPU base calculation and PSCI state corruption * Reverting NUMA_AWARE_PER_CPU fully restores stability on ZynqMP
We wanted to report this issue upstream and seek guidance on:
* Whether NUMA_AWARE_PER_CPU is expected to be LTO-safe on platforms relying on linker-defined per-CPU sections * Or if additional constraints / fixes are required for platforms like ZynqMP
We are happy to provide further logs, configuration details, or help to fixes.
Regards, Prasad Kummari