Hi Yusuf,
I hope you have gone through https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/porting-g...
1. Distinguishing between a cold boot and a warm boot.
2. In the case of a cold boot and the CPU being a secondary CPU, ensuring that the CPU is placed in a platform-specific state until the primary CPU performs the necessary steps to remove it from this state.
3. In the case of a warm boot, ensuring that the CPU jumps to a platform- specific address in the BL31 image in the same processor mode as it was when released from reset.
Secondary cores are kept in TF-A holding pen until primary core makes a request to start secondary core(from OS through PSCI CPU_ON). On receiving this call primary breaks the condition which held secondary. For example, investigate a5ds platform's plat_secondary_cold_boot_setup() & a5ds_pwr_domain_on(). Platform also provides warm_boot_entrypoint (most platform uses bl31_warm_entrypoint) from where secondary starts execution.
Primary core is responsible for platform initialization using platform helper functions mentioned https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/porting-g... (make sure your platform has implemented all the mandatory hooks).
Hope this helps
thanks Manish
________________________________ From: Mohd Yusuf Abdul Hamid via TF-A tf-a@lists.trustedfirmware.org Sent: 07 February 2022 02:32 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Subject: [TF-A] How secondary core(s) move from TF-A into Kernel space using PSCI - 4 x A55 ?
Hi, I have been stuck at this problem for more than a week. Hopefully good folks here can help clarify a few things.
Platform 4x Cortex A55 single cluster.
What I got working: 1. I can boot single core kernel to shell using TFA bl31 Baremetal (bare minimum startup + platform specific SOC enablement, EL3) -> TFA bl31 -> Kernel 2. I added PSCI in DT and can see the hook trigger service and hotplug secondary core in.
Secondary core woke up: 1. Bare minimum startup (skip SOC specific enablement) -> TFA bl31 -> go thru 'plat_secondary_cold_boot_setup' path, using 'RESET_TO_BL31:=1'
Now, I am not sure how from there, the secondary core would jump to: a. If jump to kernel's 'secondary_holding_pen' it looks like it would drop from EL3 -> EL1 and wait (however at this point Core0 is already in cpu_idle) and won't continue a.1 For this case, I am also not sure why I hit "instruction abort" in core1 - from what I read MMU hasnt been set up, which is true. I also wonder at what point MMU is set up for this path in the secondary core?
b. If jump to 'secondary_entry' I believe the core is still in EL3 at this point and I will get an exception at 'set_cpu_boot_mode_flag'
c. If someone can summarize what are the minimum requirements for the secondary core to get set up before jumping to 'secondary_holding_pen'/'secondary_entry' whichever is applicable.
Any pointers would be much appreciated.
ps: I have access to Trace32.
Mohd Yusuf Abdul Hamid
Hi Manish, Thank you so much for your reply. After weekend long hacking - and have gone thru most of the documentation you pointed out - I figured the missing piece. Core1 needs to be in EL1.
So in summary - I want to use: 1. Shared bootcode between Core0 and Core1 to start up. 2. Branch to TFA BL31 with RESET_TO_BL31=1 3. Core0 continue to boot - important piece is that BL31 drops from EL3 to EL1 before jumping to kernel
Now for Core1, the path is slightly different - instead of going thru all Core0 went thru - it branch off to 'plat_secondary_cold_boot_setup' and from there I can put the "entrypoint" from PSCI exception call.
However Core1 is still in EL3 and that caused a problem in kernel space. I did some "googling" and found sample EL3 -> EL1 code for RPi and it "sorts of" working.
Core1 can now go into kernel and virtual memory space but later stuck in 'el1_sync'/'bad_stack' for unknown reasons.
Do you have a pointer to what is the bare minimum requirement for Core1 to setup, stack, MMU? etc? Any reference code would be great.
Thanks again for the help.
Mohd Yusuf Abdul Hamid
On Mon, Feb 7, 2022 at 12:50 PM Manish Pandey2 Manish.Pandey2@arm.com wrote:
Hi Yusuf,
I hope you have gone through https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/porting-g...
Distinguishing between a cold boot and a warm boot. 2.
In the case of a cold boot and the CPU being a secondary CPU, ensuring that the CPU is placed in a platform-specific state until the primary CPU performs the necessary steps to remove it from this state. 3.
In the case of a warm boot, ensuring that the CPU jumps to a platform- specific address in the BL31 image in the same processor mode as it was when released from reset.
Secondary cores are kept in TF-A holding pen until primary core makes a request to start secondary core(from OS through PSCI CPU_ON). On receiving this call primary breaks the condition which held secondary. For example, investigate a5ds platform's plat_secondary_cold_boot_setup() & a5ds_pwr_domain_on(). Platform also provides warm_boot_entrypoint (most platform uses bl31_warm_entrypoint) from where secondary starts execution.
Primary core is responsible for platform initialization using platform helper functions mentioned https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/porting-g... (make sure your platform has implemented all the mandatory hooks).
Hope this helps
thanks Manish
*From:* Mohd Yusuf Abdul Hamid via TF-A tf-a@lists.trustedfirmware.org *Sent:* 07 February 2022 02:32 *To:* tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org *Subject:* [TF-A] How secondary core(s) move from TF-A into Kernel space using PSCI - 4 x A55 ?
Hi, I have been stuck at this problem for more than a week. Hopefully good folks here can help clarify a few things.
Platform 4x Cortex A55 single cluster.
What I got working:
- I can boot single core kernel to shell using TFA bl31 Baremetal (bare minimum startup + platform specific SOC enablement,
EL3) -> TFA bl31 -> Kernel 2. I added PSCI in DT and can see the hook trigger service and hotplug secondary core in.
Secondary core woke up:
- Bare minimum startup (skip SOC specific enablement) -> TFA bl31 -> go
thru 'plat_secondary_cold_boot_setup' path, using 'RESET_TO_BL31:=1'
Now, I am not sure how from there, the secondary core would jump to: a. If jump to kernel's 'secondary_holding_pen' it looks like it would drop from EL3 -> EL1 and wait (however at this point Core0 is already in cpu_idle) and won't continue a.1 For this case, I am also not sure why I hit "instruction abort" in core1 - from what I read MMU hasnt been set up, which is true. I also wonder at what point MMU is set up for this path in the secondary core?
b. If jump to 'secondary_entry' I believe the core is still in EL3 at this point and I will get an exception at 'set_cpu_boot_mode_flag'
c. If someone can summarize what are the minimum requirements for the secondary core to get set up before jumping to 'secondary_holding_pen'/'secondary_entry' whichever is applicable.
Any pointers would be much appreciated.
ps: I have access to Trace32.
Mohd Yusuf Abdul Hamid
tf-a@lists.trustedfirmware.org
-
Manish Pandey2 -
Mohd Yusuf Abdul Hamid