On 24.11.19 16:08, Jan Kiszka wrote:
On 24.11.19 15:49, Jan Kiszka wrote:
On 24.11.19 15:19, André Przywara wrote:
On 24/11/2019 13:21, Jan Kiszka wrote:
Hi,
I'm trying to get CPU offlining work for Linux on the RPi4. In fact, this is working already with current ATF master and the Raspberry kernel 4.19.85. Not it would be "nice" if onlining worked as well:
Indeed.
[ 94.959294] CPU1: shutdown [ 94.959311] psci: CPU1 killed. [ 106.750960] CPU1: failed to come online [ 106.755425] CPU1: failed in unknown state : 0x0
Yeah, I remember there was an issue with that, but I didn't come around to fix that yet. I would have put this as a limitation in the cover letter, but: gerrit ...
Is this a known limitation? Or am I using the wrong kernel (obviously, it's a downstream one ;) )?
I obviously cannot endorse the usage of downstream kernels ;-), but I think I saw this with mainline as well.
Is mainline working on the RPi4 already? Well, another topic...
Do you get any more messages on the console? I think RPI3_RUNTIME_UART defaults to 0 for the RPI4 (more by accident), which should give you TF-A debug messages on PSCI calls. Maybe you can quickly add some, to see where it breaks?
I've instrumented the psci cpu off/on paths, and the enabling path after disabling never gets us into psci_warmboot_entrypoint() at least.
I might dig out my RPI4, but only later next week, so you might want to beat me on it, if you have your rig set up already.
I this working on the RPi3? I don't see much deviations between both in this regard.
Likely not: rpi3_pwr_domain_off doesn't look to me like it is seriously pulling any plug. And rpi3_pwr_domain_on expect the target to already spin in plat_secondary_cold_boot_setup, in reset state.
Here is a hack to make onlining - apparently - work:
diff --git a/plat/rpi/common/rpi3_pm.c b/plat/rpi/common/rpi3_pm.c index 8c2d070c4..2a6bf076b 100644 --- a/plat/rpi/common/rpi3_pm.c +++ b/plat/rpi/common/rpi3_pm.c @@ -123,6 +123,15 @@ static void rpi3_pwr_domain_off(const psci_power_state_t *target_state) #endif }
+void __dead2 plat_secondary_cold_boot_setup(void);
+static void __dead2 +rpi3_pwr_domain_pwr_down_wfi(const psci_power_state_t *target_state) +{
- disable_mmu_el3();
- plat_secondary_cold_boot_setup();
+}
/*******************************************************************************
- Platform handler called when a power domain is about to be turned on. The
- mpidr determines the CPU to be turned on.
@@ -224,6 +233,7 @@ static void __dead2 rpi3_system_off(void) static const plat_psci_ops_t plat_rpi3_psci_pm_ops = { .cpu_standby = rpi3_cpu_standby, .pwr_domain_off = rpi3_pwr_domain_off,
- .pwr_domain_pwr_down_wfi = rpi3_pwr_domain_pwr_down_wfi, .pwr_domain_on = rpi3_pwr_domain_on, .pwr_domain_on_finish = rpi3_pwr_domain_on_finish, .system_off = rpi3_system_off,
Jan
Works well, Jailhouse is happy on the RPi4 about the newly gained PSCI support.
How should we continue with the patch below?
Jan
---
Subject: [PATCH] rpi3/4: Add support for offlining CPUs From: Jan Kiszka jan.kiszka@siemens.com
The hooks were populated but the power down left the CPU in limbo-land. What we need to do - until there is a way to actually power off - is to turn off the MMU and enter the spinning loop as if we were cold-booted. This allows the on-call to pick up the CPU again.
Signed-off-by: Jan Kiszka jan.kiszka@siemens.com --- plat/rpi/common/rpi3_pm.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/plat/rpi/common/rpi3_pm.c b/plat/rpi/common/rpi3_pm.c index 8c2d070c4..2a6bf076b 100644 --- a/plat/rpi/common/rpi3_pm.c +++ b/plat/rpi/common/rpi3_pm.c @@ -123,6 +123,15 @@ static void rpi3_pwr_domain_off(const psci_power_state_t *target_state) #endif }
+void __dead2 plat_secondary_cold_boot_setup(void); + +static void __dead2 +rpi3_pwr_domain_pwr_down_wfi(const psci_power_state_t *target_state) +{ + disable_mmu_el3(); + plat_secondary_cold_boot_setup(); +} + /******************************************************************************* * Platform handler called when a power domain is about to be turned on. The * mpidr determines the CPU to be turned on. @@ -224,6 +233,7 @@ static void __dead2 rpi3_system_off(void) static const plat_psci_ops_t plat_rpi3_psci_pm_ops = { .cpu_standby = rpi3_cpu_standby, .pwr_domain_off = rpi3_pwr_domain_off, + .pwr_domain_pwr_down_wfi = rpi3_pwr_domain_pwr_down_wfi, .pwr_domain_on = rpi3_pwr_domain_on, .pwr_domain_on_finish = rpi3_pwr_domain_on_finish, .system_off = rpi3_system_off, -- 2.16.4