Hi,
I have a question about hafnium executing PSCI_CPU_ON.
So what I observe is,
if I define single SP with 2 Physical CPUs & 2 VCPUS defined in SPMC manifest for hypervisor and secure partition, respectively. And when I do have > 2 cpus running in non-secure world, precisely at the moment when Linux brings up 3rd CPU with PSCI_CPU_ON, EL3 PSCI calls will be forwarded to hafnium running in secure EL2. And this is where it hits.
And since hafnium doesn't find 3rd CPU defined in manifest it will hit "Dead stop here if no more CPU. */
.global cpu_entry cpu_entry: #if SECURE_WORLD == 1
/* Get number of cpus gathered from DT. */ adrp x3, cpu_count add x3, x3, :lo12:cpu_count ldr w3, [x3]
/* Prevent number of CPUs to be higher than supported by platform. */ cmp w3, #MAX_CPUS bhi .
/* x0 points to first cpu in cpus array. */ adrp x0, cpus add x0, x0, :lo12:cpus
/* Get current core affinity. */ get_core_affinity x1, x2
/* Dead stop here if no more cpu. */ 0: cbz w3, 0b
sub w3, w3, #1
/* Get cpu id pointed to by x0 in cpu array. */ ldr x2, [x0, CPU_ID]
If I have defined this 3rd physical CPU in hafnium manifest then hafniu proceed further from above code. But then will hit plat_psci_cpu_resume and eventually it will abort with following check CHECK(vcpu_index < vm->vcpu_count);
cpu_init: /* Call into C code, x0 holds the CPU pointer. */ bl cpu_main
/* Run the vCPU returned by cpu_main. */ bl vcpu_restore_all_and_run
it looks like hafnium expects >= the number of physical Pes running at non-secure side to be defined in SPMC manifest file. Also it expects the same number of VCPUs to be defined in manifest file.
but my understanding is, there could be number of PEs running at non-secure side could be greater than number of Pes/VCPUs at secure side. But from the code flow of hafnium, I don't think it supports that.
Regards, Oza.
Hi Oza,
it looks like hafnium expects >= the number of physical Pes running at non-secure side to be defined in SPMC manifest file. Also it expects the same number of VCPUs to be defined in manifest file.
Yes for a MP SP the number of vCPUs must match the number of PEs in the system: https://hafnium.readthedocs.io/en/latest/secure-partition-manager/secure-par...
For the reference TF-A/Hafnium software stack, the SPMC manifest defines all PEs existing in the system.
but my understanding is, there could be number of PEs running at non-secure side could be greater than number of Pes/VCPUs at secure side.
But from the code flow of hafnium, I don't think it supports that.
From a hardware standpoint if there are Nx PEs in the system, then Nx PEs support both normal and secure worlds. A platform may want to segregate PEs for either normal or secure world execution, but that's not directly supported by Arm's reference SW stack.
Regards, Olivier.
From: Pawandeep Oza (QUIC) via Hafnium hafnium@lists.trustedfirmware.org Sent: 17 April 2024 23:10 To: 'hafnium@lists.trustedfirmware.org' hafnium@lists.trustedfirmware.org Subject: [Hafnium] Hafnium - number of PEs vs number of VCPU Hi,
I have a question about hafnium executing PSCI_CPU_ON.
So what I observe is,
if I define single SP with 2 Physical CPUs & 2 VCPUS defined in SPMC manifest for hypervisor and secure partition, respectively. And when I do have > 2 cpus running in non-secure world, precisely at the moment when Linux brings up 3rd CPU with PSCI_CPU_ON, EL3 PSCI calls will be forwarded to hafnium running in secure EL2. And this is where it hits.
And since hafnium doesn't find 3rd CPU defined in manifest it will hit "Dead stop here if no more CPU. */
.global cpu_entry cpu_entry: #if SECURE_WORLD == 1
/* Get number of cpus gathered from DT. */ adrp x3, cpu_count add x3, x3, :lo12:cpu_count ldr w3, [x3]
/* Prevent number of CPUs to be higher than supported by platform. */ cmp w3, #MAX_CPUS bhi .
/* x0 points to first cpu in cpus array. */ adrp x0, cpus add x0, x0, :lo12:cpus
/* Get current core affinity. */ get_core_affinity x1, x2
/* Dead stop here if no more cpu. */ 0: cbz w3, 0b
sub w3, w3, #1
/* Get cpu id pointed to by x0 in cpu array. */ ldr x2, [x0, CPU_ID]
If I have defined this 3rd physical CPU in hafnium manifest then hafniu proceed further from above code. But then will hit plat_psci_cpu_resume and eventually it will abort with following check CHECK(vcpu_index < vm->vcpu_count);
cpu_init: /* Call into C code, x0 holds the CPU pointer. */ bl cpu_main
/* Run the vCPU returned by cpu_main. */ bl vcpu_restore_all_and_run
it looks like hafnium expects >= the number of physical Pes running at non-secure side to be defined in SPMC manifest file. Also it expects the same number of VCPUs to be defined in manifest file.
but my understanding is, there could be number of PEs running at non-secure side could be greater than number of Pes/VCPUs at secure side. But from the code flow of hafnium, I don't think it supports that.
Regards, Oza.
-- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org
hafnium@lists.trustedfirmware.org
-
Olivier Deprez -
Pawandeep Oza (QUIC)