Hi,
I am trying to enable SPM on iMX8MP platform, which is cortex-A53. The BL2 is already enabled and the dts file is needed. But I am not sure how to write below dts information when I referencing fvp_spmc_manifest.dts. Could you help to give some suggestion or where should I find the anwsers?
1. Is fvp_spmc_manifest.dts enough to enable SPM? I see there are dts files, such as fvp_fw_config.dts. What's necessary dts nodes to enable SPM?
2. Does vcpu_count means CPU number?
3. I see load_address of attribute is optee-os load address. But I do not know how should I decide the load addresses of hypervisors. Is this address decided by virtual machine, or it is decided in runtime? I cannot find the 0x7100000 in trusted-services project.
4. I am confused on "Secure Partitions are bundled as independent package files" in below link. Does this bundle package means fip image, or into another file by jason file? Could you help point to the files and image generation command? https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
Regards, Jun
Hi Jun,
First a bit of background. The FF-A Secure Partition Manager (SPM) is split to two components: 1. The SPM Dispatcher or SPMD. This component runs at EL3 and the reference implementation is part of TF-A. 2. The SPM Core or SPMC. This component can run at different exception levels in the secure world. Currently two implementations exist at tf.org, Hafnium which runs at S-EL2 (secure hypervisor) and is maintained by the TF-A team, and OP-TEE SPMC which runs at S-EL1 and is maintained by the TS team. (See the chapter 2.4 of the "PSA Firmware_Framework for A (PSA-FF-A)" document here https://developer.arm.com/docs/den0077/latest for more details)
And now let me address your questions.
1, 2, 3. fvp_spmc_manifest.dts The handling and content of the SPMC manifest is SPMC implementation specific. This file in the TF-A repo is irrelevant (not used) for the OP-TEE SPMC.
4. "Secure Partitions are bundled as independent package files" One of the differences between Hafnium and OP-TEE SPMC is how they boot the SPs. Hafnium expects the SPs being loaded by BL2 from the FIP package, and expect BL2 to tell the load-address of the SPs in the SPMC manifest. OP-TEE in turn has the SPs linked into the OP-TEE binary and uses symbols bound by the linker to find the SPs.
/George
PS: I suggest taking this discussion off the TF-A mailing list as for the TF-A community this is out of scope.
-----Original Message----- From: Jun Nie jun.nie@linaro.org Sent: 19 April 2021 16:21 To: Manish Pandey2 Manish.Pandey2@arm.com; tf-a@lists.trustedfirmware.org Cc: Olivier Deprez Olivier.Deprez@arm.com; Achin Gupta Achin.Gupta@arm.com; Shebu Varghese Kuriakose Shebu.VargheseKuriakose@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com Subject: Question on how enable TF-A SPM on iMX8M
Hi,
I am trying to enable SPM on iMX8MP platform, which is cortex-A53. The BL2 is already enabled and the dts file is needed. But I am not sure how to write below dts information when I referencing fvp_spmc_manifest.dts. Could you help to give some suggestion or where should I find the anwsers?
1. Is fvp_spmc_manifest.dts enough to enable SPM? I see there are dts files, such as fvp_fw_config.dts. What's necessary dts nodes to enable SPM?
2. Does vcpu_count means CPU number?
3. I see load_address of attribute is optee-os load address. But I do not know how should I decide the load addresses of hypervisors. Is this address decided by virtual machine, or it is decided in runtime? I cannot find the 0x7100000 in trusted-services project.
4. I am confused on "Secure Partitions are bundled as independent package files" in below link. Does this bundle package means fip image, or into another file by jason file? Could you help point to the files and image generation command? https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
Regards, Jun
Hi Jun,
On top of what Gyorgy suggested, please find specific answers to your queries
I am trying to enable SPM on iMX8MP platform, which is cortex-A53. The BL2 is already enabled and the dts file is needed. But I am not sure how to write below dts information when I referencing fvp_spmc_manifest.dts. Could you help to give some suggestion or where should I find the anwsers?
1. Is fvp_spmc_manifest.dts enough to enable SPM? I see there are dts files, such as fvp_fw_config.dts. What's necessary dts nodes to enable SPM?
Arm's FVP platform use fconf mechanism to load different dts files, the first dts file which keeps information about all the dt files is "fw_config.dts", these dt files are consumed by different SW components. They are loaded(by BL2) at base address provided in their fw_config.dts entry and x0 register is used to pass dt address to SW component. HW_CONFIG dt consumed by Linux, TOS_FW_CONFIG consumed by trusted OS(BL32), NT_FW_CONFIG consumed by non-secure FW(BL33) I will explain you through example of one of the configurations which is present in CI https://review.trustedfirmware.org/plugins/gitiles/ci/tf-a-ci-scripts/+/refs... In this configuration, BL33 : NS Hypervisor, BL32: Secure hypervisor (SPM) TOS_FW_CONFIG in this case is fvp_spmc_manifest.dts (plat/arm/board/fvp/platform.mk +271), which is loaded 0x04001500 by BL2 and when BL31 fills x0 register with this address when jumping to BL32. To avoid going through fconf for now, you can quickly test it by loading and passing base address of fvp_spmc_manifest.dts.
2. Does vcpu_count means CPU number? Yes
3. I see load_address of attribute is optee-os load address. But I do not know how should I decide the load addresses of hypervisors. Is this address decided by virtual machine, or it is decided in runtime? I cannot find the 0x7100000 in trusted-services project. If i take spmc example again then, BL2 loads BL32 image at BL32_BASE which is Trusted DRAM base (0x6000000, include/plat/arm/common/arm_def.h +500), So when BL31 prepare to run BL32 it jumps to BL32_BASE with x0 populated with TOS_FW_CONFIG base(mentioned earlier).
4. I am confused on "Secure Partitions are bundled as independent package files" in below link. Does this bundle package means fip image, or into another file by jason file? Could you help point to the files and image generation command? https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon... Secure partitions are packaged as SP package which in turn are embedded into FIP as blobs. The json file is a human readable form to provide SP packages by platform (max 8 supported now). TF-A build expects a json file SP_LAYOUT_FILE (docs/getting_started/build-options.rst +608). In our current setup we have 3 Secure partitions (named cactus) are exported from TF-A test repository script to generate json file https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tre%E2%80%8Be/tools/gene...https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/tools/generate_json/generate_json.sh (can be manually created as well) And the fvp json file looks like below { "cactus-primary" : { "image": "cactus.bin", "pm": "cactus.dts", "owner": "SiP" },
"cactus-secondary" : { "image": "cactus.bin", "pm": "cactus-secondary.dts", "owner": "Plat" },
"cactus-tertiary" : { "image": "cactus.bin", "pm": "cactus-tertiary.dts" } }
For a quick test, what you can do is preload SPMC binary and is manifests and Secure partitions(can reuse cactus for now).
@Gyorgy Szingmailto:Gyorgy.Szing@arm.com: For now, IMO its OK to keep discussions on TF-A mailing list (as it mostly related with loading of the images).
________________________________ From: Gyorgy Szing Gyorgy.Szing@arm.com Sent: 19 April 2021 16:48 To: Jun Nie jun.nie@linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: Olivier Deprez Olivier.Deprez@arm.com; Achin Gupta Achin.Gupta@arm.com; Shebu Varghese Kuriakose Shebu.VargheseKuriakose@arm.com; nd nd@arm.com; nd nd@arm.com Subject: RE: Question on how enable TF-A SPM on iMX8M
Hi Jun,
First a bit of background. The FF-A Secure Partition Manager (SPM) is split to two components: 1. The SPM Dispatcher or SPMD. This component runs at EL3 and the reference implementation is part of TF-A. 2. The SPM Core or SPMC. This component can run at different exception levels in the secure world. Currently two implementations exist at tf.org, Hafnium which runs at S-EL2 (secure hypervisor) and is maintained by the TF-A team, and OP-TEE SPMC which runs at S-EL1 and is maintained by the TS team. (See the chapter 2.4 of the "PSA Firmware_Framework for A (PSA-FF-A)" document here https://developer.arm.com/docs/den0077/latest for more details)
And now let me address your questions.
1, 2, 3. fvp_spmc_manifest.dts The handling and content of the SPMC manifest is SPMC implementation specific. This file in the TF-A repo is irrelevant (not used) for the OP-TEE SPMC.
4. "Secure Partitions are bundled as independent package files" One of the differences between Hafnium and OP-TEE SPMC is how they boot the SPs. Hafnium expects the SPs being loaded by BL2 from the FIP package, and expect BL2 to tell the load-address of the SPs in the SPMC manifest. OP-TEE in turn has the SPs linked into the OP-TEE binary and uses symbols bound by the linker to find the SPs.
/George
PS: I suggest taking this discussion off the TF-A mailing list as for the TF-A community this is out of scope.
-----Original Message----- From: Jun Nie jun.nie@linaro.org Sent: 19 April 2021 16:21 To: Manish Pandey2 Manish.Pandey2@arm.com; tf-a@lists.trustedfirmware.org Cc: Olivier Deprez Olivier.Deprez@arm.com; Achin Gupta Achin.Gupta@arm.com; Shebu Varghese Kuriakose Shebu.VargheseKuriakose@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com Subject: Question on how enable TF-A SPM on iMX8M
Hi,
I am trying to enable SPM on iMX8MP platform, which is cortex-A53. The BL2 is already enabled and the dts file is needed. But I am not sure how to write below dts information when I referencing fvp_spmc_manifest.dts. Could you help to give some suggestion or where should I find the anwsers?
1. Is fvp_spmc_manifest.dts enough to enable SPM? I see there are dts files, such as fvp_fw_config.dts. What's necessary dts nodes to enable SPM?
2. Does vcpu_count means CPU number?
3. I see load_address of attribute is optee-os load address. But I do not know how should I decide the load addresses of hypervisors. Is this address decided by virtual machine, or it is decided in runtime? I cannot find the 0x7100000 in trusted-services project.
4. I am confused on "Secure Partitions are bundled as independent package files" in below link. Does this bundle package means fip image, or into another file by jason file? Could you help point to the files and image generation command? https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
Regards, Jun
Hi Jun,
Just to clarify a bit. You mention a CA53 based platform (hence Armv8.0 and no secure EL2 extension). The only possibility to deploy an FF-A compliant SPM is by using OP-TEE at S-EL1.
It means the questions (1.,2.,3.,4.) are not really relevant in this context as they apply to an SEL2 based SPM configuration, which is what Manish explained. Similarly, the doc link you pointed (https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...) relates to using Hafnium as a SPM on Armv8.4+ platforms.
For your specific case, you should refer to OP-TEE and Trusted Services documentation (and Gyorgy's comments)
Regards, Olivier.
________________________________________ From: Manish Pandey2 Manish.Pandey2@arm.com Sent: 20 April 2021 00:49 To: Gyorgy Szing; Jun Nie; tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Shebu Varghese Kuriakose; nd Subject: Re: Question on how enable TF-A SPM on iMX8M
Hi Jun,
On top of what Gyorgy suggested, please find specific answers to your queries
I am trying to enable SPM on iMX8MP platform, which is cortex-A53. The BL2 is already enabled and the dts file is needed. But I am not sure how to write below dts information when I referencing fvp_spmc_manifest.dts. Could you help to give some suggestion or where should I find the anwsers?
1. Is fvp_spmc_manifest.dts enough to enable SPM? I see there are dts files, such as fvp_fw_config.dts. What's necessary dts nodes to enable SPM?
Arm's FVP platform use fconf mechanism to load different dts files, the first dts file which keeps information about all the dt files is "fw_config.dts", these dt files are consumed by different SW components. They are loaded(by BL2) at base address provided in their fw_config.dts entry and x0 register is used to pass dt address to SW component. HW_CONFIG dt consumed by Linux, TOS_FW_CONFIG consumed by trusted OS(BL32), NT_FW_CONFIG consumed by non-secure FW(BL33) I will explain you through example of one of the configurations which is present in CI https://review.trustedfirmware.org/plugins/gitiles/ci/tf-a-ci-scripts/+/refs... In this configuration, BL33 : NS Hypervisor, BL32: Secure hypervisor (SPM) TOS_FW_CONFIG in this case is fvp_spmc_manifest.dts (plat/arm/board/fvp/platform.mk +271), which is loaded 0x04001500 by BL2 and when BL31 fills x0 register with this address when jumping to BL32. To avoid going through fconf for now, you can quickly test it by loading and passing base address of fvp_spmc_manifest.dts.
2. Does vcpu_count means CPU number? Yes
3. I see load_address of attribute is optee-os load address. But I do not know how should I decide the load addresses of hypervisors. Is this address decided by virtual machine, or it is decided in runtime? I cannot find the 0x7100000 in trusted-services project. If i take spmc example again then, BL2 loads BL32 image at BL32_BASE which is Trusted DRAM base (0x6000000, include/plat/arm/common/arm_def.h +500), So when BL31 prepare to run BL32 it jumps to BL32_BASE with x0 populated with TOS_FW_CONFIG base(mentioned earlier).
4. I am confused on "Secure Partitions are bundled as independent package files" in below link. Does this bundle package means fip image, or into another file by jason file? Could you help point to the files and image generation command? https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon... Secure partitions are packaged as SP package which in turn are embedded into FIP as blobs. The json file is a human readable form to provide SP packages by platform (max 8 supported now). TF-A build expects a json file SP_LAYOUT_FILE (docs/getting_started/build-options.rst +608). In our current setup we have 3 Secure partitions (named cactus) are exported from TF-A test repository script to generate json file https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tre%E2%80%8Be/tools/gene...https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/tools/generate_json/generate_json.sh (can be manually created as well) And the fvp json file looks like below { "cactus-primary" : { "image": "cactus.bin", "pm": "cactus.dts", "owner": "SiP" },
"cactus-secondary" : { "image": "cactus.bin", "pm": "cactus-secondary.dts", "owner": "Plat" },
"cactus-tertiary" : { "image": "cactus.bin", "pm": "cactus-tertiary.dts" } }
For a quick test, what you can do is preload SPMC binary and is manifests and Secure partitions(can reuse cactus for now).
@Gyorgy Szingmailto:Gyorgy.Szing@arm.com: For now, IMO its OK to keep discussions on TF-A mailing list (as it mostly related with loading of the images).
________________________________ From: Gyorgy Szing Gyorgy.Szing@arm.com Sent: 19 April 2021 16:48 To: Jun Nie jun.nie@linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: Olivier Deprez Olivier.Deprez@arm.com; Achin Gupta Achin.Gupta@arm.com; Shebu Varghese Kuriakose Shebu.VargheseKuriakose@arm.com; nd nd@arm.com; nd nd@arm.com Subject: RE: Question on how enable TF-A SPM on iMX8M
Hi Jun,
First a bit of background. The FF-A Secure Partition Manager (SPM) is split to two components: 1. The SPM Dispatcher or SPMD. This component runs at EL3 and the reference implementation is part of TF-A. 2. The SPM Core or SPMC. This component can run at different exception levels in the secure world. Currently two implementations exist at tf.org, Hafnium which runs at S-EL2 (secure hypervisor) and is maintained by the TF-A team, and OP-TEE SPMC which runs at S-EL1 and is maintained by the TS team. (See the chapter 2.4 of the "PSA Firmware_Framework for A (PSA-FF-A)" document here https://developer.arm.com/docs/den0077/latest for more details)
And now let me address your questions.
1, 2, 3. fvp_spmc_manifest.dts The handling and content of the SPMC manifest is SPMC implementation specific. This file in the TF-A repo is irrelevant (not used) for the OP-TEE SPMC.
4. "Secure Partitions are bundled as independent package files" One of the differences between Hafnium and OP-TEE SPMC is how they boot the SPs. Hafnium expects the SPs being loaded by BL2 from the FIP package, and expect BL2 to tell the load-address of the SPs in the SPMC manifest. OP-TEE in turn has the SPs linked into the OP-TEE binary and uses symbols bound by the linker to find the SPs.
/George
PS: I suggest taking this discussion off the TF-A mailing list as for the TF-A community this is out of scope.
-----Original Message----- From: Jun Nie jun.nie@linaro.org Sent: 19 April 2021 16:21 To: Manish Pandey2 Manish.Pandey2@arm.com; tf-a@lists.trustedfirmware.org Cc: Olivier Deprez Olivier.Deprez@arm.com; Achin Gupta Achin.Gupta@arm.com; Shebu Varghese Kuriakose Shebu.VargheseKuriakose@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com Subject: Question on how enable TF-A SPM on iMX8M
Hi,
I am trying to enable SPM on iMX8MP platform, which is cortex-A53. The BL2 is already enabled and the dts file is needed. But I am not sure how to write below dts information when I referencing fvp_spmc_manifest.dts. Could you help to give some suggestion or where should I find the anwsers?
1. Is fvp_spmc_manifest.dts enough to enable SPM? I see there are dts files, such as fvp_fw_config.dts. What's necessary dts nodes to enable SPM?
2. Does vcpu_count means CPU number?
3. I see load_address of attribute is optee-os load address. But I do not know how should I decide the load addresses of hypervisors. Is this address decided by virtual machine, or it is decided in runtime? I cannot find the 0x7100000 in trusted-services project.
4. I am confused on "Secure Partitions are bundled as independent package files" in below link. Does this bundle package means fip image, or into another file by jason file? Could you help point to the files and image generation command? https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
Regards, Jun
Hi,
I already built manifest.dtb into TF-A fip image, the fip image size increases about the size of dtb. But I still see below error from spmd_setup(). What should be done beside below patch?
ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc
+ifeq ($(MX8MP_SPMC_MANIFEST_DTS),) +MX8MP_SPMC_MANIFEST_DTS := plat/imx/imx8m/imx8mp/fdts/imx_spmc_manifest.dts +endif + +FDT_SOURCES += ${MX8MP_SPMC_MANIFEST_DTS} +IMX_TOS_FW_CONFIG := ${BUILD_PLAT}/fdts/$(notdir $(basename ${MX8MP_SPMC_MANIFEST_DTS})).dtb + +# Add the TOS_FW_CONFIG to FIP and specify the same to certtool +$(eval $(call TOOL_ADD_PAYLOAD,${IMX_TOS_FW_CONFIG},--tos-fw-config,${IMX_TOS_FW_CONFIG}))
Regards, Jun
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
________________________________________ From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-development, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/?h=imx_lf_... https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4.70_2.3.0%...
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-development, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/?h=imx_lf_... https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4.70_2.3.0%...
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi Jun,
It seems to me that OP-TEE is not initialising at all since the SPMD in EL3 cannot locate its manifest file.
This should not cause an issue when Linux later attempts to turn on secondary CPUs. Are there any TF-A logs on CPU1 at all?
cheers, Achin ________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 08:26 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org; Achin Gupta Achin.Gupta@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-development, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/?h=imx_lf_... https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4.70_2.3.0%...
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
HI Achin,
There is no log from CPU1 when I set log level as 40. The BL31 size limitation prevent me to set log level as 50. Could you help confirm that all manifest file related packing is done in optee-os side for Cortex-A53 case? I mean nothing should checked in TF-A side for invalid manifest case. I see code just check NULL pointer and emit "Invalid or absent SPM Core manifest".
Regards, Jun
Achin Gupta Achin.Gupta@arm.com 于2021年4月22日周四 下午5:09写道：
Hi Jun,
It seems to me that OP-TEE is not initialising at all since the SPMD in EL3 cannot locate its manifest file.
This should not cause an issue when Linux later attempts to turn on secondary CPUs. Are there any TF-A logs on CPU1 at all?
cheers, Achin ________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 08:26 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org; Achin Gupta Achin.Gupta@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-development, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/?h=imx_lf_... https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4.70_2.3.0%...
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
I have some manifest code in TF-A side in below link. Maybe I should remove this patch from TF-A? Is there any document that describe the manifest loading and SP image loading process? I only find Hafnium case in this link: https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/commit/?h=imx_l...
Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午5:57写道：
HI Achin,
There is no log from CPU1 when I set log level as 40. The BL31 size limitation prevent me to set log level as 50. Could you help confirm that all manifest file related packing is done in optee-os side for Cortex-A53 case? I mean nothing should checked in TF-A side for invalid manifest case. I see code just check NULL pointer and emit "Invalid or absent SPM Core manifest".
Regards, Jun
Achin Gupta Achin.Gupta@arm.com 于2021年4月22日周四 下午5:09写道：
Hi Jun,
It seems to me that OP-TEE is not initialising at all since the SPMD in EL3 cannot locate its manifest file.
This should not cause an issue when Linux later attempts to turn on secondary CPUs. Are there any TF-A logs on CPU1 at all?
cheers, Achin ________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 08:26 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org; Achin Gupta Achin.Gupta@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-development, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/?h=imx_lf_... https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4.70_2.3.0%...
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi Jun,
The SPMC manifest is processed by SPMD in TF-A, and this is where you get your error from. Further manifest processing is SPMC specific and in this use-case that is running in OP-TEE. The OP-TEE SPMC uses the manifest from the DTB embedded into the OP-TEE binary.
The current OP-TEE SPMC implementation boots SPs same way as early TAs. The SPs sre linked into the OP-TEE binary, and booted from there. This means, the SPs are not visible in the FIP package, as that only sees the OP-TEE binary with the SPs as a single blob.
So first you have to sort out TF-A/SPMD getting an SPMC manifest having information about the SPMC. For this you need to understand how i.MX8 boots, and how the interface between the boot loader stage and TF-A does look like. I suggest starting here: https://trustedfirmware-a.readthedocs.io/en/latest/plat/imx8m.html
Then if you configure your OP-TEE build correctly (the same way as we have in our non yocto based build), the SPs shall boot.
/George
-----Original Message----- From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 12:03 To: Achin Gupta Achin.Gupta@arm.com Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
I have some manifest code in TF-A side in below link. Maybe I should remove this patch from TF-A? Is there any document that describe the manifest loading and SP image loading process? I only find Hafnium case in this link: https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/commit/?h=imx_l...
Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午5:57写道：
HI Achin,
There is no log from CPU1 when I set log level as 40. The BL31 size limitation prevent me to set log level as 50. Could you help confirm that all manifest file related packing is done in optee-os side for Cortex-A53 case? I mean nothing should checked in TF-A side for invalid manifest case. I see code just check NULL pointer and emit "Invalid or absent SPM Core manifest".
Regards, Jun
Achin Gupta Achin.Gupta@arm.com 于2021年4月22日周四 下午5:09写道：
Hi Jun,
It seems to me that OP-TEE is not initialising at all since the SPMD in EL3 cannot locate its manifest file.
This should not cause an issue when Linux later attempts to turn on secondary CPUs. Are there any TF-A logs on CPU1 at all?
cheers, Achin ________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 08:26 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org; Achin Gupta Achin.Gupta@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-dev elopment, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/ ?h=imx_lf_v2.4%2bbl2&id=7f4ae63301c7ba85449cd491a4770c19159dc24c https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4 .70_2.3.0%2bfio%2bspm&id=3f73eeecde874e6efb90ecb1076b9b96232fc3e4
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi,
As Gyorgy and Achin mentioned, the first problem is SPMD missing to find the SPMC manifest and hence aborting and not booting OP-TEE.
From the log BL2 loads image ids 3,4,5,21 so BL31, BL32, BL33, BL32_EXTRA1_IMAGE_ID
It misses image id 26 TOS_FW_CONFIG_ID which should normally be the SPMC manifest (https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co...)
The second problem causing linux secondary core boot/PSCI failure is probably different, I'd say the linux device tree does not match the HW you use (cpuid/mpidr in cpu nodes). Something to do with the dtb passed by Uboot to linux.
Regards, Olivier.
________________________________________ From: Gyorgy Szing Gyorgy.Szing@arm.com Sent: 22 April 2021 12:30 To: Jun Nie; Achin Gupta Cc: Olivier Deprez; tf-a@lists.trustedfirmware.org; Manish Pandey2; nd Subject: RE: How does SPM impact PSCI functions?
Hi Jun,
The SPMC manifest is processed by SPMD in TF-A, and this is where you get your error from. Further manifest processing is SPMC specific and in this use-case that is running in OP-TEE. The OP-TEE SPMC uses the manifest from the DTB embedded into the OP-TEE binary.
The current OP-TEE SPMC implementation boots SPs same way as early TAs. The SPs sre linked into the OP-TEE binary, and booted from there. This means, the SPs are not visible in the FIP package, as that only sees the OP-TEE binary with the SPs as a single blob.
So first you have to sort out TF-A/SPMD getting an SPMC manifest having information about the SPMC. For this you need to understand how i.MX8 boots, and how the interface between the boot loader stage and TF-A does look like. I suggest starting here: https://trustedfirmware-a.readthedocs.io/en/latest/plat/imx8m.html
Then if you configure your OP-TEE build correctly (the same way as we have in our non yocto based build), the SPs shall boot.
/George
-----Original Message----- From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 12:03 To: Achin Gupta Achin.Gupta@arm.com Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
I have some manifest code in TF-A side in below link. Maybe I should remove this patch from TF-A? Is there any document that describe the manifest loading and SP image loading process? I only find Hafnium case in this link: https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/commit/?h=imx_l...
Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午5:57写道：
HI Achin,
There is no log from CPU1 when I set log level as 40. The BL31 size limitation prevent me to set log level as 50. Could you help confirm that all manifest file related packing is done in optee-os side for Cortex-A53 case? I mean nothing should checked in TF-A side for invalid manifest case. I see code just check NULL pointer and emit "Invalid or absent SPM Core manifest".
Regards, Jun
Achin Gupta Achin.Gupta@arm.com 于2021年4月22日周四 下午5:09写道：
Hi Jun,
It seems to me that OP-TEE is not initialising at all since the SPMD in EL3 cannot locate its manifest file.
This should not cause an issue when Linux later attempts to turn on secondary CPUs. Are there any TF-A logs on CPU1 at all?
cheers, Achin ________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 08:26 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org; Achin Gupta Achin.Gupta@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-dev elopment, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/ ?h=imx_lf_v2.4%2bbl2&id=7f4ae63301c7ba85449cd491a4770c19159dc24c https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4 .70_2.3.0%2bfio%2bspm&id=3f73eeecde874e6efb90ecb1076b9b96232fc3e4
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi Olivier,
I already have this line code in my last patch of TF-A. And I add debug code as below and confirm the --tos-fw-config is built-in. The unpack of fip.bin also shows tos-fw-config.bin existence. So there should be some other bug that make manifest invalid. TF-A v2.4 is enough to support SPM logic, right? Any other suggestions on what to check?
In plat/imx/imx8m/imx8mp/platform.mk: $(eval $(call TOOL_ADD_PAYLOAD,${IMX_TOS_FW_CONFIG},--tos-fw-config,${IMX_TOS_FW_CONFIG}))
Debug code: +++ b/Makefile @@ -1240,6 +1240,7 @@ endif
${BUILD_PLAT}/${FIP_NAME}: ${FIP_DEPS} ${FIPTOOL} ${Q}${FIPTOOL} create ${FIP_ARGS} $@ + $(error FIP_ARGS: ${FIP_ARGS})
FIP_ARGS: --tos-fw-config /home/niej/IMX/gateway-arm/lmp-project/build/tmp-lmp/work/cortexa53-crypto-mx8mp-lmp-linux/imx-atf/2.2+gitAUTOINC+7f4ae63301-r0/git/build/imx8mp/release/fdts/imx_spmc_manifest.dtb --tos-fw-extra1 /home/niej/IMX/gateway-arm/lmp-project/build/deploy/images/imx8mprsb3720/optee/tee-pager_v2.bin --soc-fw /home/niej/IMX/gateway-arm/lmp-project/build/tmp-lmp/work/cortexa53-crypto-mx8mp-lmp-linux/imx-atf/2.2+gitAUTOINC+7f4ae63301-r0/git/build/imx8mp/release/bl31.bin --tos-fw /home/niej/IMX/gateway-arm/lmp-project/build/deploy/images/imx8mprsb3720/optee/tee-header_v2.bin --nt-fw /home/niej/IMX/gateway-arm/lmp-project/build/deploy/images/imx8mprsb3720/u-boot.bin
./fiptool unpack fip.bin fip.bin fiptool nt-fw.bin soc-fw.bin tos-fw.bin tos-fw-config.bin tos-fw-extra1.bin tos-fw-extra2.bin
$ fdtdump tos-fw-config.bin
**** fdtdump is a low-level debugging tool, not meant for general use. **** If you want to decompile a dtb, you probably want **** dtc -I dtb -O dts <filename>
/dts-v1/; // magic: 0xd00dfeed // totalsize: 0x27b (635) // off_dt_struct: 0x38 // off_dt_strings: 0x1fc // off_mem_rsvmap: 0x28 // version: 17 // last_comp_version: 16 // boot_cpuid_phys: 0x0 // size_dt_strings: 0x7f // size_dt_struct: 0x1c4
/ { compatible = "arm,ffa-core-manifest-1.0"; #address-cells = <0x00000002>; #size-cells = <0x00000001>; attribute { spmc_id = <0x00008000>; maj_ver = <0x00000001>; min_ver = <0x00000000>; exec_state = <0x00000000>; load_address = <0x00000000 0x56000000>; entrypoint = <0x00000000 0x56000000>; binary_size = <0x00080000>; }; optee_spmc { compatible = "arm,optee-spmc"; maj_ver = <0x00000001>; min_ver = <0x00000000>; sp1 { debug_name = "crypto"; load_address = <0x00000000 0x61000000>; format = "elf"; }; sp2 { debug_name = "secure_storage"; load_address = <0x00000000 0x60000000>; format = "elf"; }; }; };
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 下午8:58写道：
Hi,
As Gyorgy and Achin mentioned, the first problem is SPMD missing to find the SPMC manifest and hence aborting and not booting OP-TEE. From the log BL2 loads image ids 3,4,5,21 so BL31, BL32, BL33, BL32_EXTRA1_IMAGE_ID It misses image id 26 TOS_FW_CONFIG_ID which should normally be the SPMC manifest (https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co...)
The second problem causing linux secondary core boot/PSCI failure is probably different, I'd say the linux device tree does not match the HW you use (cpuid/mpidr in cpu nodes). Something to do with the dtb passed by Uboot to linux.
Regards, Olivier.
From: Gyorgy Szing Gyorgy.Szing@arm.com Sent: 22 April 2021 12:30 To: Jun Nie; Achin Gupta Cc: Olivier Deprez; tf-a@lists.trustedfirmware.org; Manish Pandey2; nd Subject: RE: How does SPM impact PSCI functions?
Hi Jun,
The SPMC manifest is processed by SPMD in TF-A, and this is where you get your error from. Further manifest processing is SPMC specific and in this use-case that is running in OP-TEE. The OP-TEE SPMC uses the manifest from the DTB embedded into the OP-TEE binary.
The current OP-TEE SPMC implementation boots SPs same way as early TAs. The SPs sre linked into the OP-TEE binary, and booted from there. This means, the SPs are not visible in the FIP package, as that only sees the OP-TEE binary with the SPs as a single blob.
So first you have to sort out TF-A/SPMD getting an SPMC manifest having information about the SPMC. For this you need to understand how i.MX8 boots, and how the interface between the boot loader stage and TF-A does look like. I suggest starting here: https://trustedfirmware-a.readthedocs.io/en/latest/plat/imx8m.html
Then if you configure your OP-TEE build correctly (the same way as we have in our non yocto based build), the SPs shall boot.
/George
-----Original Message----- From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 12:03 To: Achin Gupta Achin.Gupta@arm.com Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
I have some manifest code in TF-A side in below link. Maybe I should remove this patch from TF-A? Is there any document that describe the manifest loading and SP image loading process? I only find Hafnium case in this link: https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/commit/?h=imx_l...
Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午5:57写道：
HI Achin,
There is no log from CPU1 when I set log level as 40. The BL31 size limitation prevent me to set log level as 50. Could you help confirm that all manifest file related packing is done in optee-os side for Cortex-A53 case? I mean nothing should checked in TF-A side for invalid manifest case. I see code just check NULL pointer and emit "Invalid or absent SPM Core manifest".
Regards, Jun
Achin Gupta Achin.Gupta@arm.com 于2021年4月22日周四 下午5:09写道：
Hi Jun,
It seems to me that OP-TEE is not initialising at all since the SPMD in EL3 cannot locate its manifest file.
This should not cause an issue when Linux later attempts to turn on secondary CPUs. Are there any TF-A logs on CPU1 at all?
cheers, Achin ________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 08:26 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org; Achin Gupta Achin.Gupta@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-dev elopment, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/ ?h=imx_lf_v2.4%2bbl2&id=7f4ae63301c7ba85449cd491a4770c19159dc24c https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4 .70_2.3.0%2bfio%2bspm&id=3f73eeecde874e6efb90ecb1076b9b96232fc3e4
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi,
I thought I already find the root cause. The BL32 args is set only in "#if defined(SPD_opteed) || defined(SPD_trusty)" on this iMX8MP platform. While arm board has option " #if defined(SPD_spmd)". I think I need to search all SPD_spmd to check what should be done for a new platform.
Thanks for your replies!
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午10:05写道：
Hi Olivier,
I already have this line code in my last patch of TF-A. And I add debug code as below and confirm the --tos-fw-config is built-in. The unpack of fip.bin also shows tos-fw-config.bin existence. So there should be some other bug that make manifest invalid. TF-A v2.4 is enough to support SPM logic, right? Any other suggestions on what to check?
In plat/imx/imx8m/imx8mp/platform.mk: $(eval $(call TOOL_ADD_PAYLOAD,${IMX_TOS_FW_CONFIG},--tos-fw-config,${IMX_TOS_FW_CONFIG}))
Debug code: +++ b/Makefile @@ -1240,6 +1240,7 @@ endif
${BUILD_PLAT}/${FIP_NAME}: ${FIP_DEPS} ${FIPTOOL} ${Q}${FIPTOOL} create ${FIP_ARGS} $@
$(error FIP_ARGS: ${FIP_ARGS})FIP_ARGS: --tos-fw-config /home/niej/IMX/gateway-arm/lmp-project/build/tmp-lmp/work/cortexa53-crypto-mx8mp-lmp-linux/imx-atf/2.2+gitAUTOINC+7f4ae63301-r0/git/build/imx8mp/release/fdts/imx_spmc_manifest.dtb --tos-fw-extra1 /home/niej/IMX/gateway-arm/lmp-project/build/deploy/images/imx8mprsb3720/optee/tee-pager_v2.bin --soc-fw /home/niej/IMX/gateway-arm/lmp-project/build/tmp-lmp/work/cortexa53-crypto-mx8mp-lmp-linux/imx-atf/2.2+gitAUTOINC+7f4ae63301-r0/git/build/imx8mp/release/bl31.bin --tos-fw /home/niej/IMX/gateway-arm/lmp-project/build/deploy/images/imx8mprsb3720/optee/tee-header_v2.bin --nt-fw /home/niej/IMX/gateway-arm/lmp-project/build/deploy/images/imx8mprsb3720/u-boot.bin
./fiptool unpack fip.bin fip.bin fiptool nt-fw.bin soc-fw.bin tos-fw.bin tos-fw-config.bin tos-fw-extra1.bin tos-fw-extra2.bin
$ fdtdump tos-fw-config.bin
**** fdtdump is a low-level debugging tool, not meant for general use. **** If you want to decompile a dtb, you probably want **** dtc -I dtb -O dts <filename>
/dts-v1/; // magic: 0xd00dfeed // totalsize: 0x27b (635) // off_dt_struct: 0x38 // off_dt_strings: 0x1fc // off_mem_rsvmap: 0x28 // version: 17 // last_comp_version: 16 // boot_cpuid_phys: 0x0 // size_dt_strings: 0x7f // size_dt_struct: 0x1c4
/ { compatible = "arm,ffa-core-manifest-1.0"; #address-cells = <0x00000002>; #size-cells = <0x00000001>; attribute { spmc_id = <0x00008000>; maj_ver = <0x00000001>; min_ver = <0x00000000>; exec_state = <0x00000000>; load_address = <0x00000000 0x56000000>; entrypoint = <0x00000000 0x56000000>; binary_size = <0x00080000>; }; optee_spmc { compatible = "arm,optee-spmc"; maj_ver = <0x00000001>; min_ver = <0x00000000>; sp1 { debug_name = "crypto"; load_address = <0x00000000 0x61000000>; format = "elf"; }; sp2 { debug_name = "secure_storage"; load_address = <0x00000000 0x60000000>; format = "elf"; }; }; };
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 下午8:58写道：
Hi,
As Gyorgy and Achin mentioned, the first problem is SPMD missing to find the SPMC manifest and hence aborting and not booting OP-TEE. From the log BL2 loads image ids 3,4,5,21 so BL31, BL32, BL33, BL32_EXTRA1_IMAGE_ID It misses image id 26 TOS_FW_CONFIG_ID which should normally be the SPMC manifest (https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co...)
The second problem causing linux secondary core boot/PSCI failure is probably different, I'd say the linux device tree does not match the HW you use (cpuid/mpidr in cpu nodes). Something to do with the dtb passed by Uboot to linux.
Regards, Olivier.
From: Gyorgy Szing Gyorgy.Szing@arm.com Sent: 22 April 2021 12:30 To: Jun Nie; Achin Gupta Cc: Olivier Deprez; tf-a@lists.trustedfirmware.org; Manish Pandey2; nd Subject: RE: How does SPM impact PSCI functions?
Hi Jun,
The SPMC manifest is processed by SPMD in TF-A, and this is where you get your error from. Further manifest processing is SPMC specific and in this use-case that is running in OP-TEE. The OP-TEE SPMC uses the manifest from the DTB embedded into the OP-TEE binary.
The current OP-TEE SPMC implementation boots SPs same way as early TAs. The SPs sre linked into the OP-TEE binary, and booted from there. This means, the SPs are not visible in the FIP package, as that only sees the OP-TEE binary with the SPs as a single blob.
So first you have to sort out TF-A/SPMD getting an SPMC manifest having information about the SPMC. For this you need to understand how i.MX8 boots, and how the interface between the boot loader stage and TF-A does look like. I suggest starting here: https://trustedfirmware-a.readthedocs.io/en/latest/plat/imx8m.html
Then if you configure your OP-TEE build correctly (the same way as we have in our non yocto based build), the SPs shall boot.
/George
-----Original Message----- From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 12:03 To: Achin Gupta Achin.Gupta@arm.com Cc: Olivier Deprez Olivier.Deprez@arm.com; tf-a@lists.trustedfirmware.org; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
I have some manifest code in TF-A side in below link. Maybe I should remove this patch from TF-A? Is there any document that describe the manifest loading and SP image loading process? I only find Hafnium case in this link: https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon...
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/commit/?h=imx_l...
Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午5:57写道：
HI Achin,
There is no log from CPU1 when I set log level as 40. The BL31 size limitation prevent me to set log level as 50. Could you help confirm that all manifest file related packing is done in optee-os side for Cortex-A53 case? I mean nothing should checked in TF-A side for invalid manifest case. I see code just check NULL pointer and emit "Invalid or absent SPM Core manifest".
Regards, Jun
Achin Gupta Achin.Gupta@arm.com 于2021年4月22日周四 下午5:09写道：
Hi Jun,
It seems to me that OP-TEE is not initialising at all since the SPMD in EL3 cannot locate its manifest file.
This should not cause an issue when Linux later attempts to turn on secondary CPUs. Are there any TF-A logs on CPU1 at all?
cheers, Achin ________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 08:26 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org; Achin Gupta Achin.Gupta@arm.com; Gyorgy Szing Gyorgy.Szing@arm.com; Manish Pandey2 Manish.Pandey2@arm.com Subject: Re: How does SPM impact PSCI functions?
Hi Olivier,
BTW: Below log is emitted by the LInux kernel. Does FFA logic in OPTEE impact PSCI function?
[ 0.071691] psci: failed to boot CPU1 (-95)
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月22日周四 下午3:17写道：
Hi Olivier,
Below are git repo, log and build option. But OPTEE log disappear after TF-A enabling SPM. I guess it is due to manifest loading failure and OPTEE is skipped in boot chain. For the OP_TEE, I ported the SPM patches on branch https://git.trustedfirmware.org/OP-TEE/optee_os.git/log/?h=psa-dev elopment, except the last one: 1ba62d4 SPMC: Pass device regions to SP. That patch introduce build failure. Thanks for your time!
https://git.linaro.org/people/jun.nie/trusted-firmware-a.git/tree/ ?h=imx_lf_v2.4%2bbl2&id=7f4ae63301c7ba85449cd491a4770c19159dc24c https://git.linaro.org/people/jun.nie/optee_os.git/tree/?h=imx_5.4 .70_2.3.0%2bfio%2bspm&id=3f73eeecde874e6efb90ecb1076b9b96232fc3e4
U-Boot SPL 2020.04 (Jan 01 1970 - 00:00:00 +0000) DDRINFO: start DRAM init DDRINFO: DRAM rate 4000MTS DDRINFO:ddrphy calibration done DDRINFO: ddrmix config done Normal Boot Trying to boot from BOOTROM image offset 0x8000, pagesize 0x200, ivt offset 0x0 sha256,rsa2048:dev+ ## Checking hash(es) for Image fip@1 ... sha256+ OK ## Checking hash(es) for Image atf@1 ... sha256+ OK NOTICE: BL2: v2.4(release):7f4ae6330 NOTICE: BL2: Built : 00:00:00, Jan 1 1970 INFO: BL2: Doing platform setup INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x940000 INFO: Image id=3 loaded: 0x940000 - 0x94f15e INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x56000000 INFO: Image id=4 loaded: 0x56000000 - 0x5600001c INFO: OPTEE ep=0x56000000 INFO: OPTEE header info: INFO: magic=0x4554504f INFO: version=0x2 INFO: arch=0x1 INFO: flags=0x0 INFO: nb_images=0x1 INFO: BL2: Loading image id 21 INFO: Loading image id=21 at address 0x56000000 INFO: Image id=21 loaded: 0x56000000 - 0x56098438 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x40200000 INFO: Image id=5 loaded: 0x40200000 - 0x402decce NOTICE: BL2: Booting BL31 INFO: Entry point address = 0x940000 INFO: SPSR = 0x3cd NOTICE: BL31: v2.4(release):7f4ae6330 NOTICE: BL31: Built : 00:00:00, Jan 1 1970 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services ERROR: Invalid or absent SPM Core manifest. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x40200000 INFO: SPSR = 0x3c9
U-Boot 2020.04 (Jan 01 1970 - 00:00:00 +0000) CPU: i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz) CPU: Industrial temperature grade (-40C to 105C) at 31C
ATF build option: 19 PLAT=imx8mp SPD=spmd SPMD_SPM_AT_SEL2=0 \ 20 NEED_BL2=yes NEED_BL32=yes NEED_BL33=yes \ 21 BL32=${DEPLOY_DIR_IMAGE}/optee/tee-header_v2.bin \ 22 BL32_EXTRA1=${DEPLOY_DIR_IMAGE}/optee/tee-pager_v2.bin \ 23 BL33=${DEPLOY_DIR_IMAGE}/u-boot.bin \ 24 CRASH_REPORTING=1 \ 25 IMX_BOOT_UART_BASE=0x30880000 \ 26 LOG_LEVEL=40 \
OPTEE build option: 46 CFG_WITH_SP=y CFG_CORE_SEL1_SPMC=y CFG_CORE_FFA=y \ 47 CROSS_COMPILE_sp_${OPTEE_ARCH}=${HOST_PREFIX} \ 48 CFG_NXP_CAAM=y CFG_RNG_PTA=y \ 49 CFG_CORE_DYN_SHM=n CFG_DT=y CFG_EXTERNAL_DTB_OVERLAY=y CFG_DT_ADDR=0x43200000 \ 50 CFG_SPM_MM_START=0x60000000 CFG_SPM_MM_SIZE=0x2000000 \ 51 EARLY_TA_PATHS="${DEPLOY_DIR_IMAGE}/dc1eef48-b17a-4ccf-ac8b-dfcff7711b14.stripped.elf ${DEPLOY_DIR_IMAGE}/d9df52d5-16a2-4bb2-9aa4-d26d3b84e8c0.stripped.elf" \ 52 CFG_EMBED_DTB_SOURCE_FILE=${B}/../sp_manifest.dts \ 53 CFG_UART_BASE=UART3_BASE \ 54 CFG_TEE_CORE_LOG_LEVEL=3 \
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 上午12:22写道：
Hi Jun,
Can you provide the TF-A and OP-TEE git trees/branches you use? TF-A SPMD upstream expects usage of FFA_SECONDARY_EP_REGISTER FF-A v1.1 ABI that your OP-TEE version might not be using.
Also please share logs from TF-A and OP-TEE boot.
Thanks, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 21 April 2021 18:03 To: tf-a@lists.trustedfirmware.org Cc: Olivier Deprez; Achin Gupta; Gyorgy Szing; Manish Pandey2 Subject: How does SPM impact PSCI functions?
Hi,
I just find that secondary CA53 CPUs boot fail with error -95(as below definition and log) in the kernel after enabling SPM with option: SPD=spmd SPMD_SPM_AT_SEL2=0. How does SPM impact PSCI functions? Or which code lines should I check to debug this?
#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[ 0.066970] smp: Bringing up secondary CPUs ... [ 0.071681] psci: failed to boot CPU1 (-95) [ 0.075454] CPU1: failed to boot: -95 [ 0.079591] psci: failed to boot CPU2 (-95) [ 0.083308] CPU2: failed to boot: -95
Regards, Jun
Hi,
I see below code for manifest address calculation in arm platform. But it is ARM platform specific. I am not sure how the manifest load address is decided when BL2 loads it. There is no image address in this code. https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co.... Below calculation is wrong on iMX platform and cause below error.
bl32_image_ep_info.args.arg0 = BL32_BASE + BL32_SIZE - PAGE_SIZE;
125 /* SPM (hafnium in secure world) expects SPM Core manifest base address 126 * in x0, which in !RESET_TO_BL31 case loaded after base of non shared 127 * SRAM(after 4KB offset of SRAM). But in RESET_TO_BL31 case all non 128 * shared SRAM is allocated to BL31, so to avoid overwriting of manifest 129 * keep it in the last page. 130 */ 131 bl32_image_ep_info.args.arg0 = ARM_TRUSTED_SRAM_BASE + 132 PLAT_ARM_TRUSTED_SRAM_SIZE - PAGE_SIZE;
ERROR: Error while mapping SPM Core manifest (-1). WARNING: No or invalid SPM Core manifest image provided by BL2 WARNING: Booting device without SPM initialization. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world
Regards, Jun
Hi,
the TOS fw config address comes from the firmware configuration framework: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/bo...
The fvp_fw_config dtb itself must be loaded to get this information. So if not using fconf on IMX you may have to hardcode the tos fw config blob address in some way in the platform port.
Regards, Olivier.
________________________________________ From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 16:56 To: Olivier Deprez Cc: Gyorgy Szing; Achin Gupta; tf-a@lists.trustedfirmware.org; Manish Pandey2; nd Subject: To what address BL2 load SPM manifest in FIP image
Hi,
I see below code for manifest address calculation in arm platform. But it is ARM platform specific. I am not sure how the manifest load address is decided when BL2 loads it. There is no image address in this code. https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co.... Below calculation is wrong on iMX platform and cause below error.
bl32_image_ep_info.args.arg0 = BL32_BASE + BL32_SIZE - PAGE_SIZE;
125 /* SPM (hafnium in secure world) expects SPM Core manifest base address 126 * in x0, which in !RESET_TO_BL31 case loaded after base of non shared 127 * SRAM(after 4KB offset of SRAM). But in RESET_TO_BL31 case all non 128 * shared SRAM is allocated to BL31, so to avoid overwriting of manifest 129 * keep it in the last page. 130 */ 131 bl32_image_ep_info.args.arg0 = ARM_TRUSTED_SRAM_BASE + 132 PLAT_ARM_TRUSTED_SRAM_SIZE - PAGE_SIZE;
ERROR: Error while mapping SPM Core manifest (-1). WARNING: No or invalid SPM Core manifest image provided by BL2 WARNING: Booting device without SPM initialization. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world
Regards, Jun
Hi,
How fvp board handle the manifest memory map? Carve the memory out and use a dedicated memory map? I cannot find the code. But I fail in this line when I try to map and load manifest. I have to shrink DRAM size so that manifest memory is dynamically mapped.
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/xlat_ta...
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 下午11:10写道：
Hi,
the TOS fw config address comes from the firmware configuration framework: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/bo...
The fvp_fw_config dtb itself must be loaded to get this information. So if not using fconf on IMX you may have to hardcode the tos fw config blob address in some way in the platform port.
Regards, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 16:56 To: Olivier Deprez Cc: Gyorgy Szing; Achin Gupta; tf-a@lists.trustedfirmware.org; Manish Pandey2; nd Subject: To what address BL2 load SPM manifest in FIP image
Hi,
I see below code for manifest address calculation in arm platform. But it is ARM platform specific. I am not sure how the manifest load address is decided when BL2 loads it. There is no image address in this code. https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co.... Below calculation is wrong on iMX platform and cause below error.
bl32_image_ep_info.args.arg0 = BL32_BASE + BL32_SIZE - PAGE_SIZE;
125 /* SPM (hafnium in secure world) expects SPM Core manifest base address 126 * in x0, which in !RESET_TO_BL31 case loaded after base of non shared 127 * SRAM(after 4KB offset of SRAM). But in RESET_TO_BL31 case all non 128 * shared SRAM is allocated to BL31, so to avoid overwriting of manifest 129 * keep it in the last page. 130 */ 131 bl32_image_ep_info.args.arg0 = ARM_TRUSTED_SRAM_BASE + 132 PLAT_ARM_TRUSTED_SRAM_SIZE - PAGE_SIZE;
ERROR: Error while mapping SPM Core manifest (-1). WARNING: No or invalid SPM Core manifest image provided by BL2 WARNING: Booting device without SPM initialization. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world
Regards, Jun
Hi,
Anyone who is familiar with ATF memory map has idea on this? I see permission error in below line when plat_spm_core_manifest_load() calls mmap_add_dynamic_region(). While fvp does not has such issue.
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/xlat_ta...
I am wandering why fvp manifest mapping check does not fall into the error. Because static memory map on FVP include ARM_SHARED_RAM_BASE 0x04000000 size: 0x00001000. While below manifest start address and size PAGE_SIZE should fall into fully_overlapped_va check and fail.
bl32_image_ep_info.args.arg0 = ARM_TRUSTED_SRAM_BASE + PLAT_ARM_TRUSTED_SRAM_SIZE - PAGE_SIZE;
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月23日周五 下午5:35写道：
Hi,
How fvp board handle the manifest memory map? Carve the memory out and use a dedicated memory map? I cannot find the code. But I fail in this line when I try to map and load manifest. I have to shrink DRAM size so that manifest memory is dynamically mapped.
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/xlat_ta...
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 下午11:10写道：
Hi,
the TOS fw config address comes from the firmware configuration framework: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/bo...
The fvp_fw_config dtb itself must be loaded to get this information. So if not using fconf on IMX you may have to hardcode the tos fw config blob address in some way in the platform port.
Regards, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 16:56 To: Olivier Deprez Cc: Gyorgy Szing; Achin Gupta; tf-a@lists.trustedfirmware.org; Manish Pandey2; nd Subject: To what address BL2 load SPM manifest in FIP image
Hi,
I see below code for manifest address calculation in arm platform. But it is ARM platform specific. I am not sure how the manifest load address is decided when BL2 loads it. There is no image address in this code. https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co.... Below calculation is wrong on iMX platform and cause below error.
bl32_image_ep_info.args.arg0 = BL32_BASE + BL32_SIZE - PAGE_SIZE;
125 /* SPM (hafnium in secure world) expects SPM Core manifest base address 126 * in x0, which in !RESET_TO_BL31 case loaded after base of non shared 127 * SRAM(after 4KB offset of SRAM). But in RESET_TO_BL31 case all non 128 * shared SRAM is allocated to BL31, so to avoid overwriting of manifest 129 * keep it in the last page. 130 */ 131 bl32_image_ep_info.args.arg0 = ARM_TRUSTED_SRAM_BASE + 132 PLAT_ARM_TRUSTED_SRAM_SIZE - PAGE_SIZE;
ERROR: Error while mapping SPM Core manifest (-1). WARNING: No or invalid SPM Core manifest image provided by BL2 WARNING: Booting device without SPM initialization. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world
Regards, Jun
Hi,
Did you attempt increasing the maximum allowed translation tables for IMX platform?
Such as https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/imx/im...
With dynamic mapping, it can happen the number of xlat tables works for the boot time, but runs out if you attempt to map regions dynamically.
Regards, Olivier.
________________________________________ From: Jun Nie jun.nie@linaro.org Sent: 27 April 2021 10:40 To: Manish Pandey2 Cc: Gyorgy Szing; Achin Gupta; tf-a@lists.trustedfirmware.org; nd; Olivier Deprez; Lanfang Zhang Subject: Re: To what address BL2 load SPM manifest in FIP image
Hi,
Anyone who is familiar with ATF memory map has idea on this? I see permission error in below line when plat_spm_core_manifest_load() calls mmap_add_dynamic_region(). While fvp does not has such issue.
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/xlat_ta...
I am wandering why fvp manifest mapping check does not fall into the error. Because static memory map on FVP include ARM_SHARED_RAM_BASE 0x04000000 size: 0x00001000. While below manifest start address and size PAGE_SIZE should fall into fully_overlapped_va check and fail.
bl32_image_ep_info.args.arg0 = ARM_TRUSTED_SRAM_BASE + PLAT_ARM_TRUSTED_SRAM_SIZE - PAGE_SIZE;
Regards, Jun
Jun Nie jun.nie@linaro.org 于2021年4月23日周五 下午5:35写道：
Hi,
How fvp board handle the manifest memory map? Carve the memory out and use a dedicated memory map? I cannot find the code. But I fail in this line when I try to map and load manifest. I have to shrink DRAM size so that manifest memory is dynamically mapped.
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/xlat_ta...
Regards, Jun
Olivier Deprez Olivier.Deprez@arm.com 于2021年4月22日周四 下午11:10写道：
Hi,
the TOS fw config address comes from the firmware configuration framework: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/bo...
The fvp_fw_config dtb itself must be loaded to get this information. So if not using fconf on IMX you may have to hardcode the tos fw config blob address in some way in the platform port.
Regards, Olivier.
From: Jun Nie jun.nie@linaro.org Sent: 22 April 2021 16:56 To: Olivier Deprez Cc: Gyorgy Szing; Achin Gupta; tf-a@lists.trustedfirmware.org; Manish Pandey2; nd Subject: To what address BL2 load SPM manifest in FIP image
Hi,
I see below code for manifest address calculation in arm platform. But it is ARM platform specific. I am not sure how the manifest load address is decided when BL2 loads it. There is no image address in this code. https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/co.... Below calculation is wrong on iMX platform and cause below error.
bl32_image_ep_info.args.arg0 = BL32_BASE + BL32_SIZE - PAGE_SIZE;
125 /* SPM (hafnium in secure world) expects SPM Core manifest base address 126 * in x0, which in !RESET_TO_BL31 case loaded after base of non shared 127 * SRAM(after 4KB offset of SRAM). But in RESET_TO_BL31 case all non 128 * shared SRAM is allocated to BL31, so to avoid overwriting of manifest 129 * keep it in the last page. 130 */ 131 bl32_image_ep_info.args.arg0 = ARM_TRUSTED_SRAM_BASE + 132 PLAT_ARM_TRUSTED_SRAM_SIZE - PAGE_SIZE;
ERROR: Error while mapping SPM Core manifest (-1). WARNING: No or invalid SPM Core manifest image provided by BL2 WARNING: Booting device without SPM initialization. ERROR: Error initializing runtime service std_svc INFO: BL31: Preparing for EL3 exit to normal world
Regards, Jun
tf-a@lists.trustedfirmware.org
-
Achin Gupta -
Gyorgy Szing -
Jun Nie -
Manish Pandey2 -
Olivier Deprez