Hi Oilvier,
Want some clarification on this statement: " A configuration with multiple OP-TEE instances is valid although (to my knowledge) this probably lacks the appropriate OP-TEE support and has never been tested."
Not sure I understand why OPTEE needs support for multiple OPTEE vm's to run on SPMC. If OPTEE needs to be aware that there are other OPTEE partitions, then we are likely doing something wrong with the abstractions. If SPMC supports booting multiple SP's, it should not matter how many other partitions are there or booted unless there is a dependency(which in this case should not exist). Having said that, I assume you mainly wanted to point out that this has never been tested? If not, what support would OPTEE require to boot as multiple instances on an SPMC beyond what Yinhua is doing? What is being done *should* be supported with the latest code base.
Thanks Raghu
-----Original Message----- From: Hafnium hafnium-bounces@lists.trustedfirmware.org On Behalf Of Olivier Deprez via Hafnium Sent: Sunday, February 28, 2021 7:02 AM To: hafnium@lists.trustedfirmware.org; 胤桦 陈 yinhua_chen@outlook.com Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi,
For the first problem: on the FVP platform, Hafnium (SPMC) and Secure Partitions are meant to reside in the "32MB Trusted DRAM" region (0x6000000-0x7ffffff). The memory node in SPMC manifest describes the (collection of) system memory range(s) that the SPMC and SPs are allowed to use. This region is always marked secure on this platform. Using other regions is possible provided they're marked as secure (e.g. by the TZC400 controller) and declared in the memory node (list of start addresses and sizes).
For the second problem: the Hafnium code base used in this sample is rather old and the properties specified in the SPMC manifest are different for the first and other SPs. The first SP fixes vCPU count to number of PEs and maps the full system memory. Other SPs must declare the number of vCPUs and SP memory size in the manifest (https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/do...)
In your sample case you need:
vm2 { is_ffa_partition; debug_name = "op-tee2"; load_address = <0x7280000>; smc_whitelist = <0xce000000>; vcpu_count = <8>; mem_size = <1048576>; };
Using recent Hafnium master, all SPs including the first one declare a vCPU count and SP size in the manifest (http://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/boa...)
Note you'd also want to use different UUIDs for both partitions.
You can get more details on secure partitions loading from https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partiti...
On the rationale for this experiment: Hafnium effectively supports booting multiple partitions. OP-TEE has a build option to run as a single SP on top of the SPMC.
A configuration with multiple OP-TEE instances is valid although (to my knowledge) this probably lacks the appropriate OP-TEE support and has never been tested.
It may be good understanding your use case better, whether you wish to strictly demo the Hafnium multiple partitions support, or you really need to bring up multiple OP-TEE instances.
In other words, would it be enough to host services in bare metal SPs? Would a single OP-TEE partition (redirecting FF-A messages to Trusted Applications implementing secure services) also be acceptable?
Regards, Olivier.
________________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of 胤桦 陈 via Hafnium hafnium@lists.trustedfirmware.org Sent: 27 February 2021 04:11 To: raghu.ncstate@icloud.com; hafnium@lists.trustedfirmware.org Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi raghu,
For the second problem, the memory range is set by myself. The corresponding property is
memory@6000000 { device_type = "memory"; reg = <0x0 0x6000000 0x5000000>; /* Trusted DRAM */ };
in the file fvp_spmc_optee_sp_manifest.dts.
The original setting is reg = <0x0 0x6000000 0x2000000>. Since I find that OPTEE only works fine when loaded at address between 0x6000000 and 0x2000000, I guess the reason is that 0x9280000 is not in the "Trusted DRAM", so I try to reset it to reg = <0x0 0x6000000 0x5000000>. However, it seems not to be the solution.
And, I try to load OPTEE at 0x7281000 and it works fine, no matter with reg = <0x0 0x6000000 0x2000000> or reg = <0x0 0x6000000 0x5000000>.
Appreciate for your help.
Sincerely, Yinhua
________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Saturday, February 27, 2021 2:40 To: 'Manish Pandey2' Manish.Pandey2@arm.com; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org; '胤桦 陈' yinhua_chen@outlook.com Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
Yep, I was just about to reply. First problem is likely because of loading at 0x9280000.
For the second problem, I see “INFO: Memory range: 0x6000000 - 0xaffffff”. On a good run, it should show the memory range as 0x6000000 – 0x7FFFFFF(see the long in your original email). So either your spmc manifest is incorrect or there is something overwriting the memory location of manifest, which may or may not be related to a problem, but needs to be addressed anyway
Would it be possible to run a single OPTEE but loaded at 0x7281000 and see if it works fine? That would also give another data point to diagnose the issue.
From: Manish Pandey2 Manish.Pandey2@arm.com Sent: Friday, February 26, 2021 10:14 AM To: raghu.ncstate@icloud.com; hafnium@lists.trustedfirmware.org; 胤桦 陈 yinhua_chen@outlook.com Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi Yinhua,
I reproduced the first problem at my end, please find my analysis below
If you look at FVP memory map https://developer.arm.com/documentation/100966/1111-00/Arm-Neoverse-referenc... address 0x9280000 is part of NOR flash which is read-only.
Ideally this should have been mapped as RO in BL2's page table, but it is currently mapped as RW at plat/arm/board/fvp/fvp_common.c +83 (mostly a bug, will investigate it)
I did a quick experiment and changed this mapping to RO and system stuck while loading SP
INFO: Loading image id=34 at address 0x9280000 INFO: Image id=34 loaded: 0x9280000 - 0x92e2050
What's happening is, BL2 writes at RO area and thinks that write has been successful and when hafnium tries to match the SP MAGIC it fails.
Will try to investigate your second problem on Monday and let you know.
Thanks
Manish
________________________________
From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> on behalf of 胤桦 陈 via Hafnium <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Sent: 26 February 2021 08:59 To: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
This is the part "My effort and logs for the second problem"
================================My effort and logs for the second problem===================================
1. copy the optee_os directory: cp -r <path-to-workspace>/optee_os <path-to-workspace>/optee_os2 mv <path-to-workspace>/optee_os <path-to-workspace>/optee_os1
2. Set both OPTEE's conf.mk file In the file <path-to-workspace>/optee_os1/core/arch/arm/plat-vexpress/conf.mk set CFG_TZDRAM_START ?=0x06281000 In the file <path-to-workspace>/optee_os2/core/arch/arm/plat-vexpress/conf.mk set CFG_TZDRAM_START ?=0x07281000
3. Set both OPTEE's optee_sp_manifest.dts cp <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts <path-to-workspace>/trusted-firmware-a/fdts/optee2_sp_manifest.dts mv <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts <path-to-workspace>/trusted-firmware-a/fdts/optee1_sp_manifest.dts In the file <path-to-workspace>/trusted-firmware-a/fdts/optee1_sp_manifest.dts set id = <1>; load-address = <0x6280000>; In the file <path-to-workspace>/trusted-firmware-a/fdts/optee2_sp_manifest.dts set id = <2>; load-address = <0x7280000>;
4. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts Set "hypervisor" node like following
hypervisor { compatible = "hafnium,hafnium";
vm1 { is_ffa_partition; debug_name = "op-tee1"; load_address = <0x6280000>; smc_whitelist = <0xbe000000>; };
vm2 { is_ffa_partition; debug_name = "op-tee2"; load_address = <0x7280000>; smc_whitelist = <0xce000000>; }; };
5. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_tb_fw_config.dts Set "secure-partitions" node like following
secure-partitions { compatible = "arm,sp"; #ifdef OPTEE_SP_FW_CONFIG op-tee1 { uuid = <0x486178e0 0xe7f811e3 0xbc5e0002 0xa5d5c51b>; load-address = <0x6280000>; };
op-tee2 { uuid = <0x486178e0 0xe7f811e3 0xbc5e0002 0xa5d5c51b>; load-address = <0x7280000>; }; #else . . (cactus) . #endif };
6. Compile OPTEE1 and OPTEE2
7. Compile ATF
8. Run FVP
Logs from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 15:36:13, Feb 26 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x40014a2 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 15:36:16, Feb 26 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001b1b INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x6280000 INFO: Image id=34 loaded: 0x6280000 - 0x62e2050 INFO: BL2: Loading image id 35 INFO: Loading image id=35 at address 0x7280000 INFO: Image id=35 loaded: 0x7280000 - 0x72e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 15:36:21, Feb 26 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0xaffffff INFO: Valid SP package magic. the valid magic number is 0x474b5053 Panic: Could not parse manifest: Property not found.
======================================================================
Now I am trying to be deep into the code to solve the problems by myself. But if anyone have any idea, I appreciate for your sharing.
Sincerely, Yinhua ________________________________ From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> on behalf of 胤桦 陈 via Hafnium <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Sent: Friday, February 26, 2021 16:48 To: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi raghu,
Thanks for your guidance! I have never heard of fvp_tb_fw_config.dts before! I will spend much more of time on solving my problems without your help!
Now I can load one OPTEE on at addresses between 0x6000000 and 0x8000000 successfully.
Now I meet two problems, I am trying to be deep into the code to solve the problems by myself. But if you have any idea, I appreciate for your sharing.
Two Problems I met:
1. If I load the OPTEE at address not between 0x6000000 and 0x8000000, like 0x9280000, then there will be error "Invalid SP package magic".
2. I try to load two OPTEEs, the first at address 0x6280000, the second at address 0x7280000, and there is an error "Panic: Could not parse manifest: Property not found."
My effort and the detailed logs are as follows.
==================================My effort and logs for the first problem====================================
1. In the file <path-to-workspace>/optee_os/core/arch/arm/plat-vexpress/conf.mk change "CFG_TZDRAM_START ?=0x06281000" to "CFG_TZDRAM_START ?= 0x09281000"
2. In the file <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts change "load-address = <0x6280000>" to "load-address = <0x9280000>"
3. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_tb_fw_config.dts change load-address of op-tee from 0x6280000 to 0x9280000
4. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts change load_address of vm1 from 0x6280000 to 0x9280000
5. Compile OPTEE
CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make
6. Compile ATF
make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip
7. Run FVP ../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
Logs from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 15:36:13, Feb 26 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x4001466 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 15:36:16, Feb 26 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001acf INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x9280000 INFO: Image id=34 loaded: 0x9280000 - 0x92e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 15:36:21, Feb 26 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0xaffffff ERROR: Invalid SP package magic. the invalid magic number is 0xe7ff0010 Panic: Could not parse manifest: Hypervisor manifest entry not compatible with Hafnium.
================================My effort and logs for the second problem===================================
This part is sent by another email because of the limit of body length
=====================================================================================================
Now I am trying to be deep into the code to solve the problems by myself. But if anyone have any idea, I appreciate for your sharing.
Sincerely, Yinhua ________________________________ From: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com> Sent: Friday, February 26, 2021 5:35 To: yinhua_chen@outlook.commailto:yinhua_chen@outlook.com <yinhua_chen@outlook.commailto:yinhua_chen@outlook.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
From the log it looks like there was nothing loaded at 0x8001000. Possible that you are missing the second entry for OPTEE in the SPMC manifest(fvp_spmc_optee_sp_manifest.dts). Might want to use the sptool to check if both your optee images were packaged. If not, you likely have your answer. If it is present, might want to make sure that fvp_tb_fw_config.dts has an entry for the second OPTEE partition to be loaded by BL2 at 0x8001000.
For the partition to run, both OPTEE images have to be loaded by BL2, and both OPTEE images have processed by hafnium during initialization(when it processes the SPMC manifest).
Thanks Raghu
-----Original Message----- From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> On Behalf Of yinhua_chen--- via Hafnium Sent: Wednesday, February 24, 2021 7:23 PM To: hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org Subject: [Hafnium] Trying to load two OPTEEs on hafnium
Hi all,
I am trying to load two OPTEEs on hafnium.
Thanks to the help from Hafnium open source community, now I can load one OPTEE on hafnium. The guidance I am following is attached at the end.
In the guidance, the OPTEE is loaded at address 0x06280000.
I think the first step to load two OPTEEs is to load an OPTEE at an address other than 0x06280000. I tried but failed.
If there is anyone who can give me some advice, I would be very grateful.
=====================The following is the effort I made===============
# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH
1. Loading one OPTEE at address 0x06280000 successfully by following the guidance attached below.
2. In the file <path-to-workspace>/optee_os/core/arch/arm/plat-vexpress/conf.mk change "CFG_TZDRAM_START ?=0x06281000" to "CFG_TZDRAM_START ?= 0x08002000"
3. In the file <path-to-workspace>//trusted-firmware-a/fdts/optee_sp_manifest.dts change "load-address = <0x6280000>" to "load-address = <0x8001000>"
4. Compile OPTEE CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make
5. Compile ATF make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip
6. Run FVP ../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
7. The error message from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 09:39:24, Feb 25 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x4001466 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 09:39:26, Feb 25 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001acf INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x6280000 INFO: Image id=34 loaded: 0x6280000 - 0x62e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 09:39:31, Feb 25 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0x7ffffff INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x0 0x6001340 0x601e000 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10300 0x6001340 0x601e0a8 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10200 0x6001340 0x601e090 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10100 0x6001340 0x601e078 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10000 0x6001340 0x601e060 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x300 0x6001340 0x601e048 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x200 0x6001340 0x601e030 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x100 0x6001340 0x601e018 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: Loaded primary VM with 8 vCPUs, entry at 0x8001000. INFO: Hafnium initialisation completed WARNING: Stage-2 page fault: pc=0x8002000, vmid=32769, vcpu=0, vaddr=0x8002000, ipaddr=0x0, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. . . . (infinite loop . . .)
============The guidance from Oliver Deprez to load one OPTEE============
# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH
mkdir workspace; cd workspace
# Hafnium # https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/do...
git clone https://git.trustedfirmware.org/hafnium/hafnium.git; cd hafnium git fetch "https://review.trustedfirmware.org/hafnium/hafnium" refs/changes/11/6011/16 && git checkout FETCH_HEAD git checkout -b spmc git submodule update --init make PROJECT=reference cd ..
# OPTEE # https://optee.readthedocs.io/en/latest/building/prerequisites.html#prerequis...
git clone https://git.trustedfirmware.org/OP-TEE/optee_os.git -b ffa_rel_proto; cd optee_os
# Adjust TEE start address perl -pi -e 's/CFG_TZDRAM_START ?= 0x06280000/CFG_TZDRAM_START ?= 0x06281000/' core/arch/arm/plat-vexpress/conf.mk
CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make cd ..
# TF-A-tests
git clone https://git.trustedfirmware.org/TF-A/tf-a-tests.git; cd tf-a-tests make CROSS_COMPILE=aarch64-none-elf- PLAT=fvp DEBUG=1 TESTS=spm -j4 cd ..
# TF-A # git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git; cd trusted-firmware-a git checkout b4b23c780c0763ce01ac21de1a0e79287677405f -b spmc
#{ # "op-tee" : { # "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", # "pm": "fdts/optee_sp_manifest.dts" # } #} echo \ "{ \ "op-tee" : { \ "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", \ "pm": "fdts/optee_sp_manifest.dts" \ } }" > sp_layout.json
make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip cd ..
# Run FVP
../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
=============================
I appreciate for any possible help!
Thanks
Yinhua -- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium
-- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium -- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium
Hi Raghu,
You're absolutely right on the long term goal.
The statement relates to what is *immediately* available from Hafnium master and OP-TEE proto branches. First thing which comes to my mind is that OP-TEE (as a SP) assumes it owns the GIC. Two OP-TEE instances would compete to configure the GIC on their own which looks wrong. And ultimately OP-TEE would even be forbidden access to the GIC. Thus, OP-TEE needs to be enhanced to use the Hf GIC para-virtualized interface which is WIP. Other simple things like which UART should each instance use to print debug output? Currently both think they own the same UART and might lead to garbage output. It'd be better each SP uses the Hafnium debug print hyper call (or EL3 print smc service) for example. Beyond booting and those "known" details, when considering the run-time with FF-A/OP-TEE driver in linux, scenarios like communication between one TEE and the secure storage supplicant might not work straight if two OP-TEE instances exist. Global Platform client APIs permit communication with multiple TEEs, but this has to be stitched with FF-A. Without this, it's not possible to launch TAs that are not already embedded into the OP-TEE image.
All in all, indeed launching multiple OP-TEE instances shall be possible, but there is still a (moderate) path to go and deserves appropriate R&D planning.
It is also questioned (in terms of redundancy, footprint, attack surface) whether it's really needed to instantiate multiple instances of the same full fledged TOS (beyond the beauty of the academic experiment) rather than simpler secure services hosted in bare metal secure partitions... I understand the latter is acceptable to Yinhua.
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 28 February 2021 20:17 To: Olivier Deprez Cc: hafnium@lists.trustedfirmware.org Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
Hi Oilvier,
Want some clarification on this statement: " A configuration with multiple OP-TEE instances is valid although (to my knowledge) this probably lacks the appropriate OP-TEE support and has never been tested."
Not sure I understand why OPTEE needs support for multiple OPTEE vm's to run on SPMC. If OPTEE needs to be aware that there are other OPTEE partitions, then we are likely doing something wrong with the abstractions. If SPMC supports booting multiple SP's, it should not matter how many other partitions are there or booted unless there is a dependency(which in this case should not exist). Having said that, I assume you mainly wanted to point out that this has never been tested? If not, what support would OPTEE require to boot as multiple instances on an SPMC beyond what Yinhua is doing? What is being done *should* be supported with the latest code base.
Thanks Raghu
-----Original Message----- From: Hafnium hafnium-bounces@lists.trustedfirmware.org On Behalf Of Olivier Deprez via Hafnium Sent: Sunday, February 28, 2021 7:02 AM To: hafnium@lists.trustedfirmware.org; ط�� �� yinhua_chen@outlook.com Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi,
For the first problem: on the FVP platform, Hafnium (SPMC) and Secure Partitions are meant to reside in the "32MB Trusted DRAM" region (0x6000000-0x7ffffff). The memory node in SPMC manifest describes the (collection of) system memory range(s) that the SPMC and SPs are allowed to use. This region is always marked secure on this platform. Using other regions is possible provided they're marked as secure (e.g. by the TZC400 controller) and declared in the memory node (list of start addresses and sizes).
For the second problem: the Hafnium code base used in this sample is rather old and the properties specified in the SPMC manifest are different for the first and other SPs. The first SP fixes vCPU count to number of PEs and maps the full system memory. Other SPs must declare the number of vCPUs and SP memory size in the manifest (https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/do...)
In your sample case you need:
vm2 { is_ffa_partition; debug_name = "op-tee2"; load_address = <0x7280000>; smc_whitelist = <0xce000000>; vcpu_count = <8>; mem_size = <1048576>; };
Using recent Hafnium master, all SPs including the first one declare a vCPU count and SP size in the manifest (http://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/boa...)
Note you'd also want to use different UUIDs for both partitions.
You can get more details on secure partitions loading from https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partiti...
On the rationale for this experiment: Hafnium effectively supports booting multiple partitions. OP-TEE has a build option to run as a single SP on top of the SPMC.
A configuration with multiple OP-TEE instances is valid although (to my knowledge) this probably lacks the appropriate OP-TEE support and has never been tested.
It may be good understanding your use case better, whether you wish to strictly demo the Hafnium multiple partitions support, or you really need to bring up multiple OP-TEE instances.
In other words, would it be enough to host services in bare metal SPs? Would a single OP-TEE partition (redirecting FF-A messages to Trusted Applications implementing secure services) also be acceptable?
Regards, Olivier.
________________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of ط�� �� via Hafnium hafnium@lists.trustedfirmware.org Sent: 27 February 2021 04:11 To: raghu.ncstate@icloud.com; hafnium@lists.trustedfirmware.org Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi raghu,
For the second problem, the memory range is set by myself. The corresponding property is
memory@6000000 { device_type = "memory"; reg = <0x0 0x6000000 0x5000000>; /* Trusted DRAM */ };
in the file fvp_spmc_optee_sp_manifest.dts.
The original setting is reg = <0x0 0x6000000 0x2000000>. Since I find that OPTEE only works fine when loaded at address between 0x6000000 and 0x2000000, I guess the reason is that 0x9280000 is not in the "Trusted DRAM", so I try to reset it to reg = <0x0 0x6000000 0x5000000>. However, it seems not to be the solution.
And, I try to load OPTEE at 0x7281000 and it works fine, no matter with reg = <0x0 0x6000000 0x2000000> or reg = <0x0 0x6000000 0x5000000>.
Appreciate for your help.
Sincerely, Yinhua
________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Saturday, February 27, 2021 2:40 To: 'Manish Pandey2' Manish.Pandey2@arm.com; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org; 'ط�� ��' yinhua_chen@outlook.com Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
Yep, I was just about to reply. First problem is likely because of loading at 0x9280000.
For the second problem, I see ��INFO: Memory range: 0x6000000 - 0xaffffff��. On a good run, it should show the memory range as 0x6000000 �C 0x7FFFFFF(see the long in your original email). So either your spmc manifest is incorrect or there is something overwriting the memory location of manifest, which may or may not be related to a problem, but needs to be addressed anyway
Would it be possible to run a single OPTEE but loaded at 0x7281000 and see if it works fine? That would also give another data point to diagnose the issue.
From: Manish Pandey2 Manish.Pandey2@arm.com Sent: Friday, February 26, 2021 10:14 AM To: raghu.ncstate@icloud.com; hafnium@lists.trustedfirmware.org; ط�� �� yinhua_chen@outlook.com Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi Yinhua,
I reproduced the first problem at my end, please find my analysis below
If you look at FVP memory map https://developer.arm.com/documentation/100966/1111-00/Arm-Neoverse-referenc... address 0x9280000 is part of NOR flash which is read-only.
Ideally this should have been mapped as RO in BL2's page table, but it is currently mapped as RW at plat/arm/board/fvp/fvp_common.c +83 (mostly a bug, will investigate it)
I did a quick experiment and changed this mapping to RO and system stuck while loading SP
INFO: Loading image id=34 at address 0x9280000 INFO: Image id=34 loaded: 0x9280000 - 0x92e2050
What's happening is, BL2 writes at RO area and thinks that write has been successful and when hafnium tries to match the SP MAGIC it fails.
Will try to investigate your second problem on Monday and let you know.
Thanks
Manish
________________________________
From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> on behalf of ط�� �� via Hafnium <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Sent: 26 February 2021 08:59 To: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
This is the part "My effort and logs for the second problem"
================================My effort and logs for the second problem===================================
1. copy the optee_os directory: cp -r <path-to-workspace>/optee_os <path-to-workspace>/optee_os2 mv <path-to-workspace>/optee_os <path-to-workspace>/optee_os1
2. Set both OPTEE's conf.mk file In the file <path-to-workspace>/optee_os1/core/arch/arm/plat-vexpress/conf.mk set CFG_TZDRAM_START ?=0x06281000 In the file <path-to-workspace>/optee_os2/core/arch/arm/plat-vexpress/conf.mk set CFG_TZDRAM_START ?=0x07281000
3. Set both OPTEE's optee_sp_manifest.dts cp <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts <path-to-workspace>/trusted-firmware-a/fdts/optee2_sp_manifest.dts mv <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts <path-to-workspace>/trusted-firmware-a/fdts/optee1_sp_manifest.dts In the file <path-to-workspace>/trusted-firmware-a/fdts/optee1_sp_manifest.dts set id = <1>; load-address = <0x6280000>; In the file <path-to-workspace>/trusted-firmware-a/fdts/optee2_sp_manifest.dts set id = <2>; load-address = <0x7280000>;
4. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts Set "hypervisor" node like following
hypervisor { compatible = "hafnium,hafnium";
vm1 { is_ffa_partition; debug_name = "op-tee1"; load_address = <0x6280000>; smc_whitelist = <0xbe000000>; };
vm2 { is_ffa_partition; debug_name = "op-tee2"; load_address = <0x7280000>; smc_whitelist = <0xce000000>; }; };
5. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_tb_fw_config.dts Set "secure-partitions" node like following
secure-partitions { compatible = "arm,sp"; #ifdef OPTEE_SP_FW_CONFIG op-tee1 { uuid = <0x486178e0 0xe7f811e3 0xbc5e0002 0xa5d5c51b>; load-address = <0x6280000>; };
op-tee2 { uuid = <0x486178e0 0xe7f811e3 0xbc5e0002 0xa5d5c51b>; load-address = <0x7280000>; }; #else . . (cactus) . #endif };
6. Compile OPTEE1 and OPTEE2
7. Compile ATF
8. Run FVP
Logs from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 15:36:13, Feb 26 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x40014a2 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 15:36:16, Feb 26 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001b1b INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x6280000 INFO: Image id=34 loaded: 0x6280000 - 0x62e2050 INFO: BL2: Loading image id 35 INFO: Loading image id=35 at address 0x7280000 INFO: Image id=35 loaded: 0x7280000 - 0x72e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 15:36:21, Feb 26 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0xaffffff INFO: Valid SP package magic. the valid magic number is 0x474b5053 Panic: Could not parse manifest: Property not found.
======================================================================
Now I am trying to be deep into the code to solve the problems by myself. But if anyone have any idea, I appreciate for your sharing.
Sincerely, Yinhua ________________________________ From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> on behalf of ط�� �� via Hafnium <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Sent: Friday, February 26, 2021 16:48 To: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi raghu,
Thanks for your guidance! I have never heard of fvp_tb_fw_config.dts before! I will spend much more of time on solving my problems without your help!
Now I can load one OPTEE on at addresses between 0x6000000 and 0x8000000 successfully.
Now I meet two problems, I am trying to be deep into the code to solve the problems by myself. But if you have any idea, I appreciate for your sharing.
Two Problems I met:
1. If I load the OPTEE at address not between 0x6000000 and 0x8000000, like 0x9280000, then there will be error "Invalid SP package magic".
2. I try to load two OPTEEs, the first at address 0x6280000, the second at address 0x7280000, and there is an error "Panic: Could not parse manifest: Property not found."
My effort and the detailed logs are as follows.
==================================My effort and logs for the first problem====================================
1. In the file <path-to-workspace>/optee_os/core/arch/arm/plat-vexpress/conf.mk change "CFG_TZDRAM_START ?=0x06281000" to "CFG_TZDRAM_START ?= 0x09281000"
2. In the file <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts change "load-address = <0x6280000>" to "load-address = <0x9280000>"
3. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_tb_fw_config.dts change load-address of op-tee from 0x6280000 to 0x9280000
4. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts change load_address of vm1 from 0x6280000 to 0x9280000
5. Compile OPTEE
CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make
6. Compile ATF
make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip
7. Run FVP ../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
Logs from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 15:36:13, Feb 26 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x4001466 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 15:36:16, Feb 26 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001acf INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x9280000 INFO: Image id=34 loaded: 0x9280000 - 0x92e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 15:36:21, Feb 26 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0xaffffff ERROR: Invalid SP package magic. the invalid magic number is 0xe7ff0010 Panic: Could not parse manifest: Hypervisor manifest entry not compatible with Hafnium.
================================My effort and logs for the second problem===================================
This part is sent by another email because of the limit of body length
=====================================================================================================
Now I am trying to be deep into the code to solve the problems by myself. But if anyone have any idea, I appreciate for your sharing.
Sincerely, Yinhua ________________________________ From: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com> Sent: Friday, February 26, 2021 5:35 To: yinhua_chen@outlook.commailto:yinhua_chen@outlook.com <yinhua_chen@outlook.commailto:yinhua_chen@outlook.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
From the log it looks like there was nothing loaded at 0x8001000. Possible that you are missing the second entry for OPTEE in the SPMC manifest(fvp_spmc_optee_sp_manifest.dts). Might want to use the sptool to check if both your optee images were packaged. If not, you likely have your answer. If it is present, might want to make sure that fvp_tb_fw_config.dts has an entry for the second OPTEE partition to be loaded by BL2 at 0x8001000.
For the partition to run, both OPTEE images have to be loaded by BL2, and both OPTEE images have processed by hafnium during initialization(when it processes the SPMC manifest).
Thanks Raghu
-----Original Message----- From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> On Behalf Of yinhua_chen--- via Hafnium Sent: Wednesday, February 24, 2021 7:23 PM To: hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org Subject: [Hafnium] Trying to load two OPTEEs on hafnium
Hi all,
I am trying to load two OPTEEs on hafnium.
Thanks to the help from Hafnium open source community, now I can load one OPTEE on hafnium. The guidance I am following is attached at the end.
In the guidance, the OPTEE is loaded at address 0x06280000.
I think the first step to load two OPTEEs is to load an OPTEE at an address other than 0x06280000. I tried but failed.
If there is anyone who can give me some advice, I would be very grateful.
=====================The following is the effort I made===============
# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH
1. Loading one OPTEE at address 0x06280000 successfully by following the guidance attached below.
2. In the file <path-to-workspace>/optee_os/core/arch/arm/plat-vexpress/conf.mk change "CFG_TZDRAM_START ?=0x06281000" to "CFG_TZDRAM_START ?= 0x08002000"
3. In the file <path-to-workspace>//trusted-firmware-a/fdts/optee_sp_manifest.dts change "load-address = <0x6280000>" to "load-address = <0x8001000>"
4. Compile OPTEE CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make
5. Compile ATF make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip
6. Run FVP ../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
7. The error message from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 09:39:24, Feb 25 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x4001466 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 09:39:26, Feb 25 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001acf INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x6280000 INFO: Image id=34 loaded: 0x6280000 - 0x62e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 09:39:31, Feb 25 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0x7ffffff INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x0 0x6001340 0x601e000 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10300 0x6001340 0x601e0a8 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10200 0x6001340 0x601e090 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10100 0x6001340 0x601e078 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10000 0x6001340 0x601e060 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x300 0x6001340 0x601e048 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x200 0x6001340 0x601e030 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x100 0x6001340 0x601e018 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: Loaded primary VM with 8 vCPUs, entry at 0x8001000. INFO: Hafnium initialisation completed WARNING: Stage-2 page fault: pc=0x8002000, vmid=32769, vcpu=0, vaddr=0x8002000, ipaddr=0x0, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. . . . (infinite loop . . .)
============The guidance from Oliver Deprez to load one OPTEE============
# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH
mkdir workspace; cd workspace
# Hafnium # https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/do...
git clone https://git.trustedfirmware.org/hafnium/hafnium.git; cd hafnium git fetch "https://review.trustedfirmware.org/hafnium/hafnium" refs/changes/11/6011/16 && git checkout FETCH_HEAD git checkout -b spmc git submodule update --init make PROJECT=reference cd ..
# OPTEE # https://optee.readthedocs.io/en/latest/building/prerequisites.html#prerequis...
git clone https://git.trustedfirmware.org/OP-TEE/optee_os.git -b ffa_rel_proto; cd optee_os
# Adjust TEE start address perl -pi -e 's/CFG_TZDRAM_START ?= 0x06280000/CFG_TZDRAM_START ?= 0x06281000/' core/arch/arm/plat-vexpress/conf.mk
CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make cd ..
# TF-A-tests
git clone https://git.trustedfirmware.org/TF-A/tf-a-tests.git; cd tf-a-tests make CROSS_COMPILE=aarch64-none-elf- PLAT=fvp DEBUG=1 TESTS=spm -j4 cd ..
# TF-A # git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git; cd trusted-firmware-a git checkout b4b23c780c0763ce01ac21de1a0e79287677405f -b spmc
#{ # "op-tee" : { # "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", # "pm": "fdts/optee_sp_manifest.dts" # } #} echo \ "{ \ "op-tee" : { \ "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", \ "pm": "fdts/optee_sp_manifest.dts" \ } }" > sp_layout.json
make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip cd ..
# Run FVP
../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
=============================
I appreciate for any possible help!
Thanks
Yinhua -- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium
-- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium -- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium
Thanks Olivier. Agree, makes sense.
-Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Monday, March 1, 2021 1:29 AM To: raghu.ncstate@icloud.com Cc: hafnium@lists.trustedfirmware.org Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi Raghu,
You're absolutely right on the long term goal.
The statement relates to what is *immediately* available from Hafnium master and OP-TEE proto branches. First thing which comes to my mind is that OP-TEE (as a SP) assumes it owns the GIC. Two OP-TEE instances would compete to configure the GIC on their own which looks wrong. And ultimately OP-TEE would even be forbidden access to the GIC. Thus, OP-TEE needs to be enhanced to use the Hf GIC para-virtualized interface which is WIP. Other simple things like which UART should each instance use to print debug output? Currently both think they own the same UART and might lead to garbage output. It'd be better each SP uses the Hafnium debug print hyper call (or EL3 print smc service) for example. Beyond booting and those "known" details, when considering the run-time with FF-A/OP-TEE driver in linux, scenarios like communication between one TEE and the secure storage supplicant might not work straight if two OP-TEE instances exist. Global Platform client APIs permit communication with multiple TEEs, but this has to be stitched with FF-A. Without this, it's not possible to launch TAs that are not already embedded into the OP-TEE image.
All in all, indeed launching multiple OP-TEE instances shall be possible, but there is still a (moderate) path to go and deserves appropriate R&D planning.
It is also questioned (in terms of redundancy, footprint, attack surface) whether it's really needed to instantiate multiple instances of the same full fledged TOS (beyond the beauty of the academic experiment) rather than simpler secure services hosted in bare metal secure partitions... I understand the latter is acceptable to Yinhua.
Regards, Olivier.
________________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: 28 February 2021 20:17 To: Olivier Deprez Cc: hafnium@lists.trustedfirmware.org Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
Hi Oilvier,
Want some clarification on this statement: " A configuration with multiple OP-TEE instances is valid although (to my knowledge) this probably lacks the appropriate OP-TEE support and has never been tested."
Not sure I understand why OPTEE needs support for multiple OPTEE vm's to run on SPMC. If OPTEE needs to be aware that there are other OPTEE partitions, then we are likely doing something wrong with the abstractions. If SPMC supports booting multiple SP's, it should not matter how many other partitions are there or booted unless there is a dependency(which in this case should not exist). Having said that, I assume you mainly wanted to point out that this has never been tested? If not, what support would OPTEE require to boot as multiple instances on an SPMC beyond what Yinhua is doing? What is being done *should* be supported with the latest code base.
Thanks Raghu
-----Original Message----- From: Hafnium hafnium-bounces@lists.trustedfirmware.org On Behalf Of Olivier Deprez via Hafnium Sent: Sunday, February 28, 2021 7:02 AM To: hafnium@lists.trustedfirmware.org; 胤桦 陈 yinhua_chen@outlook.com Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi,
For the first problem: on the FVP platform, Hafnium (SPMC) and Secure Partitions are meant to reside in the "32MB Trusted DRAM" region (0x6000000-0x7ffffff). The memory node in SPMC manifest describes the (collection of) system memory range(s) that the SPMC and SPs are allowed to use. This region is always marked secure on this platform. Using other regions is possible provided they're marked as secure (e.g. by the TZC400 controller) and declared in the memory node (list of start addresses and sizes).
For the second problem: the Hafnium code base used in this sample is rather old and the properties specified in the SPMC manifest are different for the first and other SPs. The first SP fixes vCPU count to number of PEs and maps the full system memory. Other SPs must declare the number of vCPUs and SP memory size in the manifest (https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/do...)
In your sample case you need:
vm2 { is_ffa_partition; debug_name = "op-tee2"; load_address = <0x7280000>; smc_whitelist = <0xce000000>; vcpu_count = <8>; mem_size = <1048576>; };
Using recent Hafnium master, all SPs including the first one declare a vCPU count and SP size in the manifest (http://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/boa...)
Note you'd also want to use different UUIDs for both partitions.
You can get more details on secure partitions loading from https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partiti...
On the rationale for this experiment: Hafnium effectively supports booting multiple partitions. OP-TEE has a build option to run as a single SP on top of the SPMC.
A configuration with multiple OP-TEE instances is valid although (to my knowledge) this probably lacks the appropriate OP-TEE support and has never been tested.
It may be good understanding your use case better, whether you wish to strictly demo the Hafnium multiple partitions support, or you really need to bring up multiple OP-TEE instances.
In other words, would it be enough to host services in bare metal SPs? Would a single OP-TEE partition (redirecting FF-A messages to Trusted Applications implementing secure services) also be acceptable?
Regards, Olivier.
________________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of 胤桦 陈 via Hafnium hafnium@lists.trustedfirmware.org Sent: 27 February 2021 04:11 To: raghu.ncstate@icloud.com; hafnium@lists.trustedfirmware.org Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi raghu,
For the second problem, the memory range is set by myself. The corresponding property is
memory@6000000 { device_type = "memory"; reg = <0x0 0x6000000 0x5000000>; /* Trusted DRAM */ };
in the file fvp_spmc_optee_sp_manifest.dts.
The original setting is reg = <0x0 0x6000000 0x2000000>. Since I find that OPTEE only works fine when loaded at address between 0x6000000 and 0x2000000, I guess the reason is that 0x9280000 is not in the "Trusted DRAM", so I try to reset it to reg = <0x0 0x6000000 0x5000000>. However, it seems not to be the solution.
And, I try to load OPTEE at 0x7281000 and it works fine, no matter with reg = <0x0 0x6000000 0x2000000> or reg = <0x0 0x6000000 0x5000000>.
Appreciate for your help.
Sincerely, Yinhua
________________________________ From: raghu.ncstate@icloud.com raghu.ncstate@icloud.com Sent: Saturday, February 27, 2021 2:40 To: 'Manish Pandey2' Manish.Pandey2@arm.com; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org; '胤桦 陈' yinhua_chen@outlook.com Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
Yep, I was just about to reply. First problem is likely because of loading at 0x9280000.
For the second problem, I see “INFO: Memory range: 0x6000000 - 0xaffffff”. On a good run, it should show the memory range as 0x6000000 – 0x7FFFFFF(see the long in your original email). So either your spmc manifest is incorrect or there is something overwriting the memory location of manifest, which may or may not be related to a problem, but needs to be addressed anyway
Would it be possible to run a single OPTEE but loaded at 0x7281000 and see if it works fine? That would also give another data point to diagnose the issue.
From: Manish Pandey2 Manish.Pandey2@arm.com Sent: Friday, February 26, 2021 10:14 AM To: raghu.ncstate@icloud.com; hafnium@lists.trustedfirmware.org; 胤桦 陈 yinhua_chen@outlook.com Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi Yinhua,
I reproduced the first problem at my end, please find my analysis below
If you look at FVP memory map https://developer.arm.com/documentation/100966/1111-00/Arm-Neoverse-referenc... address 0x9280000 is part of NOR flash which is read-only.
Ideally this should have been mapped as RO in BL2's page table, but it is currently mapped as RW at plat/arm/board/fvp/fvp_common.c +83 (mostly a bug, will investigate it)
I did a quick experiment and changed this mapping to RO and system stuck while loading SP
INFO: Loading image id=34 at address 0x9280000 INFO: Image id=34 loaded: 0x9280000 - 0x92e2050
What's happening is, BL2 writes at RO area and thinks that write has been successful and when hafnium tries to match the SP MAGIC it fails.
Will try to investigate your second problem on Monday and let you know.
Thanks
Manish
________________________________
From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> on behalf of 胤桦 陈 via Hafnium <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Sent: 26 February 2021 08:59 To: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
This is the part "My effort and logs for the second problem"
================================My effort and logs for the second problem===================================
1. copy the optee_os directory: cp -r <path-to-workspace>/optee_os <path-to-workspace>/optee_os2 mv <path-to-workspace>/optee_os <path-to-workspace>/optee_os1
2. Set both OPTEE's conf.mk file In the file <path-to-workspace>/optee_os1/core/arch/arm/plat-vexpress/conf.mk set CFG_TZDRAM_START ?=0x06281000 In the file <path-to-workspace>/optee_os2/core/arch/arm/plat-vexpress/conf.mk set CFG_TZDRAM_START ?=0x07281000
3. Set both OPTEE's optee_sp_manifest.dts cp <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts <path-to-workspace>/trusted-firmware-a/fdts/optee2_sp_manifest.dts mv <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts <path-to-workspace>/trusted-firmware-a/fdts/optee1_sp_manifest.dts In the file <path-to-workspace>/trusted-firmware-a/fdts/optee1_sp_manifest.dts set id = <1>; load-address = <0x6280000>; In the file <path-to-workspace>/trusted-firmware-a/fdts/optee2_sp_manifest.dts set id = <2>; load-address = <0x7280000>;
4. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts Set "hypervisor" node like following
hypervisor { compatible = "hafnium,hafnium";
vm1 { is_ffa_partition; debug_name = "op-tee1"; load_address = <0x6280000>; smc_whitelist = <0xbe000000>; };
vm2 { is_ffa_partition; debug_name = "op-tee2"; load_address = <0x7280000>; smc_whitelist = <0xce000000>; }; };
5. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_tb_fw_config.dts Set "secure-partitions" node like following
secure-partitions { compatible = "arm,sp"; #ifdef OPTEE_SP_FW_CONFIG op-tee1 { uuid = <0x486178e0 0xe7f811e3 0xbc5e0002 0xa5d5c51b>; load-address = <0x6280000>; };
op-tee2 { uuid = <0x486178e0 0xe7f811e3 0xbc5e0002 0xa5d5c51b>; load-address = <0x7280000>; }; #else . . (cactus) . #endif };
6. Compile OPTEE1 and OPTEE2
7. Compile ATF
8. Run FVP
Logs from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 15:36:13, Feb 26 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x40014a2 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 15:36:16, Feb 26 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001b1b INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x6280000 INFO: Image id=34 loaded: 0x6280000 - 0x62e2050 INFO: BL2: Loading image id 35 INFO: Loading image id=35 at address 0x7280000 INFO: Image id=35 loaded: 0x7280000 - 0x72e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 15:36:21, Feb 26 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0xaffffff INFO: Valid SP package magic. the valid magic number is 0x474b5053 Panic: Could not parse manifest: Property not found.
======================================================================
Now I am trying to be deep into the code to solve the problems by myself. But if anyone have any idea, I appreciate for your sharing.
Sincerely, Yinhua ________________________________ From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> on behalf of 胤桦 陈 via Hafnium <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Sent: Friday, February 26, 2021 16:48 To: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: Re: [Hafnium] Trying to load two OPTEEs on hafnium
Hi raghu,
Thanks for your guidance! I have never heard of fvp_tb_fw_config.dts before! I will spend much more of time on solving my problems without your help!
Now I can load one OPTEE on at addresses between 0x6000000 and 0x8000000 successfully.
Now I meet two problems, I am trying to be deep into the code to solve the problems by myself. But if you have any idea, I appreciate for your sharing.
Two Problems I met:
1. If I load the OPTEE at address not between 0x6000000 and 0x8000000, like 0x9280000, then there will be error "Invalid SP package magic".
2. I try to load two OPTEEs, the first at address 0x6280000, the second at address 0x7280000, and there is an error "Panic: Could not parse manifest: Property not found."
My effort and the detailed logs are as follows.
==================================My effort and logs for the first problem====================================
1. In the file <path-to-workspace>/optee_os/core/arch/arm/plat-vexpress/conf.mk change "CFG_TZDRAM_START ?=0x06281000" to "CFG_TZDRAM_START ?= 0x09281000"
2. In the file <path-to-workspace>/trusted-firmware-a/fdts/optee_sp_manifest.dts change "load-address = <0x6280000>" to "load-address = <0x9280000>"
3. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_tb_fw_config.dts change load-address of op-tee from 0x6280000 to 0x9280000
4. In the file <path-to-workspace>/trusted-firmware-a/plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts change load_address of vm1 from 0x6280000 to 0x9280000
5. Compile OPTEE
CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make
6. Compile ATF
make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip
7. Run FVP ../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
Logs from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 15:36:13, Feb 26 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x4001466 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 15:36:16, Feb 26 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001acf INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x9280000 INFO: Image id=34 loaded: 0x9280000 - 0x92e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 15:36:21, Feb 26 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0xaffffff ERROR: Invalid SP package magic. the invalid magic number is 0xe7ff0010 Panic: Could not parse manifest: Hypervisor manifest entry not compatible with Hafnium.
================================My effort and logs for the second problem===================================
This part is sent by another email because of the limit of body length
=====================================================================================================
Now I am trying to be deep into the code to solve the problems by myself. But if anyone have any idea, I appreciate for your sharing.
Sincerely, Yinhua ________________________________ From: raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com <raghu.ncstate@icloud.commailto:raghu.ncstate@icloud.com> Sent: Friday, February 26, 2021 5:35 To: yinhua_chen@outlook.commailto:yinhua_chen@outlook.com <yinhua_chen@outlook.commailto:yinhua_chen@outlook.com>; hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org <hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org> Subject: RE: [Hafnium] Trying to load two OPTEEs on hafnium
From the log it looks like there was nothing loaded at 0x8001000. Possible that you are missing the second entry for OPTEE in the SPMC manifest(fvp_spmc_optee_sp_manifest.dts). Might want to use the sptool to check if both your optee images were packaged. If not, you likely have your answer. If it is present, might want to make sure that fvp_tb_fw_config.dts has an entry for the second OPTEE partition to be loaded by BL2 at 0x8001000.
For the partition to run, both OPTEE images have to be loaded by BL2, and both OPTEE images have processed by hafnium during initialization(when it processes the SPMC manifest).
Thanks Raghu
-----Original Message----- From: Hafnium <hafnium-bounces@lists.trustedfirmware.orgmailto:hafnium-bounces@lists.trustedfirmware.org> On Behalf Of yinhua_chen--- via Hafnium Sent: Wednesday, February 24, 2021 7:23 PM To: hafnium@lists.trustedfirmware.orgmailto:hafnium@lists.trustedfirmware.org Subject: [Hafnium] Trying to load two OPTEEs on hafnium
Hi all,
I am trying to load two OPTEEs on hafnium.
Thanks to the help from Hafnium open source community, now I can load one OPTEE on hafnium. The guidance I am following is attached at the end.
In the guidance, the OPTEE is loaded at address 0x06280000.
I think the first step to load two OPTEEs is to load an OPTEE at an address other than 0x06280000. I tried but failed.
If there is anyone who can give me some advice, I would be very grateful.
=====================The following is the effort I made===============
# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH
1. Loading one OPTEE at address 0x06280000 successfully by following the guidance attached below.
2. In the file <path-to-workspace>/optee_os/core/arch/arm/plat-vexpress/conf.mk change "CFG_TZDRAM_START ?=0x06281000" to "CFG_TZDRAM_START ?= 0x08002000"
3. In the file <path-to-workspace>//trusted-firmware-a/fdts/optee_sp_manifest.dts change "load-address = <0x6280000>" to "load-address = <0x8001000>"
4. Compile OPTEE CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make
5. Compile ATF make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip
6. Run FVP ../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
7. The error message from fvp-uart0.log (fvp-uart1.log is empty)
NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 09:39:24, Feb 25 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x4001466 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 09:39:26, Feb 25 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001acf INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x6280000 INFO: Image id=34 loaded: 0x6280000 - 0x62e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 09:39:31, Feb 25 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0x7ffffff INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x0 0x6001340 0x601e000 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10300 0x6001340 0x601e0a8 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10200 0x6001340 0x601e090 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10100 0x6001340 0x601e078 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10000 0x6001340 0x601e060 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x300 0x6001340 0x601e048 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x200 0x6001340 0x601e030 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x100 0x6001340 0x601e018 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: Loaded primary VM with 8 vCPUs, entry at 0x8001000. INFO: Hafnium initialisation completed WARNING: Stage-2 page fault: pc=0x8002000, vmid=32769, vcpu=0, vaddr=0x8002000, ipaddr=0x0, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. . . . (infinite loop . . .)
============The guidance from Oliver Deprez to load one OPTEE============
# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH
mkdir workspace; cd workspace
# Hafnium # https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/do...
git clone https://git.trustedfirmware.org/hafnium/hafnium.git; cd hafnium git fetch "https://review.trustedfirmware.org/hafnium/hafnium" refs/changes/11/6011/16 && git checkout FETCH_HEAD git checkout -b spmc git submodule update --init make PROJECT=reference cd ..
# OPTEE # https://optee.readthedocs.io/en/latest/building/prerequisites.html#prerequis...
git clone https://git.trustedfirmware.org/OP-TEE/optee_os.git -b ffa_rel_proto; cd optee_os
# Adjust TEE start address perl -pi -e 's/CFG_TZDRAM_START ?= 0x06280000/CFG_TZDRAM_START ?= 0x06281000/' core/arch/arm/plat-vexpress/conf.mk
CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make cd ..
# TF-A-tests
git clone https://git.trustedfirmware.org/TF-A/tf-a-tests.git; cd tf-a-tests make CROSS_COMPILE=aarch64-none-elf- PLAT=fvp DEBUG=1 TESTS=spm -j4 cd ..
# TF-A # git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git; cd trusted-firmware-a git checkout b4b23c780c0763ce01ac21de1a0e79287677405f -b spmc
#{ # "op-tee" : { # "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", # "pm": "fdts/optee_sp_manifest.dts" # } #} echo \ "{ \ "op-tee" : { \ "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", \ "pm": "fdts/optee_sp_manifest.dts" \ } }" > sp_layout.json
make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip cd ..
# Run FVP
../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1
=============================
I appreciate for any possible help!
Thanks
Yinhua -- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium
-- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium -- Hafnium mailing list Hafnium@lists.trustedfirmware.orgmailto:Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium
hafnium@lists.trustedfirmware.org
-
Olivier Deprez -
raghu.ncstate＠icloud.com