Hi all,

I am trying to load two OPTEEs on hafnium.

Thanks to the help from Hafnium open source community, now I can load one OPTEE on hafnium. The guidance I am following is attached at the end.

In the guidance, the OPTEE is loaded at address 0x06280000.

I think the first step to load two OPTEEs is to load an OPTEE at an address other than 0x06280000. I tried but failed.

If there is anyone who can give me some advice, I would be very grateful.

=====================The following is the effort I made===============

# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH

1. Loading one OPTEE at address 0x06280000 successfully by following the guidance attached below.

2. In the file <path-to-workspace>/optee_os/core/arch/arm/plat-vexpress/conf.mk change "CFG_TZDRAM_START ?=0x06281000" to "CFG_TZDRAM_START ?= 0x08002000"

3. In the file <path-to-workspace>//trusted-firmware-a/fdts/optee_sp_manifest.dts change "load-address = <0x6280000>" to "load-address = <0x8001000>"

4. Compile OPTEE CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make

5. Compile ATF make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip

6. Run FVP ../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1

7. The error message from fvp-uart0.log (fvp-uart1.log is empty)

NOTICE: Booting Trusted Firmware NOTICE: BL1: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL1: Built : 09:39:24, Feb 25 2021 INFO: BL1: RAM 0x4035000 - 0x403c000 INFO: Loading image id=31 at address 0x4001010 INFO: Image id=31 loaded: 0x4001010 - 0x4001234 INFO: FCONF: Config file with image ID:31 loaded at address = 0x4001010 INFO: Loading image id=24 at address 0x4001300 INFO: Image id=24 loaded: 0x4001300 - 0x4001466 INFO: FCONF: Config file with image ID:24 loaded at address = 0x4001300 INFO: BL1: Loading BL2 INFO: Loading image id=1 at address 0x4022000 INFO: Image id=1 loaded: 0x4022000 - 0x402bb49 NOTICE: BL1: Booting BL2 INFO: Entry point address = 0x4022000 INFO: SPSR = 0x3c5 INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x4001010 INFO: FCONF: Reading firmware configuration information for: dyn_cfg INFO: FCONF: Reading TB_FW firmware configuration file from: 0x4001300 INFO: FCONF: Reading firmware configuration information for: arm_sp NOTICE: BL2: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL2: Built : 09:39:26, Feb 25 2021 INFO: BL2: Doing platform setup INFO: Configuring TrustZone Controller INFO: Total 4 regions set. INFO: BL2: Loading image id 3 INFO: Loading image id=3 at address 0x4003000 INFO: Image id=3 loaded: 0x4003000 - 0x401722c INFO: BL2: Loading image id 23 INFO: Loading image id=23 at address 0x82000000 INFO: Image id=23 loaded: 0x82000000 - 0x82002348 INFO: BL2: Loading image id 25 INFO: Loading image id=25 at address 0x4001300 INFO: Image id=25 loaded: 0x4001300 - 0x4001348 INFO: BL2: Loading image id 4 INFO: Loading image id=4 at address 0x6000000 INFO: Image id=4 loaded: 0x6000000 - 0x601e250 INFO: BL2: Skip loading image id 21 INFO: BL2: Skip loading image id 22 INFO: BL2: Loading image id 26 INFO: Loading image id=26 at address 0x4001500 INFO: Image id=26 loaded: 0x4001500 - 0x4001acf INFO: BL2: Loading image id 5 INFO: Loading image id=5 at address 0x88000000 INFO: Image id=5 loaded: 0x88000000 - 0x8800b2b0 INFO: BL2: Loading image id 34 INFO: Loading image id=34 at address 0x6280000 INFO: Image id=34 loaded: 0x6280000 - 0x62e2050 NOTICE: BL1: Booting BL31 INFO: Entry point address = 0x4003000 INFO: SPSR = 0x3cd INFO: FCONF: Reading HW_CONFIG firmware configuration file from: 0x82000000 INFO: FCONF: Reading firmware configuration information for: cpu_timer INFO: FCONF: Reading firmware configuration information for: uart_config INFO: FCONF: Reading firmware configuration information for: topology INFO: FCONF: Reading firmware configuration information for: gicv3_config NOTICE: BL31: v2.4(debug):v2.4-79-gb4b23c780-dirty NOTICE: BL31: Built : 09:39:31, Feb 25 2021 INFO: GICv3 with legacy support detected. INFO: ARM GICv3 driver initialized in EL3 INFO: BL31: Initializing runtime services INFO: SPM Core setup done. INFO: BL31: Initializing BL32 NOTICE: Initialising hafnium INFO: text: 0x6000000 - 0x601a000 INFO: rodata: 0x601a000 - 0x601e000 INFO: data: 0x601e000 - 0x60a0000 INFO: Supported bits in physical address: 40 INFO: Stage 2 has 3 page table levels with 2 pages at the root. INFO: SPM: 0x84000069 0x0 0x0 0x0 0x0 0x0 0x0 0x0 INFO: Memory range: 0x6000000 - 0x7ffffff INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x0 0x6001340 0x601e000 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10300 0x6001340 0x601e0a8 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10200 0x6001340 0x601e090 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10100 0x6001340 0x601e078 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x10000 0x6001340 0x601e060 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x300 0x6001340 0x601e048 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x200 0x6001340 0x601e030 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: SPM: 0x8400006f 0xffffffff8000ffff 0x0 0x1 0x100 0x6001340 0x601e018 0x0 INFO: In case FFA_MSG_SEND_DIRECT_REQ_SMC32, branch1 (this message is added by myself) INFO: Loaded primary VM with 8 vCPUs, entry at 0x8001000. INFO: Hafnium initialisation completed WARNING: Stage-2 page fault: pc=0x8002000, vmid=32769, vcpu=0, vaddr=0x8002000, ipaddr=0x0, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. WARNING: Stage-2 page fault: pc=0x200, vmid=32769, vcpu=0, vaddr=0x200, ipaddr=0x200, mode=0x4 NOTICE: Injecting Instruction Abort exception into VM32769. . . . (infinite loop . . .)

============The guidance from Oliver Deprez to load one OPTEE============

# NOTE: make sure gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf/bin is in PATH

mkdir workspace; cd workspace

# Hafnium # https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/do...

git clone https://git.trustedfirmware.org/hafnium/hafnium.git; cd hafnium git fetch "https://review.trustedfirmware.org/hafnium/hafnium" refs/changes/11/6011/16 && git checkout FETCH_HEAD git checkout -b spmc git submodule update --init make PROJECT=reference cd ..

# OPTEE # https://optee.readthedocs.io/en/latest/building/prerequisites.html#prerequis...

git clone https://git.trustedfirmware.org/OP-TEE/optee_os.git -b ffa_rel_proto; cd optee_os

# Adjust TEE start address perl -pi -e 's/CFG_TZDRAM_START ?= 0x06280000/CFG_TZDRAM_START ?= 0x06281000/' core/arch/arm/plat-vexpress/conf.mk

CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=fvp CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make cd ..

# TF-A-tests

git clone https://git.trustedfirmware.org/TF-A/tf-a-tests.git; cd tf-a-tests make CROSS_COMPILE=aarch64-none-elf- PLAT=fvp DEBUG=1 TESTS=spm -j4 cd ..

# TF-A # git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git; cd trusted-firmware-a git checkout b4b23c780c0763ce01ac21de1a0e79287677405f -b spmc

#{ # "op-tee" : { # "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", # "pm": "fdts/optee_sp_manifest.dts" # } #} echo \ "{ \ "op-tee" : { \ "image": "../optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin", \ "pm": "fdts/optee_sp_manifest.dts" \ } }" > sp_layout.json

make CROSS_COMPILE=aarch64-none-elf- SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=fvp BL33=../tf-a-tests/build/fvp/debug/tftf.bin DEBUG=1 BL32=../hafnium/out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ARM_ARCH_MINOR=4 SP_LAYOUT_FILE=sp_layout.json ARM_SPMC_MANIFEST_DTS=plat/arm/board/fvp/fdts/fvp_spmc_optee_sp_manifest.dts all fip cd ..

# Run FVP

../fvp/Base_RevC_AEMv8A_pkg/models/Linux64_GCC-6.4/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin -C cluster0.has_arm_v8-4=1 -C cluster1.has_arm_v8-4=1 -C cache_state_modelled=0 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log -C bp.vis.disable_visualisation=1

=============================

I appreciate for any possible help!

Thanks

Yinhua