New subject: BL31 as bootloader

26 Oct 2020

Hi Ravi
As you can see in debugger that memory at 0x80020000 address is populated correctly, can you set a breakpoint at it to check if execuion reaches it? If you cannot set a breakpoint I would suggest to add a few assembler instructions in the start of your BL33 entry point to write any specific character, e.g. '!'  to debug UART data port and branch to the next instruction in an infinite loop with "b .".  Seeing ! char on debug console would  prove that control to BL33 image has been passed.
Regards.
Alexei
________________________________
From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of rkohli2000 gmail via TF-A tf-a@lists.trustedfirmware.org
Sent: 24 October 2020 22:48
To: Ravi Kohli rkohli2000@gmail.com
Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org
Subject: Re: [TF-A] BL31 as bootloader
Hi Alexei, Manish,
I instrumented the imx8QM platform code and saw normal execution with bl31_main()
exiting normally with entry point for my BL33 image. But, I haven't seen the handoff
work yet. Can you suggest what else I can try to debug this ? I have an image copied
at 0x80020000 from what I can see in the debugger. Thanks in advance.
Regards
Ravi
On Fri, Oct 23, 2020 at 3:43 PM rkohli2000 gmail <rkohli2000@gmail.commailto:rkohli2000@gmail.com> wrote:
Hi Manish,
I have been able to copy the test app (BL33) to entry point address 0x80020000.
I confirmed from the Debugger that RAM address 0x80020000 is populated. I had to modify imx-atf
makefile target to use the "-data" option as follows:
flash_test_8: $(MKIMG) mx8qm-ahab-container.img scfw_tcm.bin bl31.bin App.bin imx8qm-mek-ca53.dtb
./$(MKIMG) -soc QM -rev B0 -append mx8qm-ahab-container.img -c -scfw scfw_tcm.bin -ap bl31.bin a53 0x80000000 -data App.bin 0x80020000 -data imx8qm-mek-ca53.dtb 0x83000000 -out flash.bin
The -data option appears to copy the image at the 0x80020000 address. Now, I think we should
be able to handoff since PC (program counter) is set :
bl33_image_ep_info.pc = plat_get_ns_image_entrypoint();
I don't see any serial console output using an image that works when using u-boot. I am not using
u-boot for certification reasons. Is there some way I can confirm that the handoff is happening from bl31.bin to
the App.bin ? At which BL31 function or module does the handoff occur ?
I don't see my HW breakpoint trigger at location location 0x80020000.
Regards
Ravi
On Oct 23, 2020, at 9:25 AM, rkohli2000 gmail via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> wrote:
Hi Manish,
Can you please point me to any example or a BL2 module which does the copy to RAM from Flash ?
Maybe I can try to copy the image to 0x80020000 before exiting BL31 from bl31_main().
The bl31.bin image boot fine on the target board (imx8qm) from location 0x80000000
to start with. That copy to 0x80000000 must happen as well. I'm not sure where assuming
its some NXP firmware doing it. Do you know where it could be ?
Regards
Ravi
On Oct 23, 2020, at 4:46 AM, Manish Pandey2 <Manish.Pandey2@arm.commailto:Manish.Pandey2@arm.com> wrote:
Hi Ravi,
This is normal behaviour when you RESET_TO_BL31.
The loading of images (from flash to RAM) is part of BL2 code and in case of directly jumping to BL31, you need a mechanism to Load these images at proper location.
Some platforms have a separate firmware which does this before starting execution of BL31, so that when BL31 hands over to BL33 it gets valid image there.
thanks
Manish
________________________________
From: rkohli2000 gmail <rkohli2000@gmail.commailto:rkohli2000@gmail.com>
Sent: 23 October 2020 05:08
To: Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com>; Manish Pandey2 <Manish.Pandey2@arm.commailto:Manish.Pandey2@arm.com>
Cc: tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] BL31 as bootloader
Hi Alexei, Manish,
I repeated the test (below) a few times and see the same result.
(1) Do I need to copy my BL33 (non-secure) image from
flash.bin some how to RAM 0x80020000 entry point address ?
Is there any example of how to copy the flash.bin image ? I not sure
what address or where to copy it from.
(2) Is there some existing debug code that I can use
to dump/print the ARMv8 RAM address space from inside
BL31.bin ?
I only found some TF-A tf_log macros in debug.h.
I would like to dump some memory address contents
(like 0x80020000) from inside bl31.bin to console just for
debugging at run-time.
On Oct 22, 2020, at 4:28 PM, rkohli2000 gmail via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> wrote:
Hi Alexei,
I captured the LOG_LEVEL=50 log (attached putty-ser0.loghttps://www.dropbox.com/s/bi3hngmljksxl3a/putty-ser0.log?dl=0) from TF-A bl31.bin.
I see bl31 boots normally but never hands off to the BL33 normal world image (App.bin)
The flash.bin image I am running is constructed using imx-mkimagehttps://source.codeaurora.org/external/imx/imx-mkimage/tree/README?h=imx_4.14.98_2.0.0_ga tool with the following target:
 flash_test_7: $(MKIMG) mx8qm-ahab-container.img scfw_tcm.bin  bl31.bin App.bin imx8qm-mek-ca53.dtb
./$(MKIMG) -soc QM -rev B0 -append mx8qm-ahab-container.img -c -flags 0x00200000 -scfw scfw_tcm.bin -ap bl31.bin a53 0x80000000 -p3 -ap App.bin  a53 0x80020000 -data imx8qm-mek-ca53.dtb 0x83000000 -out flash.bin
I used UUU tool to flash eMMC with flash.bin on the imx8QM MEK board.
I inspected RAM using a trace32 debugger SW after using a while (1) to halt the  execution at the end of bl31.bin main function (bl31_main()).

diff --git a/bl31/bl31_main.c b/bl31/bl31_main.c
index 92a2027dd..51afb13ea 100644
--- a/bl31/bl31_main.c
+++ b/bl31/bl31_main.c
@@ -147,6 +147,11 @@ void bl31_main(void)
         * from BL31
         */
        bl31_plat_runtime_setup();
+
+    INFO("BL31: leaving bl31_main\n");
+    INFO("entering while(1)\n");
+
+    while (1) {};
 }
It appears that there's no data or image at 0x80020000 just before exiting bl31_main(). See the debugger RAM dump at
address 0x80000000 (bl31.bin entry point) and 0x80020000 (BL33 or App.bin normal world app entry point).
See the debugger outputhttps://www.dropbox.com/s/revv4qev2ky5djj/ksnip_20201022-153339.png?dl=0.
When should the flash image provided in flash.bin get copied to the RAM entry point 0x80020000 ? Should bl31.bin
copy the image from flash to RAM ?
Can you suggest what could be going on ?
Regards
Ravi
On Oct 22, 2020, at 6:15 AM, Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com> wrote:
Hi Ravi,
TFTF image can be built from https://git.trustedfirmware.org/TF-A/tf-a-tests.git
You can also try to increase TF-A log level by setting LOG_LEVEL=50 and check if BL33 memory region is mapped correctly.
Before passing control to BL33 and could add code to dump initial memory at BL33 start address to see if the image was loaded with no issues.
Regards.
Alexei
________________________________
From: rkohli2000 gmail <rkohli2000@gmail.commailto:rkohli2000@gmail.com>
Sent: 21 October 2020 19:59
To: Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com>
Cc: Manish Pandey2 <Manish.Pandey2@arm.commailto:Manish.Pandey2@arm.com>; tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] BL31 as bootloader
Alexei,
I don't have a DS-5 debugger setup/licensed. I am hoping
on getting something shortly.
Can you point me to the TFTF image that I can use to
test the BL33 handoff ?
Sorry, I'm not familiar at this time.
Regards
Ravi
On Oct 21, 2020, at 12:35 PM, Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com> wrote:
You can also use TFTF image as BL33.
Alexei
________________________________
From: Manish Pandey2 <Manish.Pandey2@arm.commailto:Manish.Pandey2@arm.com>
Sent: 21 October 2020 15:26
To: rkohli2000 gmail <rkohli2000@gmail.commailto:rkohli2000@gmail.com>; Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com>
Cc: tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] BL31 as bootloader
The best way to figure out whether handing off to BL33 happened or not is by attaching a debugger(DS-5).
is there any other A53 (or A72) test image (hello world like) I can validate with to further debug ?
 - You can use a linux image and device tree to test it
Refer to plat/brcm/common/brcm_bl31_setup.c -> "brcm_bl31_early_platform_setup()" function
For linux as BL33 payload please use coder under "ARM_LINUX_KERNEL_AS_BL33"
________________________________
From: rkohli2000 gmail <rkohli2000@gmail.commailto:rkohli2000@gmail.com>
Sent: 21 October 2020 12:12
To: Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com>
Cc: Manish Pandey2 <Manish.Pandey2@arm.commailto:Manish.Pandey2@arm.com>; tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] BL31 as bootloader
Hi Alexei, Manish,
I tried the following patch to  plat/imx/imx8qm/imx8qm_bl31_setup.c:
@@ -483,11 +486,15 @@ void bl31_early_platform_setup2(u_register_t arg0, u_register_t arg1,
        bl32_image_ep_info.args.arg3 = BL32_FDT_OVERLAY_ADDR;
 #endif
 #endif
+    // DEBUG ONLY - FIXME
+    SET_PARAM_HEAD(&bl33_image_ep_info, PARAM_EP, VERSION_1,  0);
+
        SET_SECURITY_STATE(bl33_image_ep_info.h.attr, NON_SECURE);
/* init the first cluster's cci slave interface */
        cci_init(PLAT_CCI_BASE, imx8qm_cci_map, PLATFORM_CLUSTER_COUNT);
        cci_enable_snoop_dvm_reqs(MPIDR_AFFLVL1_VAL(read_mpidr_el1()));
+
 }
But no luck.
Is there some way to confirm my bl31.bin is handing off to any BL33 (normal world) test image ?
In other words, is there any other A53 (or A72) test image (hello world like) I can validate with to further debug ?
Thanks.
On Oct 21, 2020, at 6:10 AM, Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com> wrote:
Hi Ravi,
You can take a look at arm_bl31_early_platform_setup() in plat\arm\common\arm_bl31_setup.c:
/* Populate entry point information for BL33 */
SET_PARAM_HEAD(&bl33_image_ep_info,
PARAM_EP,
VERSION_1,
0);
/*
* Tell BL31 where the non-trusted software image
* is located and the entry state information
*/
bl33_image_ep_info.pc = plat_get_ns_image_entrypoint();
bl33_image_ep_info.spsr = arm_get_spsr_for_bl33_entry();
SET_SECURITY_STATE(bl33_image_ep_info.h.attr, NON_SECURE);
Regards.
Alexei
________________________________
From: TF-A <tf-a-bounces@lists.trustedfirmware.orgmailto:tf-a-bounces@lists.trustedfirmware.org> on behalf of rkohli2000 gmail via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Sent: 21 October 2020 10:57
To: Manish Pandey2 <Manish.Pandey2@arm.commailto:Manish.Pandey2@arm.com>
Cc: tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] BL31 as bootloader
Hi Manish,
The test image should not have any dependency on device tree (dtb?)
for console init.
I used the NXP provided imx8qm-mek-ca53.dtb file which should match
the MEK board.
I have tested removing "--data imx8qm-mek-ca53.dtb 0x83000000" (below)
as well but no luck.
I don't believe PC is getting set to the 0x80020000 entry point.
The test image has validated with u-boot using these tftpboot settings:
    setenv ipaddr x.x.x.x
    setenv serverip x.x.x.x
    tftpboot 0xf0000000 App.bin
    bootm 0xf0000000
I don't see any console output on the screen running it with bl31.bin as below.
Is there any debug I can add somewhere to help troubleshoot?
Can you please provide me instructions on where to patch this missing code :
"SET_PARAM_HEAD(&bl33_image_ep_info, PARAM_EP, VERSION_1,  0);"
I can try it to see if any change in behavior.
Thanks in advance.
Regards
Ravi
On Wed, Oct 21, 2020 at 5:15 AM Manish Pandey2 <Manish.Pandey2@arm.commailto:Manish.Pandey2@arm.com> wrote:
...
Hi Ravi,
Can you please confirm if control reached "test image" ? I guess, yes, as PC has right value.
Also, does your "test image" depends on device tree for console initialization?
One thing i see missing in "plat/imx/imx8qm/imx8qm_bl31_setup.c +352" is bl33 header initialization
SET_PARAM_HEAD(&bl33_image_ep_info, PARAM_EP, VERSION_1,  0);
thanks
Manish

From: TF-A <tf-a-bounces@lists.trustedfirmware.orgmailto:tf-a-bounces@lists.trustedfirmware.org> on behalf of rkohli2000 gmail via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Sent: 21 October 2020 00:10
To: Ravi Kohli <rkohli2000@gmail.commailto:rkohli2000@gmail.com>
Cc: tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] BL31 as bootloader
Hi Alexei,
I built my TF-A (imx-atf) b31.bin image using the RESET_TO_BL31 build option for the i.MX8QM MEK dev kit.
make DEBUG=1 RESET_TO_BL31=1 PLAT=imx8qm bl31
I used the imx-mkimage tool to generate a flash.bin (flash bootable image) with the following target to run on the MEK:
flash_cortex_a53: $(MKIMG) mx8qm-ahab-container.img scfw_tcm.bin bl31.bin MyA53Serial.bin imx8qm-mek-ca53.dtb
./$(MKIMG) -soc QM -rev B0 -append mx8qm-ahab-container.img -c flags 0x00200000 -scfw scfw_tcm.bin -ap bl31.bin a53 0x80000000 -c -p3 -ap MyA53Serial.bin a53 0x80020000 -p4 --data imx8qm-mek-ca53.dtb 0x83000000 -out flash.bin
Here, I allocated bl31.bin to boot from 0x80000000 and a test image at BL33 (normal world) entry point 0x80020000 (both for Cortex-A53).
I can see DEBUG console output from the bl31.bin image but no serial output from my test image I am trying to boot from 0x80020000.
NOTICE:  Memreg 3 0x38000000 -- 0x3bffffff
NOTICE:  Memreg 4 0x60000000 -- 0x6fffffff
NOTICE:  Memreg 5 0x70000000 -- 0x7fffffff
NOTICE:  Memreg 6 0x80000000 -- 0xffffffff
NOTICE:  Memreg 7 0x400000000 -- 0x43fffffff
NOTICE:  Memreg 8 0x880000000 -- 0x97fffffff
NOTICE:  Non-secure Partitioning Succeeded
NOTICE:  BL31: v2.2(debug):imx_5.4.24_er3-1-g06450210f-dirty
NOTICE:  BL31: Built : 17:33:32, Oct 20 2020
INFO: bl31_platform_setup is called
INFO:    GICv3 with legacy support detected. ARM GICv3 driver initialized in EL3
INFO:    BL31: Initializing runtime services
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x80020000
INFO:    SPSR = 0x3c9
INFO: BL31: DEBUG: image_type is: normal
Can anyone please suggest what could be the issue here that I don't see the test image console output ?
Note, the same test image works fine using u-boot.
Regards
Ravi
On Oct 7, 2020, at 12:43 PM, rkohli2000 gmail via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> wrote:
Hi Alexei,
Thanks for your reply. I have not verified the RESET_TO_BL1 for imx8qm yet.
I will try it out and confirm. And, thanks for this suggestion.
Regards
Ravi
On Oct 7, 2020, at 12:12 PM, Alexei Fedorov <Alexei.Fedorov@arm.commailto:Alexei.Fedorov@arm.com> wrote:
Hi Ravi,
Have you tried to use RESET_TO_BL31 build option for your platform?
Regards.
Alexei
________________________________
From: TF-A <tf-a-bounces@lists.trustedfirmware.orgmailto:tf-a-bounces@lists.trustedfirmware.org> on behalf of rkohli2000 gmail via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Sent: 07 October 2020 17:01
To: tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org>
Subject: [TF-A] BL31 as bootloader
Hi,
I'm a new user and sorry for some basic TF-A questions. Any guidance is appreciated.
I'm am able boot the TF-A bl31.bin image itself on my i.MX8QM MEK platform without using u-boot.
I can use the imx-mkimage tool to create a flash or eMMC bootable image (flash.bin). Here, I can
specify this container image with both bl31.bin and a separate custom app at a give flash address.
This is without any security requirements or dependencies.
Can I use the T-FA bl31.bin image to act as a first stage bootloader (without u-boot) and then launch
a "custom" bare metal app for Cortex-A53 (for example) on the i.MX8QM at the given (BL33) entry point
0x80020000 address ?
Thanks in advance.
Ravi
--
TF-A mailing list
TF-A@lists.trustedfirmware.orgmailto:TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A@lists.trustedfirmware.orgmailto:TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A@lists.trustedfirmware.orgmailto:TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A@lists.trustedfirmware.orgmailto:TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A@lists.trustedfirmware.orgmailto:TF-A@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a

    