[TF-M] Re: Issues with psa_wait() when waiting for interrupts and IPC function calls.

The simplest approach is to do all the interrupt handling in the First-Level handler - psa_flih_result_t tfm_timer1_irq_flih(void) And return PSA_FLIH_NO_SIGNAL to mark that no further handling is needed.

Best Regards, Kevin

From: Martin Gunnarsson martin.gunnarsson@ri.se Sent: Wednesday, February 23, 2022 12:52 AM To: Kevin Peng Kevin.Peng@arm.com; Bøe, Sebastian via TF-M tf-m@lists.trustedfirmware.org Subject: Re: Issues with psa_wait() when waiting for interrupts and IPC function calls.

Hi Kevin,

I have not managed to get the debugger working yet, I will continue work on that tomorrow. Ok, then I will disregard the files for the "Library Model" since I use IPC.

I see. I have studied the code for the FLIH test case. The code in the test case is not really what I want to do. What I want to accomplish with my Secure Partition is to periodically execute code within the Secure Processing Enviornment, without any communication with, or blocking the execution of the NSPE. Of course, being a single core system some kind of concurrency and sharing of resources is necessary. I tried to do this by setting an interrupt to trigger periodically, but this approach has been unsuccessful this far.

Do you know if this possible to accomplish? Having an infinite loop that execute periodically within the SPE while the NSPE can execute arbitrary code and call the SPE?

Best wishes,

Martin Gunnarsson

________________________________ From: Kevin Peng <Kevin.Peng@arm.commailto:Kevin.Peng@arm.com> Sent: Tuesday, February 22, 2022 4:07 AM To: Martin Gunnarsson <martin.gunnarsson@ri.semailto:martin.gunnarsson@ri.se>; Bøe, Sebastian via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Issues with psa_wait() when waiting for interrupts and IPC function calls.

Hi Martin,

The files you are looking at are "Library Model" codes, which is another "mode" of TF-M.

But it does not support FLIH interrupt handling and is not the default option.

I suppose you're using the "IPC Model". So please ignore these files.

LR = Link Register.

Do you have a debugger connected to your board and are you able to see the registers of the CPU?

For interrupts in TF-M, the typical use case is not what you are expecting.

If you look at the FLIH test cases, the flow is basically like this:

* A client (could be NS/S) requests a Secure Service * The Secure Partition does some operations and needs to wait for HW to do its job * The Partition waits for the interrupt by psa_wait() * When the interrupt triggers, the Partitions is woken up * It does the reset of the jobs and reply to the client.

Basically the infinite loop in the Partition entry does not handle the interrupt signal.

Best Regards,

Kevin

From: Martin Gunnarsson <martin.gunnarsson@ri.semailto:martin.gunnarsson@ri.se> Sent: Tuesday, February 22, 2022 12:26 AM To: Kevin Peng <Kevin.Peng@arm.commailto:Kevin.Peng@arm.com>; Bøe, Sebastian via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: Re: Issues with psa_wait() when waiting for interrupts and IPC function calls.

Hi,

I did some more thinking regarding the psa_wait() issues. Without access to hardware to run experiments, I looked through the code.

On line 63 in trusted-firmware-m/secure_fw/spm/cmsis_func/arch.c in the psa_wait() function there is the following comment in the code:

63 /* FIXME: By using the 'WFI' instruction this function blocks until an

64 * interrupt happens. It is necessary to do this here as tfm_core_psa_wait

65 * runs with the priority of the SVC, so it cannot be interrupted, so

66 * waiting in it for the required interrupt to happen is not an option.

67 */

In the file trusted-firmware-m/secure_fw/spm/cmsis_func/spm_func.c there is the following comment in the function tfm_spm_psa_wait():

1120 /* FIXME: Scheduling is not available in library model, and busy wait is

1121 * also not possible as this code is running in SVC context, and it

1122 * cannot be pre-empted by interrupts. So do nothing here for now

1123 */

Can this have anything to do with the issue that the NSPE is not preempted?

Sincerely,

Martin Gunnarsson

________________________________

From: Martin Gunnarsson via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Sent: Monday, February 21, 2022 10:22 AM To: Kevin Peng <Kevin.Peng@arm.commailto:Kevin.Peng@arm.com>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: [TF-M] Re: Issues with psa_wait() when waiting for interrupts and IPC function calls.

You don't often get email from tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org. Learn why this is importanthttp://aka.ms/LearnAboutSenderIdentification

Hi Kevin,

Thank you for your swift response!

I build my Application RoT for Zephyr OS, using the latest version of their main branch. I am building for the NRF5340 DK platform, it is a multicore system, but I only use the CPUAPP so I only use one v8m core.

That returning TFM_FLIH_SIGNAL from the Interrupt handler does not tigger a scheduling of the Secure Partition sounds resonable. Is there any way to force a scheduling of a Secure Partition from a First Line Interrupt Handler?

I will try to provide more information, however I do not have my NRF5340 DK on hand. I will provide more information tomorrow. I am a bit unsure what you mean by "by checking the LR" can you please clarify the term LR? By "PE" I suppose you mean Processing Enviornment?

I will provide more information about the signal values tomorrow.

Sincerely,

Martin Gunnarsson

________________________________

From: Kevin Peng <Kevin.Peng@arm.commailto:Kevin.Peng@arm.com> Sent: Monday, February 21, 2022 9:01 AM To: Martin Gunnarsson <martin.gunnarsson@ri.semailto:martin.gunnarsson@ri.se>; tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Subject: RE: Issues with psa_wait() when waiting for interrupts and IPC function calls.

You don't often get email from kevin.peng@arm.commailto:kevin.peng@arm.com. Learn why this is importanthttp://aka.ms/LearnAboutSenderIdentification

Hi Martin,

Thanks for reporting your issue.

May I ask what kind of platform are you working (is it a multi-core or single v8m core?) and what's the build configurations.

I assume you're working on a single-core platform.

Then for the first observation - If PSA_WAIT_ANY is used as a signal mask, the TIMER0_IRQ_SIGNAL will only be received when TFM_SECURE_FUNCTION_SIGNAL is received, ergo, every 10 s. Thus any stage 2 handling of an interrupt would be missed.

I guess it's because the timer 1 interrupt is preempting the NSPE execution.

In this case, due to the limitation of the TrustZone, the Secure interrupt handler only sets the signal for the Secure Partition but does not really schedule the Partition to run.

And then when the NSPE request the TFM_SECURE_FUNCTION, the Secure Partitions is activated.

At this point, both the SECURE_FUNCTION and timer signal are set. The Secure Partition can handle them at will.

For the second observation, I cannot figure out why.

Could you provide more information?

For example what PE is preempted by the timer interrupt, by checking the LR (which should be a EXC_RETURN payload) at the very begging of the interrupt handling entry.

And what's the "signals" value for every psa_wait()?

Best Regards,

Kevin

From: Martin Gunnarsson via TF-M <tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org> Sent: Thursday, February 17, 2022 11:46 PM To: tf-m@lists.trustedfirmware.orgmailto:tf-m@lists.trustedfirmware.org Subject: [TF-M] Issues with psa_wait() when waiting for interrupts and IPC function calls.

Hello,

I am trying to write an Application RoT for a research project I am a part of. I have run into an issue that I have been unable to solve independently.

My problem is the unexpected behavior of psa_wait() when using signal masks.

I have two signals, TFM_TIMER1_IRQ_SIGNAL and TFM_SECURE_FUNCTION_SIGNAL. The TIMER1_IRQ signal is triggered by an FLIH interrupt handler that returns TFM_FLIH_SIGNAL. The interrupt is triggered every 1 second by TIMER1. The TFM_SECURE_FUNCTION_SIGNAL is triggered by a periodic function call from the NSPE (a Zephyr thread) and is called every 10 seconds.

Observed result:

If PSA_WAIT_ANY is used as a signal mask, the TIMER0_IRQ_SIGNAL will only be received when TFM_SECURE_FUNCTION_SIGNAL is received, ergo, every 10 s. Thus any stage 2 handling of an interrupt would be missed.

If TFM_TIMER1_IRQ_SIGNAL is used as a signal mask, the signal will be received every 1 s as expected. The TFM_SECURE_FUNCTION_SIGNAL will never be received, as can be expected in the NSPEtoo.

I have tried a "hybrid approach" where I call psa_wait(TFM_TIMER1_IRQ_SIGNAL, PSA_BLOCK) /* handle signals .... */ and then psa_wait(TFM_SECURE_FUNCTION_SIGNAL, PSA_POLL). The stage 2 IRS is called, and the secure function is also called. But not with the regularity that I would like.

Am I missing something obvious? I have read the Secure Interrupt Integration Guide and carefully studied the SLIH and FLIH test services for the TF-M regression tests. Have I forgotten to reset a signal somewhere? Does the interrupt not preempt the NSPE?

I would be immensely grateful for any or all responses. If you need more information, please let me know.

Best wishes,

Martin Gunnarsson

PhD Student

RISE Research Institutes of Sweden & Lund University

Complete code of my Application RoT:

#include <stdbool.h>

#include <stdint.h>

#include <stddef.h>

#include "tfm_api.h"

#include "tfm_secure_api.h"

#include "psa/service.h"

#include "tfm_sp_log.h"

#include "psa_manifest/pid.h"

#include "psa_manifest/tfm_secure_partition.h"

#include <hal/nrf_gpio.h>

#include <hal/nrf_timer.h>

#include <helpers/nrfx_reset_reason.h>

#include <nrf_board.h>

#include <region_defs.h>

#define TIMER_RELOAD_VALUE (1*1024*1024)

static volatile uint32_t interrupt_ctr = 0;

static volatile uint32_t function_ctr = 0;

static void timer_init(NRF_TIMER_Type * TIMER)

{

nrf_timer_mode_set(TIMER, NRF_TIMER_MODE_TIMER);

nrf_timer_bit_width_set(TIMER, NRF_TIMER_BIT_WIDTH_32);

nrf_timer_frequency_set(TIMER, NRF_TIMER_FREQ_1MHz);

nrf_timer_cc_set(TIMER, NRF_TIMER_CC_CHANNEL0, TIMER_RELOAD_VALUE);

/* Clear the timer once event is generated. */

nrf_timer_shorts_enable(TIMER, NRF_TIMER_SHORT_COMPARE0_CLEAR_MASK);

}

static void timer_stop(NRF_TIMER_Type * TIMER)

{

nrf_timer_task_trigger(TIMER, NRF_TIMER_TASK_STOP);

nrf_timer_int_disable(TIMER, NRF_TIMER_INT_COMPARE0_MASK);

nrf_timer_event_clear(TIMER, NRF_TIMER_EVENT_COMPARE0);

}

static void timer_start(NRF_TIMER_Type * TIMER)

{

timer_stop(TIMER);

nrf_timer_task_trigger(TIMER, NRF_TIMER_TASK_CLEAR);

nrf_timer_int_enable(TIMER, NRF_TIMER_INT_COMPARE0_MASK);

nrf_timer_task_trigger(TIMER, NRF_TIMER_TASK_START);

}

static void timer_event_clear(NRF_TIMER_Type *TIMER)

{

nrf_timer_event_clear(TIMER, NRF_TIMER_EVENT_COMPARE0);

}

void tfm_secure_controller_secure_timer_start(void)

{

timer_init(NRF_TIMER1);

timer_start(NRF_TIMER1);

}

void tfm_secure_controller_secure_timer_clear_intr(void)

{

timer_event_clear(NRF_TIMER1);

}

psa_flih_result_t tfm_timer1_irq_flih(void)

{

tfm_secure_controller_secure_timer_clear_intr();

interrupt_ctr++;

return PSA_FLIH_SIGNAL;

}

static psa_status_t tfm_sp_secure_function_ipc(psa_msg_t *msg)

{

function_ctr++;

LOG_INFFMT("Secure function called %u times, interrupt ctr: %u\n", function_ctr, interrupt_ctr );

return PSA_SUCCESS;

}

void tfm_sp_secure_function(psa_msg_t *msg) {

psa_status_t status;

status = tfm_sp_secure_function_ipc(msg);

psa_reply(msg->handle, status);

}

void tfm_sp_entry(void)

{

LOG_INFFMT("Secure Partition: Init\n");

tfm_secure_controller_secure_timer_start();

psa_irq_enable(TFM_TIMER1_IRQ_SIGNAL);

psa_signal_t signals = 0;

uint32_t ctr = 0;

while (1) {

signals = psa_wait(TFM_TIMER1_IRQ_SIGNAL, PSA_BLOCK);

if ( (signals & TFM_TIMER1_IRQ_SIGNAL) == TFM_TIMER1_IRQ_SIGNAL) {

ctr++;

LOG_INFFMT("FLIH stage 2 %u\n", interrupt_ctr);

psa_reset_signal(TFM_TIMER1_IRQ_SIGNAL);

}

if ( ctr % 10 == 0 ) {

signals = psa_wait(PSA_WAIT_ANY, PSA_POLL);

if ( (signals & TFM_SECURE_FUNCTION_SIGNAL) == TFM_SECURE_FUNCTION_SIGNAL) {

psa_msg_t msg;

psa_get(signals, &msg);

tfm_sp_secure_function(&msg);

}

}

}

}