Hi,

I did some more thinking regarding the psa_wait() issues. Without access to hardware to run experiments, I looked through the code.
On line 63 in trusted-firmware-m/secure_fw/spm/cmsis_func/arch.c in the psa_wait() function there is the following comment in the code:
 63     /* FIXME: By using the 'WFI' instruction this function blocks until an
 64      * interrupt happens. It is necessary to do this here as tfm_core_psa_wait
 65      * runs with the priority of the SVC, so it cannot be interrupted, so
 66      * waiting in it for the required interrupt to happen is not an option.
 67      */

In the file trusted-firmware-m/secure_fw/spm/cmsis_func/spm_func.c there is the following comment in the function tfm_spm_psa_wait():

1120         /* FIXME: Scheduling is not available in library model, and busy wait is
1121          * also not possible as this code is running in SVC context, and it
1122          * cannot be pre-empted by interrupts. So do nothing here for now
1123          */

Can this have anything to do with the issue that the NSPE is not preempted?

Sincerely,

Martin Gunnarsson


From: Martin Gunnarsson via TF-M <tf-m@lists.trustedfirmware.org>
Sent: Monday, February 21, 2022 10:22 AM
To: Kevin Peng <Kevin.Peng@arm.com>; tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.org>
Subject: [TF-M] Re: Issues with psa_wait() when waiting for interrupts and IPC function calls.
 
You don't often get email from tf-m@lists.trustedfirmware.org. Learn why this is important
Hi Kevin,

Thank you for your swift response!
I build my Application RoT for Zephyr OS, using the latest version of their main branch. I am building for the NRF5340 DK platform, it is a multicore system, but I only use the CPUAPP so I only use one v8m core.

That returning TFM_FLIH_SIGNAL from the Interrupt handler does not tigger a scheduling of the Secure Partition sounds resonable. Is there any way to force a scheduling of a Secure Partition from a First Line Interrupt Handler?

I will try to provide more information, however I do not have my NRF5340 DK on hand. I will provide more information tomorrow. I am a bit unsure what you mean by "by checking the LR" can you please clarify the term LR? By "PE" I suppose you mean Processing Enviornment?

I will provide more information about the signal values tomorrow.

Sincerely,
Martin Gunnarsson




From: Kevin Peng <Kevin.Peng@arm.com>
Sent: Monday, February 21, 2022 9:01 AM
To: Martin Gunnarsson <martin.gunnarsson@ri.se>; tf-m@lists.trustedfirmware.org <tf-m@lists.trustedfirmware.org>
Subject: RE: Issues with psa_wait() when waiting for interrupts and IPC function calls.
 

You don't often get email from kevin.peng@arm.com. Learn why this is important

Hi Martin,

 

Thanks for reporting your issue.

May I ask what kind of platform are you working (is it a multi-core or single v8m core?) and what’s the build configurations.

 

I assume you’re working on a single-core platform.

Then for the first observation - If PSA_WAIT_ANY is used as a signal mask, the TIMER0_IRQ_SIGNAL will only be received when TFM_SECURE_FUNCTION_SIGNAL is received, ergo, every 10 s. Thus any stage 2 handling of an interrupt would be missed.

I guess it’s because the timer 1 interrupt is preempting the NSPE execution.

In this case, due to the limitation of the TrustZone, the Secure interrupt handler only sets the signal for the Secure Partition but does not really schedule the Partition to run.

And then when the NSPE request the TFM_SECURE_FUNCTION, the Secure Partitions is activated.

At this point, both the SECURE_FUNCTION and timer signal are set. The Secure Partition can handle them at will.

 

For the second observation, I cannot figure out why.

Could you provide more information?

For example what PE is preempted by the timer interrupt, by checking the LR (which should be a EXC_RETURN payload) at the very begging of the interrupt handling entry.

And what’s the “signals” value for every psa_wait()?

 

Best Regards,

Kevin

 

From: Martin Gunnarsson via TF-M <tf-m@lists.trustedfirmware.org>
Sent: Thursday, February 17, 2022 11:46 PM
To: tf-m@lists.trustedfirmware.org
Subject: [TF-M] Issues with psa_wait() when waiting for interrupts and IPC function calls.

 

Hello,

 

I am trying to write an Application RoT for a research project I am a part of. I have run into an issue that I have been unable to solve independently.

 

My problem is the unexpected behavior of psa_wait() when using signal masks.

I have two signals, TFM_TIMER1_IRQ_SIGNAL and TFM_SECURE_FUNCTION_SIGNAL. The TIMER1_IRQ signal is triggered by an FLIH interrupt handler that returns TFM_FLIH_SIGNAL. The interrupt is triggered every 1 second by TIMER1. The TFM_SECURE_FUNCTION_SIGNAL is triggered by a periodic function call from the NSPE (a Zephyr thread) and is called every 10 seconds.

 

Observed result:

If PSA_WAIT_ANY is used as a signal mask, the TIMER0_IRQ_SIGNAL will only be received when TFM_SECURE_FUNCTION_SIGNAL is received, ergo, every 10 s. Thus any stage 2 handling of an interrupt would be missed.

 

If TFM_TIMER1_IRQ_SIGNAL is used as a signal mask, the signal will be received every 1 s as expected. The TFM_SECURE_FUNCTION_SIGNAL will never be received, as can be expected in the NSPEtoo.

 

I have tried a "hybrid approach" where I call psa_wait(TFM_TIMER1_IRQ_SIGNAL, PSA_BLOCK) /* handle signals .... */ and then psa_wait(TFM_SECURE_FUNCTION_SIGNAL, PSA_POLL). The stage 2 IRS is called, and the secure function is also called. But not with the regularity that I would like.

 

Am I missing something obvious? I have read the Secure Interrupt Integration Guide and carefully studied the SLIH and FLIH test services for the TF-M regression tests. Have I forgotten to reset a signal somewhere? Does the interrupt not preempt the NSPE?

 

I would be immensely grateful for any or all responses. If you need more information, please let me know.

 

Best wishes,

 

Martin Gunnarsson

PhD Student

RISE Research Institutes of Sweden & Lund University

 

Complete code of my Application RoT:

#include <stdbool.h>

#include <stdint.h>

#include <stddef.h>

 

#include "tfm_api.h"

#include "tfm_secure_api.h"

 

#include "psa/service.h"

#include "tfm_sp_log.h"

 

#include "psa_manifest/pid.h"

#include "psa_manifest/tfm_secure_partition.h"

 

#include <hal/nrf_gpio.h>

#include <hal/nrf_timer.h>

#include <helpers/nrfx_reset_reason.h>

#include <nrf_board.h>

#include <region_defs.h>

 

 

#define TIMER_RELOAD_VALUE (1*1024*1024)

static volatile uint32_t interrupt_ctr = 0;

static volatile uint32_t function_ctr = 0;

 

static void timer_init(NRF_TIMER_Type * TIMER)

{

    nrf_timer_mode_set(TIMER, NRF_TIMER_MODE_TIMER);

    nrf_timer_bit_width_set(TIMER, NRF_TIMER_BIT_WIDTH_32);

    nrf_timer_frequency_set(TIMER, NRF_TIMER_FREQ_1MHz);

    nrf_timer_cc_set(TIMER, NRF_TIMER_CC_CHANNEL0, TIMER_RELOAD_VALUE);

    /* Clear the timer once event is generated. */

    nrf_timer_shorts_enable(TIMER, NRF_TIMER_SHORT_COMPARE0_CLEAR_MASK);

}

 

static void timer_stop(NRF_TIMER_Type * TIMER)

{

    nrf_timer_task_trigger(TIMER, NRF_TIMER_TASK_STOP);

    nrf_timer_int_disable(TIMER, NRF_TIMER_INT_COMPARE0_MASK);

    nrf_timer_event_clear(TIMER, NRF_TIMER_EVENT_COMPARE0);

}

 

static void timer_start(NRF_TIMER_Type * TIMER)

{

    timer_stop(TIMER);

 

    nrf_timer_task_trigger(TIMER, NRF_TIMER_TASK_CLEAR);

    nrf_timer_int_enable(TIMER, NRF_TIMER_INT_COMPARE0_MASK);

 

    nrf_timer_task_trigger(TIMER, NRF_TIMER_TASK_START);

}

 

static void timer_event_clear(NRF_TIMER_Type *TIMER)

{

    nrf_timer_event_clear(TIMER, NRF_TIMER_EVENT_COMPARE0);

}

 

void tfm_secure_controller_secure_timer_start(void)

{

    timer_init(NRF_TIMER1);

    timer_start(NRF_TIMER1);

}

 

void tfm_secure_controller_secure_timer_clear_intr(void)

{

    timer_event_clear(NRF_TIMER1);

}

 

 

psa_flih_result_t tfm_timer1_irq_flih(void)

{

  tfm_secure_controller_secure_timer_clear_intr();

  interrupt_ctr++;

  return PSA_FLIH_SIGNAL;

}

 

static psa_status_t tfm_sp_secure_function_ipc(psa_msg_t *msg)

{

    function_ctr++;

    LOG_INFFMT("Secure function called %u times, interrupt ctr: %u\n", function_ctr, interrupt_ctr );

    return PSA_SUCCESS;

}

 

void tfm_sp_secure_function(psa_msg_t *msg) {

  psa_status_t status;

  status = tfm_sp_secure_function_ipc(msg);

  psa_reply(msg->handle, status);

}

 

void tfm_sp_entry(void)

{

    LOG_INFFMT("Secure Partition: Init\n");

   

    tfm_secure_controller_secure_timer_start();

    psa_irq_enable(TFM_TIMER1_IRQ_SIGNAL);

    psa_signal_t signals = 0;

    uint32_t ctr = 0;

 

    while (1) {

        signals = psa_wait(TFM_TIMER1_IRQ_SIGNAL, PSA_BLOCK);

        if ( (signals & TFM_TIMER1_IRQ_SIGNAL) == TFM_TIMER1_IRQ_SIGNAL) {

          ctr++;

          LOG_INFFMT("FLIH stage 2 %u\n", interrupt_ctr);

          psa_reset_signal(TFM_TIMER1_IRQ_SIGNAL);

        }

     

        if ( ctr % 10 == 0 ) {

          signals = psa_wait(PSA_WAIT_ANY, PSA_POLL);

          if ( (signals & TFM_SECURE_FUNCTION_SIGNAL) == TFM_SECURE_FUNCTION_SIGNAL) {

            psa_msg_t msg;

            psa_get(signals, &msg);

            tfm_sp_secure_function(&msg);

       

          }

        }

    }  

}