New subject: [Tf-m] How to run PSA API compliance tests with TF-M SST service

3 Feb 2019


      Adding TF-M mailing list and Prasanth
Hi Murat,
Looking at the list of subscribed members, you are not yet registered to TF-M mailing list.
Please try subscribing using the link in https://www.trustedfirmware.org/contact/. You will receive a mail which you need to acknowledge before you get subscribed. 
Check your spam as well for the mail asking for acknowledgement.
On the topic below, the extended tests are for testing against the optional/extended SST Protected Storage APIs. These APIs are optional and so not required to pass for PSA Functional API certification.
The SST patch set below doesn’t support those optional SST PS APIs.
From what I last heard, Marc and team has verified all the other mandatory SST PS Tests on MuscaA using the patches below.
TF-M team or Prasanth might be able to advise on what might be going wrong.
Regards,
Shebu
-----Original Message-----
From: murat@za-ya.co murat@za-ya.co 
Sent: Sunday, February 3, 2019 10:29 AM
To: Marc Moreno Berengue Marc.MorenoBerengue@arm.com
Cc: nd nd@arm.com; Shebu Varghese Kuriakose Shebu.VargheseKuriakose@arm.com
Subject: RE: [Tf-m] [TF-M] How to run PSA API compliance tests with TF-M SST service
Dear Marc,
Thank you for introductions.
We are now running PSA SST tests on Musca board, and as we see there are only three test suites (p011, p012, p013), the other test suites are empty. 
And when we run the PSA SST tests, SST tests skip even existing three test cases.
This is because the first test case asks for supported API and expects " PSA_PS_SUPPORT_SET_EXTENDED "
static const test_data p011_data[] = {
    {
        VAL_PS_GET_SUPPORT, PSA_PS_SUPPORT_SET_EXTENDED /* Check if optional PS API supported */
    },
And TF-m (https://review.trustedfirmware.org/#/c/trusted-firmware-m/+/589/ ) does not the support extended PSA API yet as we see from
enum tfm_sst_err_t tfm_sst_get_support(uint32_t *support_flags)
    {
        /*
         * This function returns a bitmask with flags set for all of the optional
         * features supported by the SST service implementation.
         *
         * SST service does not support the optional extended PSA PS API yet.                         <<<<<<<<<<<<<<<<<<<<< NO SUPPORT
         */
*support_flags = 0;									<<<<<<<<<<<<<<<<<<<<< NO SUPPORT
return TFM_SST_ERR_SUCCESS;
    }
And PSA Test Suite skips the all remaining test case if there is no EXTENDED API Support.
0.000 - 00>Test Case not needed as Optional PS APIs are not supported.
    0.000 - 00>TEST RESULT: SKIPPED (Skip Code=0x2b)
    0.000 - 00>******************************************
    0.000 - 00>************  Protected Storage Suite Report **********
    0.000 - 00>TOTAL TESTS     : 3
    0.000 - 00>TOTAL PASSED    : 0
    0.000 - 00>TOTAL SIM ERROR : 0
    0.000 - 00>TOTAL FAILED    : 0
    0.000 - 00>TOTAL SKIPPED   : 3
    0.000 - 00>******************************************
    0.000 - 00>Entering standby..
Is it the expected behaviour for Musca Port or are we missing something?
(We are registered but we don’t receive any e-mail from the TF-m mail list, I don’t know how to see your response 😊 )
Thank you.
Best Regards.
Murat
-----Original Message-----
From: TF-M tf-m-bounces@lists.trustedfirmware.org On Behalf Of Marc Moreno Berengue via TF-M
Sent: 01 February 2019 18:25
To: tf-m@lists.trustedfirmware.org
Cc: nd nd@arm.com
Subject: [Tf-m] [TF-M] How to run PSA API compliance tests with TF-M SST service
Hi all,
The Secure Storage (SST) service has been updated to align it with the PSA Protected Storage APIs version 1.0.
There are a set of patches which implements this change.
Please, find more information about those changes in the following ticket:
https://developer.trustedfirmware.org/T218
It's possible to run the PSA API Compliance tests (https://github.com/ARM-software/psa-arch-tests/tree/master/api-tests/dev_api...)
with the TF-M SST service by following the instructions below.
(The instructions assume that psa-arch-tests and trusted-firmware-m directories are at the same level in the filesystem):
1. Checkout https://github.com/ARM-software/psa-arch-tests/
  2. Checkout https://review.trustedfirmware.org/#/c/trusted-firmware-m/+/589/
  3. Copy the `interface/include/psa_protected_storage.h` to `interface/include/psa/protected_storage.h` as
     this is the header name expected by the PSA API Compliance tests
  4. Build the SST tests for the PSA API compliance as described in the `psa-arch-tests/api-tests`
     for one of the supported targets,  providing the include path of the TF-M interface
     (i.e. trusted-firmware-m/interface/include).
     Set crypto and internal trusted storage as not implemented in
     `psa-arch-tests/api-tests/platform/targets/<TARGET_NAME>/Makefile`.
     Set PSA_CRYPTO_IMPLEMENTED:=0 and PSA_INTERNAL_TRUSTED_STORAGE_IMPLEMENTED:=0
     *   The list of tests being run is specified in the file `psa-arch-tests/api-tests/dev_apis/protected_storage/testsuite.db`
     *   For example, this command will build the tests for AN521
     "./tools/scripts/setup.sh --target tgt_dev_apis_tfm_an521 --toolchain ARMCLANG --cpu_arch armv8m_ml --verbose 2 --suite protected_storage --include ../../trusted-firmware-m/interface/include --archive_tests"
  5. Build TF-M by using the ConfigPsaApiTest.cmake, enabling the PSA API Compliance tests for SST service
     by adding the following cmake switch during the configuration step: "-DPSA_API_TEST_SECURE_STORAGE=ON"
     *   For example, this command will build the TF-M + NS app with PSA API Protected Storage tests for AN521
     "cmake -G"Unix Makefiles" -DPROJ_CONFIG=`readlink -f ../ConfigPsaApiTest.cmake` -DTARGET_PLATFORM=AN521 -DCOMPILER=ARMCLANG -DPSA_API_TEST_SECURE_STORAGE=ON ../"
This will make our NS test app run a subset of the PSA API compliance tests for SST service.
Thanks,
Marc Moreno
--
TF-M mailing list
TF-M@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-m

Re: [Tf-m] [TF-M] How to run PSA API compliance tests with TF-M SST service