Hi Amit, 1. The NS bit based protection is enforced by the TZASC. EL3 and Secure world need to access both secure and Non Secure memory and the Secure/NS mapping in MMU allows to make this access with the right NS bit setting. If the NS bit matches the TZASC configuration, then the access will not fault. 2. By default, some peripherals and Trusted SRAM are Secure world access only and the protection is enforced by using peripheral side filter.

Best Regards Soby Mathew

-----Original Message----- From: Nagal, Amit amit.nagal@amd.com Sent: Thursday, April 11, 2024 9:15 PM To: Soby Mathew Soby.Mathew@arm.com; tf-a@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: RE: el3 secure memory access from lower exception levels.

Hi Soby,

Thanks for reply. I am using armv8.2 based platform .

what is the significance of NS bit then ? actually , I was thinking this will be ensured by NS-bit .

Referring https://developer.arm.com/documentation/den0024/a/Security/TrustZone- hardware-architecture , it says: "The memory system is divided by means of an additional bit that accompanies the address of peripherals and memory. This bit, called the NS-bit, indicates whether the access is Secure or Non-secure. Software running in the Normal World can only make Non-secure accesses to memory, because the core always sets the NS bit to 1 in any memory transaction generated by the Normal World."

It further says : Trying to perform a Non-secure access to external memory marked as Secure causes the memory system to disallow the request and the slave device returns an error response. so does this mean , to return error , the memory system mandatorily needs TZASC ?

Can you please share an example for " peripheral side access filters must be present to protect the memory"

Regards Amit.

-----Original Message----- From: Soby Mathew Soby.Mathew@arm.com Sent: Thursday, April 11, 2024 9:50 PM To: Nagal, Amit amit.nagal@amd.com; tf-a@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: RE: el3 secure memory access from lower exception levels.

Hi Amit, Stage 1 MMU only affects the software executing the same exception level. Hence Stage 1 MMU at EL3 cannot protect against access by EL2. To protect Secure memory from NS world, either there must be additional IP like Trustzone controller which must be configured appropriately, or peripheral side access filters must be present to protect the memory. If FEAT_RME (from Arm v9.2 onwards) is present, then each world isolation can be achieved using GPF mechanism.

Best Regards Soby Mathew

...

-----Original Message----- From: Nagal, Amit via TF-A tf-a@lists.trustedfirmware.org Sent: Thursday, April 11, 2024 5:09 PM To: tf-a@lists.trustedfirmware.org Subject: [TF-A] el3 secure memory access from lower exception levels.

Hi ,

As I understand , there are separate page tables in MMU for EL2 and EL3 . If a memory region is mapped as secure memory (MT_SECURE ,

MT_MEMORY,

...

MT_RW) in bl31 code which runs in EL3 , and the same memory region is mapped again with same attributes (MT_SECURE , MT_MEMORY, MT_RW )

in

...

EL2 , will EL2 be able to read and write to EL3 secure memory region ?

Regards Amit -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

Le 12 avr. 2024 à 16:03, ff via TF-A tf-a@lists.trustedfirmware.org a écrit :

Hi Nagal,

You request allows me to share my understanding and would be happy to get feedback from experts…

Arm v8 architecture (I exclude v9) has:

• 2 secure states which can be used to isolate resources such as memory and devices
• 4 exception levels which is mainly used to control access to system registers (VBAR_EL3 is accessible only in EL3 while VBAR_EL1 is accessible in EL3, EL2 and EL1)

So there are in theory 8 execution contexts: S-EL3, S-EL2, S-EL1, S-EL0 NS-EL3, NS-EL2, NS-EL1, NS-EL0 Practically, it is said that EL3 is always in secure mode, so there are only 7 execution modes.

The SCR_EL3.NS (bit 0) is 0 for Secure State and 1 for Non Secure State. Changing EL is not straightforward but happens at some IRQ, instructions (SMC, HVC, ERET).

The architecture states that there are TWO distinct address spaces for Secure and Non Secure. It is thus architecturally possible to have two DRAM different DIMMs mapped at address Secure-0000000000000000 and NonSecure-0000000000000000.

meaning Secure-0000000000000000 and NonSecure-0000000000000000 point to TWO distinct DRAM bytes

Concretely, no implementation does that and the two address spaces are aliased to a single 52 bits space.

When an address is emitted by a processor core, its is routed to DRAM controller, SRAM, devices depending on that address and an externally visible bit (=SCR_EL3.NS when MMU is OFF, =page descriptor.NS when MMU is ON and SCR_EL3.NS==1). Note that some processors can partition SRAM intoL3 and Secure

When MU is ON and SCR_EL3.NS==0, sorry

memory.

The bus controller is programmed with an address map at very early stages (before the DRAM controller is up and running). Let’s assume the bus controller defines the following routing:

• 00000000-7fffffff is DRAM
• 80000000-8fffffff is Secure SRAM
• 90000000-9fffffff is Secure devices
• c0000000-ffffffff is NS devices

In the following, we can simply assume identity mapping if MMU ON

1. Secure mode

1.a) EL3, MMU OFF: request of anythings outside 80000000-9fffffff will result in a fault 1.b) EL3, MMU ON, the entire space can be used if the page descriptors that point to NS resources have NS bit = 1 1.c) S-EL2, MMU OFF, request of anythings outside 80000000-9fffffff will result in a fault 1.d) S-EL2, MMU ON, the entire space can be used if the page descriptors that point to NS resources have NS bit = 1; can install stage 2 mappings to restrict S-ELx access 1.e) S-EL1, MMU OFF, request of anythings outside 80000000-9fffffff will result in a fault; request to Secure space may be further restricted 1.f) S-EL1, MMU ON, request to anything but what is allowed by S-EL2 may result in a fault; if not restricted by S-EL2, S-EL1 can map any NS memory, including the one « owned » by NS-EL2/NS-EL1 (practically OP-TEE can mess-up with Xen or Linux kernel memory)

2. NS mode

2.a) NS-EL2, NS-EL1, NS-EL0, MMU OFF request anything inside 80000000-9fffffff will result in a fault 2.b) NS-EL2, NS-EL1, NS-EL0, MMU ON, request anything inside 80000000-9fffffff will result in a fault (regardless of NS bit in page descriptors which is ignored when running in NS state; you could have thought that bye setting the page descriptor bit NS to 0 you could target Secure memory but no...) 2.c) NS-EL2 can restrict what is visible to upper NS-ELs with stage 2 mappings, NS-EL1 can restrict what NS-EL0 can view (typical OS stuff) 2.d) S-EL3 cannot make some of Secure memory visible to NS-ELx, but the contrary is true: NS-EL1 can « register NS memory » next to S-EL3 code so that it can be used as shared memory

(Some hardware vendors call that bus controller feature a hardware firewall. Raspberry PIs do NOT have such hardware so all the rules we talked about here are moot: there are no secure memory or secure devices. You can run EL3, S-EL2, S-EL1 software but they are not protected from NS-EL1)

There are other side effects related to address space aliasing of both Secure and NS memory. In theory there should be non Secure Memory declaration in the device tree passed to Operating Systems like Linux. For various reasons, it is…. We learnt the hard way that it should be marked « no-map » because in certain conditions, the Secure Memory can be targeted by pre-fetchers and result in faults while there is no explicit access to Secure memory.

Cordially,

...

Le 11 avr. 2024 à 22:14, Nagal, Amit via TF-A tf-a@lists.trustedfirmware.org a écrit :

Hi Soby,

Thanks for reply. I am using armv8.2 based platform .

what is the significance of NS bit then ? actually , I was thinking this will be ensured by NS-bit .

Referring https://developer.arm.com/documentation/den0024/a/Security/TrustZone-hardwar... , it says: "The memory system is divided by means of an additional bit that accompanies the address of peripherals and memory. This bit, called the NS-bit, indicates whether the access is Secure or Non-secure. Software running in the Normal World can only make Non-secure accesses to memory, because the core always sets the NS bit to 1 in any memory transaction generated by the Normal World."

It further says : Trying to perform a Non-secure access to external memory marked as Secure causes the memory system to disallow the request and the slave device returns an error response. so does this mean , to return error , the memory system mandatorily needs TZASC ?

Can you please share an example for " peripheral side access filters must be present to protect the memory"

Regards Amit.

-----Original Message----- From: Soby Mathew Soby.Mathew@arm.com Sent: Thursday, April 11, 2024 9:50 PM To: Nagal, Amit amit.nagal@amd.com; tf-a@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: RE: el3 secure memory access from lower exception levels.

Hi Amit, Stage 1 MMU only affects the software executing the same exception level. Hence Stage 1 MMU at EL3 cannot protect against access by EL2. To protect Secure memory from NS world, either there must be additional IP like Trustzone controller which must be configured appropriately, or peripheral side access filters must be present to protect the memory. If FEAT_RME (from Arm v9.2 onwards) is present, then each world isolation can be achieved using GPF mechanism.

Best Regards Soby Mathew

...

-----Original Message----- From: Nagal, Amit via TF-A tf-a@lists.trustedfirmware.org Sent: Thursday, April 11, 2024 5:09 PM To: tf-a@lists.trustedfirmware.org Subject: [TF-A] el3 secure memory access from lower exception levels.

Hi ,

As I understand , there are separate page tables in MMU for EL2 and EL3 . If a memory region is mapped as secure memory (MT_SECURE , MT_MEMORY, MT_RW) in bl31 code which runs in EL3 , and the same memory region is mapped again with same attributes (MT_SECURE , MT_MEMORY, MT_RW ) in EL2 , will EL2 be able to read and write to EL3 secure memory region ?

Regards Amit -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

Hi FF,

Thanks for the reply and detailed explanation . it’s helpful.

The bus controller is programmed with an address map at very early stages

Can you please provide an example of any hardware implemented with bus controller firewall ? Is it the same as TZASC ?

In theory there should be non Secure Memory declaration in the device tree passed to Operating Systems like Linux.

You mean "No secure memory declaration" right ?

Raspberry PIs do NOT have such hardware so all the rules , we talked about here are moot: there are no secure memory or secure devices.

So what I am inferring is , that to have secure memory protected from NS world , we should have support both in processor side via NS bit as well as protection is needed to be enforced via TZASC or bus controller .

Regards Amit

-----Original Message----- From: Soby Mathew Soby.Mathew@arm.com Sent: Friday, April 12, 2024 7:49 PM To: ff ff@shokubai.tech Cc: Nagal, Amit amit.nagal@amd.com; tf-a@lists.trustedfirmware.org; nd nd@arm.com Subject: RE: [TF-A] Re: el3 secure memory access from lower exception levels.

Hi ff, You are correct in your assessment AFAICS.

...

We learnt the hard way that it should be marked « no-map » because in

certain conditions, the Secure Memory can be targeted by pre-fetchers and result in faults while there is no explicit access to Secure memory.

Yes, that's right. NS world should not map any Secure memory into MMU as there can be speculative accesses , and depending on the type of Address space protection, there can be errors even if there is no explicit access.

Best Regards Soby Mathew

...

-----Original Message----- From: ff via TF-A tf-a@lists.trustedfirmware.org Sent: Friday, April 12, 2024 3:08 PM To: ff ff@shokubai.tech Cc: Nagal, Amit amit.nagal@amd.com; Soby Mathew Soby.Mathew@arm.com; tf-a@lists.trustedfirmware.org; nd

nd@arm.com

...

Subject: [TF-A] Re: el3 secure memory access from lower exception levels.

...

Le 12 avr. 2024 à 16:03, ff via TF-A tf-a@lists.trustedfirmware.org a écrit :

Hi Nagal,

You request allows me to share my understanding and would be happy to get feedback from experts…

Arm v8 architecture (I exclude v9) has:

• 2 secure states which can be used to isolate resources such as

memory and devices

• 4 exception levels which is mainly used to control access to

system registers (VBAR_EL3 is accessible only in EL3 while VBAR_EL1 is accessible in EL3, EL2 and EL1)

So there are in theory 8 execution contexts: S-EL3, S-EL2, S-EL1, S-EL0 NS-EL3, NS-EL2, NS-EL1, NS-EL0 Practically, it is said that EL3 is always in secure mode, so there are only 7

execution modes.

...

The SCR_EL3.NS (bit 0) is 0 for Secure State and 1 for Non Secure State. Changing EL is not straightforward but happens at some IRQ, instructions

(SMC, HVC, ERET).

...

The architecture states that there are TWO distinct address spaces for Secure

and Non Secure.

...

It is thus architecturally possible to have two DRAM different DIMMs mapped

at address Secure-0000000000000000 and NonSecure-0000000000000000.

meaning Secure-0000000000000000 and NonSecure-0000000000000000

point

...

to TWO distinct DRAM bytes

...

Concretely, no implementation does that and the two address spaces are

aliased to a single 52 bits space.

...

When an address is emitted by a processor core, its is routed to DRAM controller, SRAM, devices depending on that address and an externally visible bit (=SCR_EL3.NS when MMU is OFF, =page descriptor.NS when MMU is ON and SCR_EL3.NS==1). Note that some processors can partition SRAM intoL3 and Secure

When MU is ON and SCR_EL3.NS==0, sorry

...

memory.

The bus controller is programmed with an address map at very early stages

(before the DRAM controller is up and running).

...

Let’s assume the bus controller defines the following routing:

• 00000000-7fffffff is DRAM
• 80000000-8fffffff is Secure SRAM
• 90000000-9fffffff is Secure devices
• c0000000-ffffffff is NS devices

In the following, we can simply assume identity mapping if MMU ON

1. Secure mode

1.a) EL3, MMU OFF: request of anythings outside 80000000-9fffffff will result in a fault 1.b) EL3, MMU ON, the entire space can be used if the page descriptors that point to NS resources have NS bit = 1 1.c) S-EL2, MMU OFF, request of anythings outside 80000000-9fffffff will result in a fault 1.d) S-EL2, MMU ON, the entire space can be used if the page descriptors that point to NS resources have NS bit = 1; can install stage 2 mappings to restrict S-ELx access 1.e) S-EL1, MMU OFF, request of anythings outside 80000000-9fffffff will result in a fault; request to Secure space may be further restricted 1.f) S-EL1, MMU ON, request to anything but what is allowed by S-EL2 may result in a fault; if not restricted by S-EL2, S-EL1 can map any NS memory, including the one « owned » by NS-EL2/NS-EL1 (practically OP-TEE can mess-up with Xen or Linux kernel memory)

2. NS mode

2.a) NS-EL2, NS-EL1, NS-EL0, MMU OFF request anything inside 80000000-9fffffff will result in a fault 2.b) NS-EL2, NS-EL1, NS-EL0, MMU ON, request anything inside 80000000-9fffffff will result in a fault (regardless of NS bit in page descriptors which is ignored when running in NS state; you could have thought that bye setting the page descriptor bit NS to 0 you could target Secure memory but no...) 2.c) NS-EL2 can restrict what is visible to upper NS-ELs with stage 2 mappings, NS-EL1 can restrict what NS-EL0 can view (typical OS stuff) 2.d) S-EL3 cannot make some of Secure memory visible to NS-ELx, but the contrary is true: NS-EL1 can « register NS memory » next to S-EL3 code so that it can be used as shared memory

(Some hardware vendors call that bus controller feature a hardware firewall. Raspberry PIs do NOT have such hardware so all the rules we talked about here are moot: there are no secure memory or secure devices. You can run EL3, S-EL2, S-EL1 software but they are not protected from NS-EL1)

There are other side effects related to address space aliasing of both Secure

and NS memory.

...

In theory there should be non Secure Memory declaration in the device tree

passed to Operating Systems like Linux.

...

For various reasons, it is…. We learnt the hard way that it should be marked « no-map » because in

certain conditions, the Secure Memory can be targeted by pre-fetchers and result in faults while there is no explicit access to Secure memory.

...

Cordially,

...

Le 11 avr. 2024 à 22:14, Nagal, Amit via TF-A <tf-

a@lists.trustedfirmware.org> a écrit :

...

...

Hi Soby,

Thanks for reply. I am using armv8.2 based platform .

what is the significance of NS bit then ? actually , I was thinking this will be

ensured by NS-bit .

...

...

Referring

https://developer.arm.com/documentation/den0024/a/Security/TrustZone- hardware-architecture , it says:

...

...

"The memory system is divided by means of an additional bit that

accompanies the address of peripherals and memory. This bit, called the NS-bit, indicates whether the access is Secure or Non-secure.

...

...

Software running in the Normal World can only make Non-secure accesses

to memory, because the core always sets the NS bit to 1 in any memory transaction generated by the Normal World."

...

...

It further says : Trying to perform a Non-secure access to external memory

marked as Secure causes the memory system to disallow the request and the slave device returns an error response.

...

...

so does this mean , to return error , the memory system mandatorily needs

TZASC ?

...

...

Can you please share an example for " peripheral side access filters must be

present to protect the memory"

...

...

Regards Amit.

-----Original Message----- From: Soby Mathew Soby.Mathew@arm.com Sent: Thursday, April 11, 2024 9:50 PM To: Nagal, Amit amit.nagal@amd.com; tf-a@lists.trustedfirmware.org Cc: nd nd@arm.com Subject: RE: el3 secure memory access from lower exception levels.

Hi Amit, Stage 1 MMU only affects the software executing the same exception

level.

...

Hence Stage 1 MMU at EL3 cannot protect against access by EL2. To protect Secure memory from NS world, either there must be additional IP like Trustzone controller which must be configured appropriately, or peripheral side access filters must be present to protect the memory. If FEAT_RME (from Arm v9.2 onwards) is present, then each world isolation can be achieved using GPF mechanism.

...

...

Best Regards Soby Mathew

...

-----Original Message----- From: Nagal, Amit via TF-A tf-a@lists.trustedfirmware.org Sent: Thursday, April 11, 2024 5:09 PM To: tf-a@lists.trustedfirmware.org Subject: [TF-A] el3 secure memory access from lower exception levels.

Hi ,

As I understand , there are separate page tables in MMU for EL2 and

EL3 .

...

...

...

...

If a memory region is mapped as secure memory (MT_SECURE , MT_MEMORY, MT_RW) in bl31 code which runs in EL3 , and the same memory region is mapped again with same attributes (MT_SECURE , MT_MEMORY,

MT_RW )

...

...

...

in EL2 , will EL2 be able to read and write to EL3 secure memory region ?

Regards Amit -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org