Hi,

See my comment below [OD].

Regards, Olivier.

From: ff ff@shokubai.tech Sent: 05 February 2024 11:01 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Subject: Re: MMIO with pre/post indexing instructions

Le 5 févr. 2024 à 10:38, Olivier Deprez Olivier.Deprez@arm.com a écrit :

Hi,

See few questions below [OD]

Regards, Olivier.

From: ff via TF-A tf-a@lists.trustedfirmware.org Sent: 04 February 2024 16:47 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Subject: [TF-A] MMIO with pre/post indexing instructions   Hi,

I am using hypervisors to allow execution of EL3 code directly

[OD] Can you elaborate a bit on the exact flow? IIUC 'hypervisor' implies running at EL2 so is this in context of calling an EL3 service from EL2 by SMC?

and had to deal with MMIO caused by instructions that have post-indexing parameters. Xen community had to deal with this in 2022: https://www.mail-archive.com/xen-devel@lists.xenproject.org/msg113335.html This does not cause any trouble and have implemented ways to deal with this expeditiously.

Yet, I wonder if it is desirable to from a synchronisation perspective?

[OD] If I understand properly, this is about Xen handling and recovering a data abort triggered by a VM. IIUC there is no such recovery logic at EL3. I have hard time understanding how the mentioned thread applies to TF-A's EL3 exception handling. Can you provide a bit more details?

Oh sorry for providing may be misleading information. The topic is really MMIO and what instructions the compiler generate when accessing those regions. For the same platform (Marvell macchiatobin) TFA compile options allows generation of post-indexing instructions to access MMIO regions while there none coming from Linux kernel (tested with 4.4 and 6.7). I am asking wether those instructions are OK to use for MMIO or not.

[OD] I believe I understand why this is done in context of Xen (EL2) handling and recovering a data abort originated from a VM (EL1). But I miss the use case as for why a similar 'workaround' would be required at EL3. AFAIK EL3 does not attempt recovering an data abort that would have occurred at a lower EL. Even if this was the case, the similar workaround would have to be done at EL2 and not EL3. Others on the mailing list may have differing opinions!

Cheers

FF

Compiling TFA 2.10 for PLAT=a80x0_mcbin

The following line

https://elixir.bootlin.com/arm-trusted-firmware/v2.10.0/source/drivers/marve...

Is compiled as:

win_reg = mmio_read_32(AMB_WIN_CR_OFFSET(win_id));                                         4026d9c:       f9470260        ldr     x0, [x19, #3584]                                                   4026da0:       9100e002        add     x2, x0, #0x38                                                      4026da4:       b9400001        ldr     w1, [x0]                                                                          win_reg &= ~WIN_ENABLE_BIT;                                                                4026da8:       121f7821        and     w1, w1, #0xfffffffe                                                       *(volatile uint32_t*)addr = value;                                                               // post-indexing   4026dac:       b8008401        str     w1, [x0], #8

#gcc --version gcc (Ubuntu 10.3.0-1ubuntu1) 10.3.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

#objdump --version GNU objdump (GNU Binutils for Ubuntu) 2.36.1 Copyright (C) 2021 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) any later version. This program has absolutely no warranty.

#I can’t get access to latests compiler and binutils versions to overcome # « ../.. array subscript 0 is outside array bounds of ‘volatile void[0] » # « ../.. has a LOAD segment with RWX permissions » # so I don’t know if this behaviour is changed when using this latests toolchain

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

Top posting for a related topic, assuming that indexing instructions can be used fine along cache and atomicity constraints:

The cost of handling the pre-post indexing case is to get the opcode (meaning translating guest virtual into guest physical then host memory ) and check it for all MMIO operations.

The cost of checking whether the hypervisor need to handle indexing instructions would be greatly reduced if: - the ESR_EL2 had an bit 14 equivalent (acquire/release instruction triggered the data abort) for pre-post indexing. - a (non existing) PAR_EL2 would give the PA of the instruction causing the data abort (actually in all cases that would be great to have this updated with the instruction that caused the exit to EL2): the cost of GVA to GPA can be expensive while the GPA to host pointer is somewhat easy and VMM specific

Bottom line: I wonder where to host this type of discussion with Arm virtualization experts ?

Le 6 févr. 2024 à 10:37, Olivier Deprez Olivier.Deprez@arm.com a écrit :

Hi,

See my comment below [OD].

Regards, Olivier.

From: ff ff@shokubai.tech Sent: 05 February 2024 11:01 To: Olivier Deprez Olivier.Deprez@arm.com Cc: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Subject: Re: MMIO with pre/post indexing instructions

...

Le 5 févr. 2024 à 10:38, Olivier Deprez Olivier.Deprez@arm.com a écrit :

Hi,

See few questions below [OD]

Regards, Olivier.

From: ff via TF-A tf-a@lists.trustedfirmware.org Sent: 04 February 2024 16:47 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Subject: [TF-A] MMIO with pre/post indexing instructions Hi,

I am using hypervisors to allow execution of EL3 code directly

[OD] Can you elaborate a bit on the exact flow? IIUC 'hypervisor' implies running at EL2 so is this in context of calling an EL3 service from EL2 by SMC?

and had to deal with MMIO caused by instructions that have post-indexing parameters. Xen community had to deal with this in 2022: https://www.mail-archive.com/xen-devel@lists.xenproject.org/msg113335.html This does not cause any trouble and have implemented ways to deal with this expeditiously.

Yet, I wonder if it is desirable to from a synchronisation perspective?

[OD] If I understand properly, this is about Xen handling and recovering a data abort triggered by a VM. IIUC there is no such recovery logic at EL3. I have hard time understanding how the mentioned thread applies to TF-A's EL3 exception handling. Can you provide a bit more details?

Oh sorry for providing may be misleading information. The topic is really MMIO and what instructions the compiler generate when accessing those regions. For the same platform (Marvell macchiatobin) TFA compile options allows generation of post-indexing instructions to access MMIO regions while there none coming from Linux kernel (tested with 4.4 and 6.7). I am asking wether those instructions are OK to use for MMIO or not.

[OD] I believe I understand why this is done in context of Xen (EL2) handling and recovering a data abort originated from a VM (EL1). But I miss the use case as for why a similar 'workaround' would be required at EL3. AFAIK EL3 does not attempt recovering an data abort that would have occurred at a lower EL. Even if this was the case, the similar workaround would have to be done at EL2 and not EL3. Others on the mailing list may have differing opinions!

...

Cheers

FF

Compiling TFA 2.10 for PLAT=a80x0_mcbin

The following line

https://elixir.bootlin.com/arm-trusted-firmware/v2.10.0/source/drivers/marve...

Is compiled as:

            win_reg = mmio_read_32(AMB_WIN_CR_OFFSET(win_id));                                      

4026d9c: f9470260 ldr x0, [x19, #3584] 4026da0: 9100e002 add x2, x0, #0x38 4026da4: b9400001 ldr w1, [x0] win_reg &= ~WIN_ENABLE_BIT; 4026da8: 121f7821 and w1, w1, #0xfffffffe *(volatile uint32_t*)addr = value; // post-indexing 4026dac: b8008401 str w1, [x0], #8

#gcc --version gcc (Ubuntu 10.3.0-1ubuntu1) 10.3.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

#objdump --version GNU objdump (GNU Binutils for Ubuntu) 2.36.1 Copyright (C) 2021 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) any later version. This program has absolutely no warranty.

#I can’t get access to latests compiler and binutils versions to overcome # « ../.. array subscript 0 is outside array bounds of ‘volatile void[0] » # « ../.. has a LOAD segment with RWX permissions » # so I don’t know if this behaviour is changed when using this latests toolchain

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org