-----Original Message----- From: Soby Mathew [mailto:Soby.Mathew@arm.com] Sent: Monday, June 08, 2020 3:36 PM To: Sandeep Tripathy; Olivier Deprez; tf-a@lists.trustedfirmware.org Cc: nd Subject: RE: [TF-A] GICV3: system interface EOI ordering RFC
Just some inputs from my side: I agree we need at least a dsb() after the write to MMIO region to silence the peripheral and before the EOI at GIC sysreg interface. Adding it to the GIC EOI
Thanks for the confirmation. Now it’s about choosing the right place.
API seems the logical thing to do but as Olivier mentions, there are interrupt handler which do not deal with MMIO (eg: the Systimer interrupt) so adding it to GICv3 API might be arduous for such handlers.
So there is a choice here to let the interrupt handlers to deal with ensuring completeness before EOI at sysreg interface or adding it to GICv3 EOI API and take the overhead for interrupt handlers which do not have to deal with MMIO.
Yes I feel either of these is a must to guarantee functionality architecturally though both approach end up with some unnecessary overhead.
If GICv3 api takes care then it is an overhead for some ISRs dealing with non-MMIO. At present I do not see an active use case in reference implementation where sys timer ISR is in a performance intensive path where one additional DSB will be perceivable. But there may be some I could be totally wrong in this assumption (pmu/debug or.. not sure). Whereas I can certainly imagine some MMIO ISRs in performance critical path where unnecessary DSB is not acceptable at all.
If the interrupt handler needs to ensure then it will generically add 'DSB' as I think the driver cannot and should not make assumptions about how EOI is done afterwards. That will be overhead for the ISRs dealing with MMIO peripherals and non GIC-v3. If we consider only GICv3+ then good. Otherwise I would prefer the 'plat_ic_end_of_interrupt' like Olivier mentioned with a #if GICv3 instead of each ISRs dealing with it.
The GICv3 legacy MMIO CPU interface is deprecated for TF-A and the sys interface is the only one GICv3 driver in TF-A supports.
Right we can ignore the GICv3 legacy mode but GICv2 will still remain ?
Best Regards Soby Mathew
Thanks Sandeep
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Sandeep Tripathy via TF-A Sent: 08 June 2020 10:34 To: Olivier Deprez Olivier.Deprez@arm.com; tf-
a@lists.trustedfirmware.org
Subject: Re: [TF-A] GICV3: system interface EOI ordering RFC
Hi Olivier,
-----Original Message----- From: Olivier Deprez [mailto:Olivier.Deprez@arm.com] Sent: Monday, June 08, 2020 1:14 PM To: tf-a@lists.trustedfirmware.org; Sandeep Tripathy Subject: Re: [TF-A] GICV3: system interface EOI ordering RFC
Hi Sandeep,
gicv3_end_of_interrupt_sel1 is a static access helper macro. Its naming precisely tells what it does at gicv3 module level. It is called from
the weak
plat_ic_end_of_interrupt function for BL32 image.
I tend to think it is the driver responsibility to ensure the module
interrupt
acknowledge register write is reaching HW in order (or "be visible to
all
observers").
The driver should be agnostic of what interrupt controller is used and its behavior. And since 'all' writes were to mmio ranges mapped Device(nGnRE) /strongly-ordered(nGnRnE) there was no such need. This is a special case for GICv3 system interface only. Adding at driver level a DSB is unnecessary for other interrupt controllers.
Also I suspect adding a dsb might not be required generically for all
kind of IP.
Here are you referring to the peripheral IP or interrupt controller IP ? The issue is reordering at arm core itself (STR to device address Vs msr(sysreg write)). So I think Its applicable for all IP.
Adding a barrier in generic code might incur an unnecessary bottleneck.
But if there is a need to *ensure* presence of at least one DSB between the write transfer from core to device clear and gicv3 eoi icc register write in a generic way then what other option we have.
Thus wouldn't it be better to add the barrier to the overridden platform function rather than in generic gicv3 code?
That can be done but I feel this is more to do with gicv3 system interface only. Inside plat_xxx one has to check #if GICV3 ...and system interface.
I have a put a comment in the review, we can continue the discussion
there.
Regards, Olivier.
Thanks Sandeep
From: TF-A tf-a-bounces@lists.trustedfirmware.org on behalf of Sandeep Tripathy via TF-A tf-a@lists.trustedfirmware.org Sent: 05 June 2020 19:43 To: tf-a@lists.trustedfirmware.org Subject: [TF-A] GICV3: system interface EOI ordering RFC
Hi, In a typical interrupt handling sequence we do 1-Read IAR 2-Do interrupt handling 3-Clear the interrupt at source , so that the device de-asserts IRQ
request to
GIC >> I am suggesting a need of DSB here in case of GICv3 +. 4-Write to GIC cpu interface to do EOI.
Till GICv2 and with GICv3 legacy interface ICC_EOI write is a write over
AMBA
interface. The Addresses are mapped with (nR) attribute. Hence the write transfers
from the
core will be generated at step 3 and 4 in order. Please ignore the additional
buffers/bridges
in path from core till peripheral which has to be dealt separately as per SOC.
Query: I understand GICv3 system interface accesses are not over this
protocol
and core will not follow the ordering rule ?
I have observed spurious interrupt issue/manifestation ( I don't have
the
transfers probed) in RTOS environment where I have a primitive GICv3 driver but I wonder why things does not fail in Linux or tf-a. If it is working because from step(3) to
step(4) we have
barriers by chace due to other device register writes then I would suggest to have one in
the EOI
clearing API itself.
RFC:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/4454
Thanks Sandeep
-- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a