Hi Sudeep, On Mon, Dec 9, 2019 at 9:07 PM Sudeep Holla sudeep.holla@arm.com wrote:
On Mon, Dec 09, 2019 at 08:36:18PM +0530, Sandeep Tripathy wrote:
Hi Sudeep, I am very specific about the core caches. The app/driver or many entity for that matter are updating cacheable coherent memory range(s).
Are we referring OS apps/drivers or something on the other masters ? Please be as specific as possible.
Linux application and drivers running on arm cluster and sharing DDR over CCN with other masters. Not sure if the specifics will make it any clearer. for ex: Broadcom smartNIC or any accelerator on PCIe. I think the details will only distract the discussion to platform specifics where as the issue is not.
Assume one application is logging some data to the said coherent cacheable ddr. on reboot /shutdown notification it will stop logging. If cpu cache is not flushed log is lost. The cache can be small but the buffer can be huge for a range flush.
On shutdown/reboot notification what they ought to do ?
If on the same slave OS, then stop all on-going transactions. There are hooks to do that. Simply, it can map to be remove device calls. And the other masters also have notification to do the same.
Done. stopped all transactions. but data lies in local caches.
If we ask them to do respective range flushes (cache mnt of a coherent memory) that is less generic compared to if the core infrastructure gives the coherency guarantee
Sure, but I need clarity on above to answer this.
(1-OS: especially more than just halting for secondary(other) cores or may be cpuhp secondary(other) cores etc, 2- tf-a psci shutdown/reset/reset2 to do graceful pwrdown to take care of the initiating core). May be this can be under a flag to choose between faster reboot vs graceful power down sequence if at all it qualifies as generic ?
I think we had enough discussions so far to tell this is not a generic requirement.
Thanks for the quick responses. looks like finally we have to agree to disagree.
[...]
And what's done in those masters with *this particular* notification ?
Anything but not cache maintenance preferably.
And what does that *anything* include in your case. Please be specific with example. I am still failing to understand the role of this fancy master who wants its slave to take care of cache maintenance for it.
Anything can be nothing also or as you mentioned above it can just stop further transactions. but it is agnostic of the underlying caches.
Why can't it stop snooping into caches(or request firmware to) that belong to/maintained by the other slave(OS) ?
Sure. the respective clusters to be taken out of snoop domain by firmware as part of psci plat specific hooks.
OK
But what about their caches. I don't think there is a pull mechanism that ccn/or other interconnect can voluntarily pull the dirty lines from exiting core caches as an alternate to have them flushed by the respective cores.
OK. SO why are dirty lines in local caches, why can't the driver/device take action on shutdown ?
Since driver/app is working on coherent memory by convention it can assume no explicit maintenance.
I am not saying other approaches are not tried/discussed. But I was not aware of it. Also I am still not aware of the full design of your system yet.
I think now we have narrowed down the discussion to very specific cache maintenance ownership issue.
Not sure yet, let's see ;)
CENH ?
sorry. pls ignore cute embedded .. hacks !
Hmmm...
-- Regards, Sudeep
Thanks Sandeep