Hi Dan and Varun,

Our motivation for adding OS-initiated mode is because it's required by the Linux PSCI CPUIdle driver to enable support for suspend-to-idle [1].

I tested the topic [2] on the FVP_Base_RevC-2xAEMvA platform with and without setting ARM_RECOM_STATE_ID_ENC, and am happy to report all the existing and new tests passed in both cases!

I uploaded a design proposal in a new CL [3]. The design allows platforms to support any number of power states. Regarding impact on the min_latency values, our team hasn't yet begun integrating our platform with Linux or any HLOS, and don't have any data to provide at this moment.

Thanks! Wing

[1] https://elixir.bootlin.com/linux/latest/source/drivers/cpuidle/cpuidle-psci.... [2] https://review.trustedfirmware.org/q/topic:psci-osi [3] https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/17896

On Tue, Nov 15, 2022 at 8:20 AM Varun Wadekar vwadekar@nvidia.com wrote:

Hi,

Usually, platforms have multiple power states behind the CPU_SUSPEND PSCI call. Did you consider these for your design? If yes, it would be interesting to see the prediction accuracy of the OSI model versus the PI model. Also do you see any impact on the min_latency values with the OSI model?

By, the way do you have a top level design for the proposal?

-Varun

*From:* Dan Handley via TF-A tf-a@lists.trustedfirmware.org *Sent:* Monday, 14 November 2022 5:50 PM *To:* Wing Li wingers@google.com; Okash Khawaja okash@google.com *Cc:* tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.org *Subject:* [TF-A] Re: PSCI OS-initiated mode

*External email: Use caution opening links or attachments*

Hi Wing/Okash

I know what OSI mode is supposed to deliver in theory – I just haven’t seen any compelling evidence of its benefit in practice. If you can share any evidence (even anecdotal) that would be great, although it’s not essential for these patches to proceed.

However, we are going to need some upstream platform(s) to exercise the new functionality. This can be a model platform, e.g. the Base FVP, not necessarily real hw. Are you able to at least get this running locally on Base FVP? I’m sure others can help with adding a config to the OpenCI.

Regards

Dan.

*From:* Wing Li wingers@google.com *Sent:* 11 November 2022 18:34 *To:* Okash Khawaja okash@google.com *Cc:* Dan Handley Dan.Handley@arm.com; tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.org *Subject:* Re: [TF-A] Re: PSCI OS-initiated mode

Hi Dan,

Thanks for your feedback!

I've tested these patches with our platform, but have not tested with any upstream platforms. I added tests for the OS-initiated mode code path in TF-A-Tests [1] that we can run against upstream platforms to verify. I'm not sure how I can get a hold of an upstream platform board; it'd be much appreciated if perhaps maintainers of upstream platforms could help run the tests?

In theory, any platform that supports CPU_SUSPEND would be able to support both platform-coordinated mode and OS-initiated mode, since the power state coordination happens either in the PSCI library implementation or in an HLOS.

Wing

[1] https://review.trustedfirmware.org/c/TF-A/tf-a-tests/+/17684 https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.trustedfirmware.org%2Fc%2FTF-A%2Ftf-a-tests%2F%2B%2F17684&data=05%7C01%7Cvwadekar%40nvidia.com%7Ca3c4a05b8e3b48ae510708dac668c75e%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638040450576300615%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8pNtAwoHRWF0NAy8k8nrRLEroUX%2Byds6U2WHGZVRjdc%3D&reserved=0

On Fri, Nov 11, 2022 at 9:43 AM Okash Khawaja okash@google.com wrote:

Hi Dan,

The idea with OSI mode is that HLOS will have more awareness about power sequences and hence will be able to optimise power usage better (e.g. using heuristics) if it had more control. This means having HLOS which contains right modules for a specific platform and specific use case, if we want to do a power/perf comparison between OSI and plat-coordinated modes.

On Fri, Nov 11, 2022 at 5:08 PM Dan Handley via TF-A tf-a@lists.trustedfirmware.org wrote:

...

Hi Wing

Thanks for your contribution. Are there any upstream platforms that

these patches have been tested with? If so, it would be good to have visibility of the sw stack under test. Ideally, there would be at least 1 platform that supported both platform-coordinated mode and OS-initiated mode, so that a fair power/performance comparison can be done.

...

Regards

Dan.

From: Wing Li via TF-A tf-a@lists.trustedfirmware.org Sent: 10 November 2022 05:53 To: tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.org Subject: [TF-A] PSCI OS-initiated mode

Hi,

The PSCI specification defines two different power state coordination

modes for CPU_SUSPEND that can be used to put a core or a group of cores into a low-power state. These modes are the platform-coordinated mode (default) and the OS-initiated mode (optional). OS-initiated mode is currently not supported by TF-A, while both modes are supported by the Linux Kernel.

...

Requesting reviews for the patches [1] adding support for OS-initiated

mode in TF-A and the corresponding tests in TF-A-Tests. Any feedback and comments are much appreciated.

...

Thanks in advance!

Wing

[1] https://review.trustedfirmware.org/q/topic:psci-osi

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.trustedfirmware.org%2Fq%2Ftopic%3Apsci-osi&data=05%7C01%7Cvwadekar%40nvidia.com%7Ca3c4a05b8e3b48ae510708dac668c75e%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638040450576300615%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ASA1YsAMpJZNal3bjJKX%2F7SwEQzrwCm2tyqgbzcEfDU%3D&reserved=0

...

-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org

suspend-to-idle normally means linux ‘freeze’ mode. For this mode, nothing related to PSCI OSI support.

BR Jacky Bai

From: Sudeep Holla via TF-A tf-a@lists.trustedfirmware.org Sent: 2022年11月21日 18:38 To: Wing Li wingers@google.com Cc: Dan Handley Dan.Handley@arm.com; Okash Khawaja okash@google.com; tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.org; Sudeep Holla sudeep.holla@arm.com Subject: [TF-A] Re: PSCI OS-initiated mode

On Mon, Nov 21, 2022 at 2:51 AM Wing Li via TF-A <tf-a@lists.trustedfirmware.org mailto:tf-a@lists.trustedfirmware.org > wrote:

Hi Dan and Varun,

Our motivation for adding OS-initiated mode is because it's required by the Linux PSCI CPUIdle driver to enable support for suspend-to-idle [1].

That is not correct. If anything is broken or not working, it needs to be fixed. Adding platform support for

OSI just to support suspend-to-idle is a completely wrong approach. Can you let me know what is not

working or how is it broken ?

Hi,

Usually, platforms have multiple power states behind the CPU_SUSPEND PSCI call. Did you consider these for your design? If yes, it would be interesting to see the prediction accuracy of the OSI model versus the PI model. Also do you see any impact on the min_latency values with the OSI model?

By, the way do you have a top level design for the proposal?

-Varun

From: Dan Handley via TF-A tf-a@lists.trustedfirmware.org Sent: Monday, 14 November 2022 5:50 PM To: Wing Li wingers@google.com; Okash Khawaja okash@google.com Cc: tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.org Subject: [TF-A] Re: PSCI OS-initiated mode

External email: Use caution opening links or attachments

Hi Wing/Okash

I know what OSI mode is supposed to deliver in theory - I just haven't seen any compelling evidence of its benefit in practice. If you can share any evidence (even anecdotal) that would be great, although it's not essential for these patches to proceed.

However, we are going to need some upstream platform(s) to exercise the new functionality. This can be a model platform, e.g. the Base FVP, not necessarily real hw. Are you able to at least get this running locally on Base FVP? I'm sure others can help with adding a config to the OpenCI.

Regards

Dan.

From: Wing Li <wingers@google.commailto:wingers@google.com> Sent: 11 November 2022 18:34 To: Okash Khawaja <okash@google.commailto:okash@google.com> Cc: Dan Handley <Dan.Handley@arm.commailto:Dan.Handley@arm.com>; tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.orgmailto:tf-a-tests@lists.trustedfirmware.org Subject: Re: [TF-A] Re: PSCI OS-initiated mode

Hi Dan,

Thanks for your feedback!

I've tested these patches with our platform, but have not tested with any upstream platforms. I added tests for the OS-initiated mode code path in TF-A-Tests [1] that we can run against upstream platforms to verify. I'm not sure how I can get a hold of an upstream platform board; it'd be much appreciated if perhaps maintainers of upstream platforms could help run the tests?

In theory, any platform that supports CPU_SUSPEND would be able to support both platform-coordinated mode and OS-initiated mode, since the power state coordination happens either in the PSCI library implementation or in an HLOS.

Wing

[1] https://review.trustedfirmware.org/c/TF-A/tf-a-tests/+/17684https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.trustedfirmware.org%2Fc%2FTF-A%2Ftf-a-tests%2F%2B%2F17684&data=05%7C01%7Cvwadekar%40nvidia.com%7Ca3c4a05b8e3b48ae510708dac668c75e%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638040450576300615%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8pNtAwoHRWF0NAy8k8nrRLEroUX%2Byds6U2WHGZVRjdc%3D&reserved=0

On Fri, Nov 11, 2022 at 9:43 AM Okash Khawaja <okash@google.commailto:okash@google.com> wrote: Hi Dan,

The idea with OSI mode is that HLOS will have more awareness about power sequences and hence will be able to optimise power usage better (e.g. using heuristics) if it had more control. This means having HLOS which contains right modules for a specific platform and specific use case, if we want to do a power/perf comparison between OSI and plat-coordinated modes.

On Fri, Nov 11, 2022 at 5:08 PM Dan Handley via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> wrote:

Hi Wing

Thanks for your contribution. Are there any upstream platforms that these patches have been tested with? If so, it would be good to have visibility of the sw stack under test. Ideally, there would be at least 1 platform that supported both platform-coordinated mode and OS-initiated mode, so that a fair power/performance comparison can be done.

Regards

Dan.

From: Wing Li via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> Sent: 10 November 2022 05:53 To: tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.orgmailto:tf-a-tests@lists.trustedfirmware.org Subject: [TF-A] PSCI OS-initiated mode

Hi,

The PSCI specification defines two different power state coordination modes for CPU_SUSPEND that can be used to put a core or a group of cores into a low-power state. These modes are the platform-coordinated mode (default) and the OS-initiated mode (optional). OS-initiated mode is currently not supported by TF-A, while both modes are supported by the Linux Kernel.

Requesting reviews for the patches [1] adding support for OS-initiated mode in TF-A and the corresponding tests in TF-A-Tests. Any feedback and comments are much appreciated.

Thanks in advance!

Wing

[1] https://review.trustedfirmware.org/q/topic:psci-osihttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.trustedfirmware.org%2Fq%2Ftopic%3Apsci-osi&data=05%7C01%7Cvwadekar%40nvidia.com%7Ca3c4a05b8e3b48ae510708dac668c75e%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638040450576300615%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ASA1YsAMpJZNal3bjJKX%2F7SwEQzrwCm2tyqgbzcEfDU%3D&reserved=0

-- TF-A mailing list -- tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.orgmailto:tf-a-leave@lists.trustedfirmware.org

Hi Wing/Okash

I know what OSI mode is supposed to deliver in theory – I just haven’t seen any compelling evidence of its benefit in practice. If you can share any evidence (even anecdotal) that would be great, although it’s not essential for these patches to proceed.

However, we are going to need some upstream platform(s) to exercise the new functionality. This can be a model platform, e.g. the Base FVP, not necessarily real hw. Are you able to at least get this running locally on Base FVP? I’m sure others can help with adding a config to the OpenCI.

Regards

Dan.

From: Wing Li wingers@google.com Sent: 11 November 2022 18:34 To: Okash Khawaja okash@google.com Cc: Dan Handley Dan.Handley@arm.com; tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.org Subject: Re: [TF-A] Re: PSCI OS-initiated mode

Hi Dan,

Thanks for your feedback!

I've tested these patches with our platform, but have not tested with any upstream platforms. I added tests for the OS-initiated mode code path in TF-A-Tests [1] that we can run against upstream platforms to verify. I'm not sure how I can get a hold of an upstream platform board; it'd be much appreciated if perhaps maintainers of upstream platforms could help run the tests?

In theory, any platform that supports CPU_SUSPEND would be able to support both platform-coordinated mode and OS-initiated mode, since the power state coordination happens either in the PSCI library implementation or in an HLOS.

Wing

[1] https://review.trustedfirmware.org/c/TF-A/tf-a-tests/+/17684

On Fri, Nov 11, 2022 at 9:43 AM Okash Khawaja <okash@google.commailto:okash@google.com> wrote: Hi Dan,

The idea with OSI mode is that HLOS will have more awareness about power sequences and hence will be able to optimise power usage better (e.g. using heuristics) if it had more control. This means having HLOS which contains right modules for a specific platform and specific use case, if we want to do a power/perf comparison between OSI and plat-coordinated modes.

On Fri, Nov 11, 2022 at 5:08 PM Dan Handley via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> wrote:

Hi Wing

Thanks for your contribution. Are there any upstream platforms that these patches have been tested with? If so, it would be good to have visibility of the sw stack under test. Ideally, there would be at least 1 platform that supported both platform-coordinated mode and OS-initiated mode, so that a fair power/performance comparison can be done.

Regards

Dan.

From: Wing Li via TF-A <tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org> Sent: 10 November 2022 05:53 To: tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org; tf-a-tests@lists.trustedfirmware.orgmailto:tf-a-tests@lists.trustedfirmware.org Subject: [TF-A] PSCI OS-initiated mode

Hi,

The PSCI specification defines two different power state coordination modes for CPU_SUSPEND that can be used to put a core or a group of cores into a low-power state. These modes are the platform-coordinated mode (default) and the OS-initiated mode (optional). OS-initiated mode is currently not supported by TF-A, while both modes are supported by the Linux Kernel.

Requesting reviews for the patches [1] adding support for OS-initiated mode in TF-A and the corresponding tests in TF-A-Tests. Any feedback and comments are much appreciated.

Thanks in advance!

Wing

[1] https://review.trustedfirmware.org/q/topic:psci-osi

-- TF-A mailing list -- tf-a@lists.trustedfirmware.orgmailto:tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.orgmailto:tf-a-leave@lists.trustedfirmware.org