Hi
Following a discussion with Civil Infrastructure Project TSC, there is a watchdog protection issue with EFI: the time between the call to ExitBootService and Linux kernel takes over watchdog service is not covered by any watchdog protection.
The EFI specification for BS.SetWatchdogTimer is very flexible as it states "perform a platform specific action that must eventually cause the platform to be reset.".
So we could naively implement a solution that would arm platform hardware watchdog in addition to EFI timer. Assuming watchdog period is long enough that it cover the time for Linux to take over the hardware watchdog, there is nothing to be done in EFI Stub to benefit from the new protection.
But this scheme fails to handle FF-A update capsules which can take a long time. So either the period is long enough to support that or we need a FF-A watchdog service. Based on Siemens feedback, time to update can last 20 minutes. StandAloneMM may also need such a protection so FF-A watchdog service seems desired.
I'd be happy to receive feedback on the problem itself (watchdog in EFI) and on the possible solution (FF-A based).
Cheers
FF
On 6/29/21 12:48 PM, François Ozog wrote:
Hi
Following a discussion with Civil Infrastructure Project TSC, there is a watchdog protection issue with EFI: the time between the call to ExitBootService and Linux kernel takes over watchdog service is not covered by any watchdog protection.
The UEFI specification requires a watchdog. So this must be an implementation specific problem. Which firmware-hardware combination are you referring to?
U-Boot provides hardware watchdogs.
Before ExitBootServices() we use a timer event to implement a software watchdog (defaulting to the five minute interval required by the UEFI spec).
When booting via UEFI we check timer events and reset the hardware watchdog whenever either of the following services is called before ExitBootServices():
* RestoreTpl() * WaitForEvent() * CheckEvent() * Stall() * EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL.ReadKeyStrokeEx() * EFI_SIMPLE_TEXT_INPUT_PROTOCOL.ReadKeyStroke() * EFI_SIMPLE_NETWORK_PROTOCOL.GetStatus() * EFI_SIMPLE_NETWORK_PROTOCOL.Transmit() * EFI_SIMPLE_NETWORK_PROTOCOL.Receive()
Between ExitBootServices() and Linux booting the hardware watchdog is not reset. So it should kick in if customized.
On some systems the hardware watchdog was disabled because the longest supported timeout was too short to boot Linux successfully.
Relevant customizing settings include:
CONFIG_WDT - Enable driver model for watchdog timer drivers CONFIG_HW_WATCHDOG
Best regards
Heinrich
The EFI specification for BS.SetWatchdogTimer is very flexible as it states "perform a platform specific action that must eventually cause the platform to be reset.".
So we could naively implement a solution that would arm platform hardware watchdog in addition to EFI timer. Assuming watchdog period is long enough that it cover the time for Linux to take over the hardware watchdog, there is nothing to be done in EFI Stub to benefit from the new protection.
But this scheme fails to handle FF-A update capsules which can take a long time. So either the period is long enough to support that or we need a FF-A watchdog service. Based on Siemens feedback, time to update can last 20 minutes. StandAloneMM may also need such a protection so FF-A watchdog service seems desired.
I'd be happy to receive feedback on the problem itself (watchdog in EFI) and on the possible solution (FF-A based).
Cheers
FF _______________________________________________ boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture
On Tue, Jun 29, 2021 at 02:23:15PM +0200, Heinrich Schuchardt wrote:
On 6/29/21 12:48 PM, François Ozog wrote:
Hi
Following a discussion with Civil Infrastructure Project TSC, there is a watchdog protection issue with EFI: the time between the call to ExitBootService and Linux kernel takes over watchdog service is not covered by any watchdog protection.
The UEFI specification requires a watchdog. So this must be an implementation specific problem. Which firmware-hardware combination are you referring to?
U-Boot provides hardware watchdogs.
Without side-tracking things too much, I do want to note the last time within U-Boot we talked about watchdogs and UEFI specification, I believe there was still some doubt how hardware watchdogs are handled vs the notion of a software watchdog, and it should probably be clarified which things we're talking about here.
On Tue, 29 Jun 2021 at 14:23, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 6/29/21 12:48 PM, François Ozog wrote:
Hi
Following a discussion with Civil Infrastructure Project TSC, there is a watchdog protection issue with EFI: the time between the call to ExitBootService and Linux kernel takes over watchdog service is not covered by any watchdog protection.
The UEFI specification requires a watchdog. So this must be an implementation specific problem. Which firmware-hardware combination are you referring to?
U-Boot provides hardware watchdogs.
Before ExitBootServices() we use a timer event to implement a software watchdog (defaulting to the five minute interval required by the UEFI spec).
When booting via UEFI we check timer events and reset the hardware watchdog whenever either of the following services is called before ExitBootServices():
- RestoreTpl()
- WaitForEvent()
- CheckEvent()
- Stall()
- EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL.ReadKeyStrokeEx()
- EFI_SIMPLE_TEXT_INPUT_PROTOCOL.ReadKeyStroke()
- EFI_SIMPLE_NETWORK_PROTOCOL.GetStatus()
- EFI_SIMPLE_NETWORK_PROTOCOL.Transmit()
- EFI_SIMPLE_NETWORK_PROTOCOL.Receive()
Between ExitBootServices() and Linux booting the hardware watchdog is not reset. So it should kick in if customized.
UEFI Spec says "Resets and sets a watchdog timer used during boot services time.". So during boot service time means if the system hangs after ExitBootServices and before Linux installs the hardware watchdog driver, there is a window of opportunity to get stuck. And that is verified on upstream EDK2 implementation. Any other behavior is not in the specification and is platform specific. I am not sure leveraging a soft-watchdog based on timer IRQ is safe during the mentioned window. So I would be inclined to use hardware watchdogs that can autonomously reset or even power cycle the board. But that needs specification and implementation enhancements .
On some systems the hardware watchdog was disabled because the longest supported timeout was too short to boot Linux successfully.
Relevant customizing settings include:
CONFIG_WDT - Enable driver model for watchdog timer drivers CONFIG_HW_WATCHDOG
Best regards
Heinrich
The EFI specification for BS.SetWatchdogTimer is very flexible as it states "perform a platform specific action that must eventually cause the platform to be reset.".
So we could naively implement a solution that would arm platform hardware watchdog in addition to EFI timer. Assuming watchdog period is long enough that it cover the time for Linux to take over the hardware watchdog, there is nothing to be done in EFI Stub to benefit from the new protection.
But this scheme fails to handle FF-A update capsules which can take a long time. So either the period is long enough to support that or we need a FF-A watchdog service. Based on Siemens feedback, time to update can last 20 minutes. StandAloneMM may also need such a protection so FF-A watchdog service seems desired.
I'd be happy to receive feedback on the problem itself (watchdog in EFI) and on the possible solution (FF-A based).
Cheers
FF _______________________________________________ boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture
-- François-Frédéric Ozog | Director Business Development T: +33.67221.6485 francois.ozog@linaro.org | Skype: ffozog
On Tue, 29 Jun 2021 at 17:00, Tom Rini trini@konsulko.com wrote:
On Tue, Jun 29, 2021 at 02:23:15PM +0200, Heinrich Schuchardt wrote:
On 6/29/21 12:48 PM, François Ozog wrote:
Hi
Following a discussion with Civil Infrastructure Project TSC, there is a watchdog protection issue with EFI: the time between the call to ExitBootService and Linux kernel takes over watchdog service is not covered by any watchdog protection.
The UEFI specification requires a watchdog. So this must be an implementation specific problem. Which firmware-hardware combination are you referring to?
U-Boot provides hardware watchdogs.
Without side-tracking things too much, I do want to note the last time within U-Boot we talked about watchdogs and UEFI specification, I believe there was still some doubt how hardware watchdogs are handled vs the notion of a software watchdog, and it should probably be clarified which things we're talking about here.
My reading is that implementation may choose between "soft" are hardware watchdog. But the watchdog is active only up until ExitBootService: "Resets and sets a watchdog timer used during boot services time.". So I can understand some implementations that use hardware watchdog enhance the behavior and keeps active after ExitBootService is called: but that is a platform specific behavior. I'd like a specification and implementations for it.
-- Tom
On Tue, Jun 29, 2021 at 05:33:29PM +0200, François Ozog wrote:
watchdog. But the watchdog is active only up until ExitBootService: "Resets and sets a watchdog timer used during boot services time.". So I can
More particularly the defintion of EFI_BOOT_SERVICES.SetWatchdogTimer() on page 227 of v2.9 of the UEFI spec says "On successful completion of EFI_BOOT_SERVICES.ExitBootServices() the watchdog timer is disabled." so for continutity of coverage the OS will need to make alternative arrangements before calling ExitBootServices().
tf-a@lists.trustedfirmware.org
-
François Ozog -
Heinrich Schuchardt -
Mark Brown -
Tom Rini