Hi,
I guess I'm late for the party. But I have some questions. Looking at the initial email from Andrew, these two points stood out.
--8<-- 1. Whilst looking at the speculative AT workaround in KVM, I compared it against the workaround in TF-A and noticed an inconsistency whereby TF-A **breaks** KVM's workaround. 2. TF-A will also have to be compatible with whatever workaround the EL2 software is using -->8--
Some questions about the unwanted dependency that this fix in TF-A creates.
1. Does KVM team always announce all the dependencies, along with their version numbers, that consumers should use? 2. With the fix in TF-A, are we forcing consumers to upgrade their TF-A blobs? Is this normal practice for consumers? 3. How do we plan to make the upgrade from TF-A side easier? Runtime detection of the feature (similar to the Spectre and Meltdown fixes) comes to mind.
-Varun
-----Original Message----- From: TF-A tf-a-bounces@lists.trustedfirmware.org On Behalf Of Manish Badarkhe via TF-A Sent: Thursday, July 30, 2020 10:48 PM To: tf-a@lists.trustedfirmware.org Cc: Will Deacon willdeacon@google.com; android-kvm@google.com; Marc Zyngier mzyngier@google.com; James Morse James.Morse@arm.com; Andrew Scull ascull@google.com Subject: Re: [TF-A] Erroneous speculative AT workaround
External email: Use caution opening links or attachments
Hi All
As per discussion over mailing list, we implemented workaround for AT speculative behaviour. If anybody interested they can look into the changes posted here: https://review.trustedfirmware.org/q/topic:%22at_errata_fix%22+(status:open%...) These changes are currently under review.
Thanks Manish Badarkhe
On 03/07/2020, 14:43, "TF-A on behalf of Soby Mathew via TF-A" <tf-a-bounces@lists.trustedfirmware.org on behalf of tf-a@lists.trustedfirmware.org> wrote:
> -----Original Message----- > From: Will Deacon willdeacon@google.com > Sent: 02 July 2020 15:47 > To: Soby Mathew Soby.Mathew@arm.com > Cc: Andrew Scull ascull@google.com; Raghu K > raghu.ncstate@icloud.com; android-kvm@google.com; Marc Zyngier > mzyngier@google.com; James Morse James.Morse@arm.com; tf- > a@lists.trustedfirmware.org > Subject: Re: [TF-A] Erroneous speculative AT workaround > > On Thu, Jul 02, 2020 at 02:15:47PM +0000, Soby Mathew wrote: > > So the fix as we currently understand would involve the following > > sequence > > : > > > > a. On Entry to EL3, save the incoming SCTLR_EL1.M and TCR_EL1.EPDx > bits and set them (ensure TCR_EL1.EPDx =1 prior to SCTLR_EL1.M =1 using > isb()) > > b. Prior to Exit from EL3, after the target context is restored, restore > the SCTLR_EL1.M and TCR_EL1.EPDx bits. > > > > The above sequence now means that any use of AT instruction targeted > > at lower EL from EL3 that require PTW will fault. So prior to use of > > AT, ensure the PTW are re-enabled and disabled back again after the AT > > instructions. > > > > If the above sequence is agreed upon to resolve the errata, then we > > can work on a patch for the same. I suspect current el1 register save > > and restore sequence in TF-A is a bit unwieldy and we may need to > > analyze all the entry points to EL3 to ensure we cover everything. > > Looks good to me, but there's still one niggle that I don't know how to solve. > If EL2 has been audited not to have any executable AT instructions, it may > not have a software workaround. However, if a secure interrupt is taken > from EL2 to EL3 while EL2 is the middle of a world switch, then there is a > small window where an AT instruction present at EL3 cold be speculatively > executed before you've had a chance to mess with SCTLR_EL1. > > Fun! Maybe it's worth documenting this somewhere?
Hi Will, Good point, this effectively means every EL2 software must implement the fix similar to KVM for this workaround to be effective (or else EL3 should also be audited to not to have any executable AT instruction). This needs to be communicated.
Since this is crossing TF-A boundary and need wider communication, I can initiate some internal discussion on how to communicate this properly.
Best Regards Soby Mathew
> > Will -- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
-- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi Varun,
On Fri, 31 Jul 2020 at 18:04, Varun Wadekar vwadekar@nvidia.com wrote:
Hi,
I guess I'm late for the party. But I have some questions. Looking at the initial email from Andrew, these two points stood out.
--8<--
- Whilst looking at the speculative AT workaround in KVM, I compared it against the workaround in TF-A and noticed an inconsistency whereby TF-A **breaks** KVM's workaround.
- TF-A will also have to be compatible with whatever workaround the EL2 software is using
-->8--
Some questions about the unwanted dependency that this fix in TF-A creates.
- Does KVM team always announce all the dependencies, along with their version numbers, that consumers should use?
I'm not sure what you mean by dependencies. There is no dependency between TF-A and KVM, other than the implicit promise that firmware doesn't break non-secure software. There is a bug in TF-A, TF-A gets fixed. Why should this be different from any other bug fix?
- With the fix in TF-A, are we forcing consumers to upgrade their TF-A blobs? Is this normal practice for consumers?
If your firmware isn't upgradeable, you have many more issues than just KVM...
- How do we plan to make the upgrade from TF-A side easier? Runtime detection of the feature (similar to the Spectre and Meltdown fixes) comes to mind.
This is a HW bug limited to a number of implementations, not a wide ranging issue spanning the whole architecture.
On the other hand, there is some scope for having a unified discovery mechanism of errata. I think someone is working on a spec.
I don't see how that's related to upgrading the firmware though.
M.
Hi Mark,
There is no dependency between TF-A and KVM, other than the implicit promise that firmware doesn't break non-secure software.
Don’t we need a certain version of TF-A to allow KVM to work as expected?
If your firmware isn't upgradeable, you have many more issues than just KVM...
With this change, we are forcing vendors to upgrade the firmware. This might involve testing and qualification on their part. I wanted to understand if this is a big deal.
This is a HW bug limited to a number of implementations, not a wide ranging issue spanning the whole architecture.
You are right. So a generic discovery mechanism does not make sense. Although static flags do create their own problems.
On the other hand, there is some scope for having a unified discovery mechanism of errata. I think someone is working on a spec
It would be interesting to see how they tackle this problem.
Cheers, Varun
-----Original Message----- From: Marc Zyngier mzyngier@google.com Sent: Friday, July 31, 2020 10:53 AM To: Varun Wadekar vwadekar@nvidia.com Cc: Manish Badarkhe Manish.Badarkhe@arm.com; Will Deacon willdeacon@google.com; android-kvm@google.com; James Morse James.Morse@arm.com; Andrew Scull ascull@google.com; tf-a@lists.trustedfirmware.org Subject: Re: [TF-A] Erroneous speculative AT workaround
External email: Use caution opening links or attachments
Hi Varun,
On Fri, 31 Jul 2020 at 18:04, Varun Wadekar vwadekar@nvidia.com wrote:
Hi,
I guess I'm late for the party. But I have some questions. Looking at the initial email from Andrew, these two points stood out.
--8<--
- Whilst looking at the speculative AT workaround in KVM, I compared it against the workaround in TF-A and noticed an inconsistency whereby TF-A **breaks** KVM's workaround.
- TF-A will also have to be compatible with whatever workaround the
EL2 software is using -->8--
Some questions about the unwanted dependency that this fix in TF-A creates.
- Does KVM team always announce all the dependencies, along with their version numbers, that consumers should use?
I'm not sure what you mean by dependencies. There is no dependency between TF-A and KVM, other than the implicit promise that firmware doesn't break non-secure software. There is a bug in TF-A, TF-A gets fixed. Why should this be different from any other bug fix?
- With the fix in TF-A, are we forcing consumers to upgrade their TF-A blobs? Is this normal practice for consumers?
If your firmware isn't upgradeable, you have many more issues than just KVM...
- How do we plan to make the upgrade from TF-A side easier? Runtime detection of the feature (similar to the Spectre and Meltdown fixes) comes to mind.
This is a HW bug limited to a number of implementations, not a wide ranging issue spanning the whole architecture.
On the other hand, there is some scope for having a unified discovery mechanism of errata. I think someone is working on a spec.
I don't see how that's related to upgrading the firmware though.
M. -- Jazz is not dead. It just smells funny...
HI Varun,
On Fri, 31 Jul 2020 at 21:57, Varun Wadekar vwadekar@nvidia.com wrote:
Hi Mark,
There is no dependency between TF-A and KVM, other than the implicit promise that firmware doesn't break non-secure software.
Don’t we need a certain version of TF-A to allow KVM to work as expected?
No. KVM predates TF-A, so there was never a dependency between the two, and KVM runs happily on systems that do not have TF-A, or even missing EL3 altogether. I fail to see why there would be a dependency between the two, to be honest.
If your firmware isn't upgradeable, you have many more issues than just KVM...
With this change, we are forcing vendors to upgrade the firmware. This might involve testing and qualification on their part. I wanted to understand if this is a big deal.
This HW erratum *is* a big deal, as you can observe TLB corruption. So each EL has to carefully manage its own workaround as well as that of the ELs it controls (EL0/EL1 for EL2, EL0/EL1/EL2 for EL3). This cannot be a coordinated workaround either as: - these are independent SW entities - traps and interrupts can occur in any context
M.
Hi,
On 01/08/2020, 15:36, "Marc Zyngier" mzyngier@google.com wrote:
HI Varun,
On Fri, 31 Jul 2020 at 21:57, Varun Wadekar vwadekar@nvidia.com wrote: > > Hi Mark, > > >> There is no dependency between TF-A and KVM, other than the implicit promise that firmware doesn't break non-secure software. > > Don’t we need a certain version of TF-A to allow KVM to work as expected?
No. KVM predates TF-A, so there was never a dependency between the two, and KVM runs happily on systems that do not have TF-A, or even missing EL3 altogether. I fail to see why there would be a dependency between the two, to be honest.
New SDEN for AT speculative errata behaviour will get published soon on developer.arm.com which clarifies why it is must for any EL2 software to implement this errata workaround if this errata workaround is enabled for EL3 software.
> > >> If your firmware isn't upgradeable, you have many more issues than just KVM... > > With this change, we are forcing vendors to upgrade the firmware. This might involve testing and qualification on their part. I wanted to understand if this is a big deal. >
This HW erratum *is* a big deal, as you can observe TLB corruption. So each EL has to carefully manage its own workaround as well as that of the ELs it controls (EL0/EL1 for EL2, EL0/EL1/EL2 for EL3). This cannot be a coordinated workaround either as: - these are independent SW entities - traps and interrupts can occur in any context
M. -- Jazz is not dead. It just smells funny...
Thanks Manish
tf-a@lists.trustedfirmware.org
-
Manish Badarkhe -
Marc Zyngier -
Varun Wadekar