Greetings, Yesterday the gerrit server on review.trustedfirmware.org
experienced
significant downtime outside of any announced or scheduled maintenance windows
starting around 5pm UTC.
The downtime was caused by ansible deployment that was run to make some
configuration changes and install a plugin. The deployment made several
unintentional upgrades to other external plugins which were incompatible
with our environment and prevented gerrit from restarting.
As a result, for a 12 hour period the gerrit server was either off line or
highly unstable. As of 5am UTC today, gerrit had been returned to its
original state and resumed stable operation.
I apologize for the unannounced downtime and any inconvenience that it
caused, and we are reviewing the events to ensure a similar situation
doesn't occur again,
Regards,
--
Kelley Spoon <kelley.spoon(a)linaro.org>
FYI at the weekend I ran an OpenCI Performance test to see if the Tuxsuite AWS changes for running FVP based TF-A tests helps with the management of load and execution times over running the FVP tests on physical servers.
In short it does help!
See https://discord.com/channels/1106321706588577904/1210643193834508338 for details and analysis.
Joanna
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hello all,
My name is Luís and I'm part of the Deployments team.
We will be migrating the TF LAVA instance from its current deployment to
our new, Kubernetes-based platform, this Friday (09/02) at 9:30 (AM) UTC.
This means that the tf.validation.linaro.org service will be down for
the expected downtime of 2 to 3 hours. This is just a migration so no
down/upgrades or configuration changes to LAVA will happen.
Once the service is back online we'll announce it. Thanks for your
collaboration.
Regards,
--
Luís Tonicha <luis.tonicha(a)linaro.org>
