May 3
Attendees: Matteo, Don, Shebu, Antonio, Dave Rodgman <dave.rodgman(a)arm.com>
Action: Action: Don, check with Joanna to see if there Is commonality
between TFC-14 and TFC-129? Windows tests/builds for TF-A and Mbed TLS. Is
the build environment reuseable? Note Joanna is out this week
Minutes:
-
Shebu: Issues regarding slow CI.
-
Don: MBed TLS - Executor # increased to get thru the queue. Not sure if
it’s been completed yet
-
Dave: Quick scan showing the queue burning down.
-
Shebu: TF-M side, tests being run.
-
Antonio: TF-M was looking good Friday.
-
Held MISRA tooling meeting last week. No ticket updates. Need to
update tickets.
-
TF-A first. TF-M will be handled separately.
-
Shebu: TF-M is marked for next quarter.
-
Don: Noted serial - after TF-A.
-
MBed TLS
-
Moving forward on Windows 11, raised the priority
-
Action: Don, check with Joanna to see if there Is commonality between
TFC-14 and TFC-129? Windows tests/builds for TF-A and Mbed TLS. Is the
build environment reuseable? Note Joanna is out this week.
-
TFC-251: Want to prioritise the documentation so that developer docs are
….
-
TFC-247: Must determine who mirrors the repo prior to configuring the
scripts. Dev team vs Open CI dev team.
-
Don added a note to TFC-184 to understand status of 3rd Lazor. Moved to
Resolved.
Attendees: Don, Glen, Joanna, Antonio, Ben, Dave R, Matteo, Shebu, Xinyu
Minutes:
-
Update
-
Arthur - Mbed TLS working on windows, FreeBSD confirmed w/ team
-
Paul wrapped up CodeCoverage; PSA Testing
-
Theo working on boards, still waiting to hear from ST
-
Performance is slow
-
Dave R / Xinyu - Mbed TLS running slowly; seeing some time outs
-
One build that has been running 19hrs.
https://github.com/Mbed-TLS/mbedtls/pull/5742
-
Xinyu - TF-M also appears to be running a bit slow.
-
https://ci.trustedfirmware.org/view/TF-M/job/tf-m-nightly/922/
-
Action: Glen log a ticket on infrastructure running slow - TFC-260
<https://linaro.atlassian.net/browse/TFC-260>
-
Discuss MISRA Planning sync - looking to have a meeting set for Friday.
Joanna & Sandrine can attend. Awaiting to hear back from Paul.
-
Start with TF-A, follow on with TF-M after that.
-
PSA Compliance Tests, TFC-1, why in SC Approved? Will check, should be
In-Progress
-
MbedTLS - inquiry from ST, Shebu will provide a response. Dave’s team
is working on a change that can potentially reduce the overall cost.
-
Retrospective on maintenance
-
Ben, security fix. A bug in Jenkins only on Prod. No root cause
found, so rolled back. Working to reproduce in staging, but haven’t done
so yet.
-
Was Jenkins maintenance planned or security reasons?
-
Ben: Was planned with notices, but was also a security fix
-
Message was sent out to TF-Triage mail list
-
https://lists.trustedfirmware.org/archives/list/tf-openci-triage@lists.trus…
-
Consider a new list for updates
-
The slow down? That looks unrelated.
-
Releases
-
TF-A - slipped the release to the middle of May with completion at
the end of the month
-
TF-M - release should complete by the end of this week.
- CI Documentation - Joanna Farley <joanna.farley(a)arm.com> they will
add a new section (CI Developer) to the current documentation. Should
grow organically and when Linaro can add to the documents.
<end>
Attendees: Glen, Joanna, Anton, Don, Xinyu, Shebu
Status
-
Status from Xinyu about the production - all up and running? TFC-256
-
Xinyu: Still open
-
https://ci.trustedfirmware.org/view/TF-M/job/tf-m-build-and-test/4224/conso…
-
Glen to follow up
-
Updated Glen
-
Mbed TLS:
-
FreeBSD is up and running w/ Bences merging the needed PR on
Thursday. Tests are running in Production.
-
Working on Windows instance setup
-
Boards - waiting on ST boards to be shipped to Cambs. Chrome books
still an issue
-
Paul working on consolidation testing for code coverage
-
Awaiting feedback from Linaro on additional resources.
Hello All,
The servers will stop processing jobs on 2022-04-15 at around
16:00 UTC, as the server will be put into "Shutdown mode".
This downtime is needed in order to upgrade Jenkins and some of
its plugins.
Notable change:
As of 2.307, Jenkins will begin a naming migration to refer to the "master"
node as "built-in". We have submitted updates to existing jobs that request
"master" as the build node to add in the "built-in" label and will continue
to support "master" as a label on the new server.
Start: 2022-04-16 12:00 UTC
End: 2022-04-16 20:00 UTC
--
Kelley Spoon <kelley.spoon(a)linaro.org>
April 12th
Attendees: Joanna, Anton, Glen, Shebu, Xinyu, Don, Dave Rodgman, Ben,
Antonio
Status
-
Updates Glen:
-
MBed TLS FreeBSD AMI installed, so Arthur back on this.
-
Paul is working on Code Coverage
-
Working w/ Google on final LAVA installs. A flashing issue.
-
ST board nearing point to bring on line. Boards requested from ST, no
updates on their delivery status
-
Mbed TLS infrastructure costs - still under investigating
-
License technologies - 6.18
-
Doesn’t affect the TF-A; interest from TF-M
-
Review of Dashboard
-
TFC-222 (downloading) and FC-198(deploy licence) are in the right
priority in the SC Approved.
-
Linaro Mgt attempting to work w/ Arm on new Licensing technology
deployment
-
Shebu: When get close to IAR compiler support (TFC-13), Shebu will
help
-
TFC-236 - Ben: Kelly targeting this for the long weekend
-
Joanna: Using Arm CI again this time for upcoming release. Learning
about CI - want to document this somewhere, any progress?
-
No progress
-
Updated SSL, would be good to document this.
-
Need a CI developer perspective - go to new doc if want to change how
CI works. Could be a new section of the existing document.
-
Glen: Will be applicable for Disaster Recovery as well
-
Don to review whether a new document or appending to the current
document makes more sense. Make proposal.
-
TFC-254 and 252 completed
-
Anton brought up the inability to import a .ics from the calendar for
pulling invites into local outlook calendar.
-
Don to follow up
April 5th
Attendees: Joanna, Anton, Glen, Shebu, Xinyu, Don, Dave Rodgman
-
Infrastructure Changes
-
Python V2, V3
-
Issue with some jobs requiring python2. Python v2 is EOL Jan
2020, its a security hole. We will no longer be supporting
Python v2.
Teams should look to see if there are any dependencies.
-
Currently python2 is the first python on the path and kelly has
submitted changes.
-
Request for Open SSL v3 - Ubuntu 22.04
-
This can be done, should test on one image and see if there are
any failures.
-
Jenkins is another item that needs to be updated.
-
Recommending upgrades to the infrastructure.
-
Joanna: Also upgrade documentation when the team performs each
upgrade.
-
Shebu: A mid-April TF-M release is coming. Want to make sure CI is
stable for that
-
Start in staging
-
Mbed TLS
-
Confirmed with the Mbed TLS team that we are working on the Ubuntu
platform. All jobs are running and providing Results
-
Awaiting Costs from ARM team;
-
We are waiting on the Systems Team to provide AWS support for the
FreeBSD AMI.
March 29
Attendees: Joanna, Anton, Glen, Shebu, Xinyu,i Don
-
Mbed TLS
-
Dave R can’t make meeting today but if need anything let him know
-
Meeting Fridays w/ MbedTLS
-
Glen: New numbers based on PRs and averaging, down to $800 (for just
the Ubuntu platform).
-
Other two platforms won’t scale linearly
-
Awaiting Dave’s internal numbers.
-
Still need a plugin to complete/deploy the last Ubuntu platform.
Also need to deploy the FreeBSD AMI.
-
Glen: Paul got TF-M code coverage per user basis working. Will submit
some PRs soon.
-
Theo is working on the read-the-docs auto-updates for TF-A tests. Also
working on ST board.
-
Had a sync w/ Tim Benton here regarding the 2nd resource usage. Theo to
leave the team after ST board is in.
-
Shebu: regarding future tasks - need to determine how to work the
documentation request focused on developer usage. TFC-242
-
New licences required for the new compilers
-
*ACTION: Glen* to confirm when the licences will expire
-
Shebu: In April, will be good to review the backlog and make sure it’s
aligned.
-
Don finalizing MISRA contract - plan to take to the board for final
approval.
-
They are moving TF-A/TF-M/Trusted services to a new github repo. There
are some new domains that have been created. The TF-A team created all the
mirroring scripts, jobs.
-
Don updated https://github.com/TrustedFirmware with references to the
new Github repos for each project.
March 15
Attendees: Don, Joanna, Dave R, Anton, Glen, Shebu
Minutes:
-
Boards - Theo worked with Google and now has the Lazor’s flashing and
running
-
System Down?
-
Ben mentioned a network error
-
Joanna: Jobs seem to be running again
-
mbedTLS
-
Meeting with the team weekly. Code reviews have been happening and
PullRequest jobs started last week on production.
-
Next is start setup of FreeBSD
-
Joanna - resourcing challenges, some of the CI work they have been
relying on Linaro to do, there still are some gaps. Docs are user
specific, vs developer specific. Would be good to have some sessions and
some knowledge transfer.
-
Anton - Would be good to understand what the roles and
responsibilities are for people. When an issue occurs, who do
they contact.
-
Joanna - have some of the general knowledge, but some of the details.
-
Shebu - balance the day to day tactical and put the documentation
over time.
-
There was an openCI R&R that Ryan Harkin <ryan.harkin(a)arm.com>
*Don post-meeting comment*: ^^^ we've attempted to put a process in place
wherein for any and all issues found, step 1 is to <Create> a ticket from
the TFC Dashboard
<https://linaro.atlassian.net/jira/software/c/projects/TFC/boards/70> with
Labels == TrustedFirmware, setting the Priority appropriately, and
including a description.
The whole team watches this project, so this _should_ be a dependable way
to submit issues. Then Glen/team assign the issue to the right person and
you(customers) can be abstracted.
The above said, we also have the shared slack channel now, and if we're
online we can help answer questions/escalate issues/etc. We can
discuss/optimize/agree on/document the above more formally if it needs to
be.
-
Shebu - Mid March, ARM would like to extend the two engineers. Update?
Don has met with Tim Benton so he is updated and understands the request.
Glen has spoken to the engineers. Expecting follow-up from Engineering
soon.
Attendees: Matteo, Don, Joanna, Dave R, Anton, Glen, Ben C
Minutes:
-
Boards - Theo is in UK; working with lab team this week. SuzyQ cable
delivered to Lab.
-
TF-A Doc Generator; close out issue next week.
-
Working on TF Slide
-
MBed TLS teams meeting weekly now to keep aligned on progress
-
Arthur beginning work on FreeBSD AMI.
-
From the lab, any updates?
-
Working with paul w/ staging broken for a bit. Kelly is going to
resume the upgrade on stage. Important - node name master - TF-A & TF-M
define the nodename and change that to “build”. Timeframe? Will send out
email for reviews and the days for the maintenance. Targeting for this
weekend/next weekend.
-
TF-A & TF-M releases are target for April.
-
Few juno’s are off line, looking to get them back on line. They are
seeing jobs stuck on juno’s with bad usb memory cards.
-
Ben to reach out to dean arnold to see if this can help.
-
https://ci.trustedfirmware.org/job/tf-a-builder/805599/
-
Ticket TFC-237 Questions from TF-A team on how credentials/tokens.
Will ask Arthur to look into this, and include kelly for any feedback.
-
Joanna will pass on slides for Misra review for the board meeting.
Attendees: Joanna, Glen, Don, Dave R, Ben Copeland, Anton, Kelley Spoon
Minutes:
-
Paul is out, unclear when he’ll return.
-
Platforms
-
Cypress still debugging; 3 out of 4 firmware will work. We will
deploy those. Leave the 4th one in a blocker awaiting cypress.
-
Asurada - Missing SuziQ Cable; will make its way to cambridge next
week
-
Lazor - Debugging “Unable to flash the device”
-
Code Coverage - stop/more?
-
Joanna: Infrastructure in place, need to verify content now.
-
PSA in progress - not much detail
-
Replicating OpenCI within ARM? Support?
-
Done by Joanna’s team. For support, just consulting and issues
should filter thru Glen Valante <glen.valante(a)gmail.com>.
-
Reason for replication, is to cover the gaps, support the differences
like Windows server.
-
Does this replicate Lava? Yes, ARM has a lava farm and Dean Arnold
is helping with connecting that.
-
Anything from TFM?
-
Are we working with Coverity, Linaro is not working on that.
-
mbedTLS
-
Meeting with Dave Rodgman <dave.rodgman(a)arm.com>a nd team. Will have
a weekly meeting with them to keep up the progress. There are several
dependencies needed to work thru that we.
-
Target is to get the ubuntu completed for the end of March.
-
What will be done by Mid April?
-
Arthur - Mbed TLS; Ubuntu target for end of March.
-
Boards - deliver existing boards, wrap up with ST, small tickets
-
Paul - Wrap up PSA; CodeCoverage for TF-M
-
Action Glen: Matteo would like to see an initial list of tickets to
be finished in the month.