*Attendees*: Don, Kelley Spoon, Ben C, Xinyu, Matteo, Riku, Joanna, SHebu,
Anton, Riku
*Minutes*:
- Musca-b's received by the lab
- Do existing ones need swapped out?
- Don: If shelf as "hot spares" would like to pull out, configure,
and validate.
- If Musca's continue to fail, need to deep dive on root cause.
- Three were shipped.
- Glen: Create a ticket to configure and validate prior to shelving
as hot spares
- Ben: TFC-200 closed out (inconsistent test results).
- Several tickets to close
- Glen: Issue TFC-199 logged by Joanna. Failed docker builds
- Arm changed compiler release process. Need a new process to get
compilers/docker images.
- Joanna: Options being explored. Automation of login.
- Riku. Arm put this in place to force login to use the tool. We're
doing a work around of their intent
- Glen: Uneasy on storing static versions.
- Joanna: This is all projects, not just TF-A. Immediate issue
resolved
- TFC-93
- Should be getting the 3rd NXP board and needs put into the lab.
- Include in daily runs?
- Xinyu: Already pushed patches to enable NXP in LAVA.
- Anton: Someone from NXP should merge the requests.
- Action: Talk to NXP (Andrej) to get permissions. Once permission,
a PR will need merged (
https://review.trustedfirmware.org/c/ci/tf-m-ci-scripts/+/12584 &
https://review.trustedfirmware.org/c/ci/tf-m-ci-scripts/+/12524 )
- Chromebooks
- Theo has these up and running, will be going into the lab to add
these.
- Cypress: On deck to be added to lab. Assigned to next sprint.
- MBedTLS - Working on installing infrastructure
- Kelly - setting up role strategy plugin for Jenkins. Working on it
now.
- Anton: Tech Forum this Thursday. Will cancel.
<end>
Attendees: Don, Xinyu, Ben, Dave R, Joanna, Riku, Shebu, Anton
Minutes:
- Glen: Adjusted TFC board based on Joanna and Shebu's priority lists.
- In Progress: Leonardo working on TFC-176 and TFC-7
- In Progress: NXP boards are in. Leaving in Progress for now. A
couple tests are still not running, but working thru it.
- Discussed who is responsible when vendor bugs found
- Anton: Maybe a TSC decision
- Shebu: Suggest platform owners. Don't know if written down anywhere.
- Joanna: No TF-A boards in, but when tests failing, need to go back
to the vendor. In some cases, if infrastructure, wouldn't go back to
vendor. If believe it's h/w, then go back to vendor.
- Anton: In TF-M has platform owners documented.
- Shebu: May want to formalize at next Board meeting. Take to TSC as
well. Members need to keep an eye on their h/w.
- What about daily checks?
- Joanna: I do for TF-A. Need escalation route to board owner.
- Shebu: Should Open-CI mailing list add platform owners?
- Anton: Not sure that's the best way. Likely a 1:1 email
- Joanna: Typically sends notes directly to Linaro team or board
owner.
- Glen: This is focused on s/w issues. Pretty good when a hard
failure related to h/w.
- Summary: Notify Board on plans. Team leads already monitor
- Chromebook TFC-45 (Asudra) are being worked on. Next Chromebook was
also received.
- Cypress h/w goes into the next sprint to install
- MBedTLS - transition work from Leonardo to Arthur.
- Also have Riku involved on the authentication model
- STG-2704: Haven't started work yet on this (Roles).
- Joanna: Slowing down builds could cause builds to bump into each other
from different GEO's.
- MBedTLS has been delayed for a while, depending upon confidence level.
- Anton: Thanks to the Open CI team for getting the Musca-b's back
online. More boards coming to Linaro.
- Glen: Paul Black from Arm reached out about changing compiler
licensing. Will need to understand this. Glen to create a new TFC ticket.
Attendees: Don, Glen, Joanna, Riku, Dave R, Anton, Ben C, Xinyu, Matteo
Minutes:
- Action: Glen to merge Shebu and Joanna's priority lists into TFC
- Joanna: Gerrit labels couldn't be access labels. Created ticket
TFC-196.
- Anton: Need the Musca-b's updated. Also got 5 more mucsa-b's to send.
- Don: Do they need set up?
- Anton: Need one board w/ OTP, but need to confirm internally, so hold
off. Otherwise should just work.
- Ben C: TIcket?
- Action: Need to create tickets
- Anton: TFC-190, 183.
- Glen: R-pi workers getting updated as well
- Ben: Plan to get one on line today.
- Xinyu - really need 2-3 Musca-b's up to run in parallel so that
release isn't extended.
- Glen: Meeting slot - move to Wednesday at same time?
- Won't work, will investigate other possibilities
- Boards advancing
- MBed TLS - working thru authentication issues.
- Riku: Plan to switch to new Open CI authentication - must include
renaming some other jobs, but better than rewriting some MBed TLS code.
Will test this on staging server first.
don
Attendees: Shebu, Xinyu, Glen, Don, Anton, Ben C, Dave R
Minutes:
- Glen: Got Joanna's priority list, still need from Shebu
- Shebu: Have one now for TF-A and TF-M
- Anton: How many licenses are needed to run IAR on open CI?
- Glen: I recall that we changed the way licenses were attained. Not
sure on details, but will follow up
- Shebu to forward the priority list to Glen.
- Glen: For MBed TLE, using the same milestones previously developed.
Leonardo is working w/ Daryl(?) on the best way to implement rule-based
authentication in order to complete Milestone 3
-
https://linaro.atlassian.net/jira/software/c/projects/TFC/boards/70?modal=d…
- Team will be syncing up to determine the best way to implement
TFC-121
- Board status
- Theo picked up Chromebook in Cambridge
- Glen: NXP - resolving a flashing issue, expect to go live this week
- Shebu: Code freeze on 12th, so whatever is available in Open CI
will be used in release testing
- Glen: Cypress follows NXP
- Updates on Musca B?
- Ben: Goal is Nov 8, NXP was a priority last week.
- Shebu: Do need before Nov 8 code freeze date
-
- Glen: Still have Leo on the team, Theo and Paul spinning up. Theo
has been in ramp down but part-time last week. Theo helping to get boards
in LAVA
- Glen: The new list will be integrated with the new priorities,
TF-A/TF-M will be interleaved. Finish the In-Progress first.
- Shebu: Agreed
- TF-M Release info
-
plan to start the process from Nov 15
-
Release target date: Nov 26
-
This time we plan to use a separate Release branch and avoid code
freeze.
- Glen: TFC-188: Tuxput - don't see getting to this year.
Attendees: Glen, Xinyu, Riku, Don, Ben C, Anton, Joanna, Matteo
Minutes:
- Glen: Plugins? TFC-121 MBed TLS plugins
- Riku: Still working on it. STG-2633
- Glen: Staffing.
Theo onboard, setting up LAVA at home and will be integrating next
Chromebook.
Leonardo remains onboard for a bit more time, will continue on MBed
TLS.
Arthur getting NXP boards into LAVA as well as TFC-82 Github auth.
support to LAVA - patch applied, need LAVA update now
- Glen: Recent LAVA change creating a Code Coverage regression. Email
thread only. TF-A was working, now not. So a new ticket will be created
- Glen: backlog / sc-approved cleanup review
In progress: Reviewed, staffed, and all looks good
SC Approved: Expect script to be started w/ Paul
Several others are queued up.
TFC-29 - Joanna to check to see if this needs worked on. Matteo, also
want to visit backlog to see if higher priorities. Moved back to backlog.
- Anton: TFC-33 and 135 in backlog may be really simple - permission
changes
- Joanna: TFC-75 - would be good to start on this one, some sync issues
on gerrit. Hoping it's another quick one. Joanna to follow up and update
the ticket.
- Joanna to prioritize TF-A tickets in backlog then we'll prioritize
those against other backlog items
*Attendees*: Don, Joanna, Matteo, Xinyu, Riku, Janos Follath, Anton, Dave
Rodgman, Ben Copeland, Shebu
*Minutes*:
- Glen: Staffing - Updated team on the staffing updates
- Theo and Paul joining.
- Leonardo transitioning over the next few weeks.
- Glen to meet w/ whole team tomorrow
- Leonardo still best person to contact on MBedTSL
- MBedTLS
- M2 completed, M3 blocked on need for plugins
- Riku to take care of plugins, targeting today to complete
- TFC-176
- Still completing
- TFC-82: Arthur was working on this. Want his updates released and
should complete
- TFC-87: Arthur has been working w/ Joanna on this. May take a couple
of passes, but initial solution in the next day
- TFC-183 - MUSCA B are off line.
- Ben: Dave looking into this. Failed due to debian update and debian
bug.
- Glen: Back it out?
- Ben recommends moving forward and resolving in next sprint (next
week). Urgency?
- Anton: Only MUSCA B offline? Preparing for next month release, so
need by then for sure.
- Agreed to put in next week
- Glen: NXP Boards coming up in Lab
- Glen: Chromebooks (Jacuzi - TFC-45) received in Cambridge, got out of
customs!
- Glen: Creating card for next set of Chromebooks (Lazor)
- Glen: Renesas beginning to ship the h/w
- Shebu: Renesas Intend to push board support for TF-M
- Matteo: Any priority calls to make?
- Glen: Some of the smaller backlog tickets. Glen will be moving items
out of backlog and assigning them.
- Matteo: To draft an email on backlog email and share w/ Glen.
- Shebu: Some SC Approved have yet to be touched.
- How much effort for Disaster Recovery? It could be a larger task.
Will break this down into subtasks and do the assessment.
- ACTION: Joanna, Anton, Matteo, Shebu to go thru backlog and SC
approved and make adjustments in the Kanban. Keep others updated via email.
Attendees: Don, Riku, Xinyu, Shebu, Dave Rodgman, Ben, Joanna
Minutes:
- Several platforms ready to add to the Cambridge lab. Includes NXP,
Cyrpress, Google
- TFC-181: patches in review
- TFC-176: Continue to work. Moed from SC Approved to In Progress.
- May have a temporary fix from Leonardo.
- Monitoring at this point. Joanna recommends holding off on ticket
TFC-96 until TFC-176 is stable so as not to add more variables into the
solution. After a few days of stability, can come back to this.
- Moved TFC-36 to REOPENED and prioritized
- Moved TFC-36 to SC Approved
- Verify MBed status. Still in Blocked.
- Action: Don check w/ Glen/Leonardo as to why the Mbed work still in
"Blocked"
- Anything else to work on here?
- Joanna: TFC-87: Would be good to start if time allows. Doesn't have
to all be done at once.
- Shebu: Update on staffing
- Two engineerings from Linaro coming on to help for next 6 months to
supplement the staff. Theodore Grey & Paul Sokolovskyy (Welcome aboard!)
- Will want to make sure the team is appropriately loaded.
- Board additions will continue.
- MBedTLC CI up and running important too.
- Over the next couple of weeks Glen will be integrating them into
the team as they spin up.
*Attendees*: Don, Glen, Anton, Shebu, Matteo, Joanna, Ben C, Dave Rodgman,
Riku
*Minutes*:
- Servers went down - certificate expiration issue resolved. Blocker
- TFC-179/TFC-80 (duplicate) - certificates updated and these can be
resolved
- Glen: Adding boards:
- Have documentation on boards. Should we add vendor boards to
documentation?
- User's manual.
- Determine if public?
- Don: Assumed it was public.
- Shebu: Worth asking Eric, Julius, Sean, and others if they're OK to
do that.
- Glen: Can share what would be published in the User's Guide first.
- Would be OK to point to vendor info if works
- Anton: TF-A/M docs could also point to this
- Platform integration updates:
- Glen: Cypress/NXP in Cambridge. Will get integrated shortly.
- Google Chromebooks.
- Arthur and Fathi's returned to sender, the 3 platforms made it
to the lab.
- Working with Google to start the 2nd platform.
- MBedTLS
- Dave's team provided the AMI files.
- Dave: Darryl expected additional AMI's beyond Ubuntu image. Will
build additional configs from those after Ubuntu.
- TFC-176: Leonardo has a work-around (3 retries), but not getting to
root cause. Likely need a longer-term solution.
- Glen asking for Expect script review from Joanna.
- TFC-82: Github auth. Working, now looking to deploy.
- Anton: TF-M health check job. Expected OpenCI team to keep it up.
Sanity test is failing.
https://ci.trustedfirmware.org/view/TF-M/job/tf-m-infra-health/
- Don: Suggest creating a ticket.
- Action: Glen/Ben to open a ticket so that can track/assign.
Thanks
don
*Attendees*: Don, Glen, Anton, Riku, Joanna, Matteo, Xinyu, Dave Rodgman,
Shebu
*Minutes*:
- Glen: Shared TFC Board
- Boards:
- Chromebooks - returning to get out of customs and then sending back
again.
- Cypress/NXP being integrated in October. Received h/w in lab
- TFC-82: Arthur working on this while have time
- TFC-36 is ready for Arm team to review. Leonardo
- TFC-176: Intermittent failures. Leonardo continuing to isolate this
- TFC-20 integrated including a resolved regression. Any issues noticed?
- Arm provided AMI files. Go back to mbedTLS?
- Matteo: Will we finish TFC-36? Need to make sure this finishes.
- Glen: Agreed
- Joanna: Would like to continue w/ TFC-176 continue at least part
time.
- Glen recommends re-evaluating after his work day today
- Joanna: Agree to keep it as a background task
- TFC-172: In backlog, not any data in the ticket on quantifying slow
or how to replace.
- Xinyu: No longer reproducing. Xinyu to update ticket and we will
close
- TFC-171: Seems to be solved from other infrastructure improvements
- Joanna: OK to close resolved. Not sure what it was but not
occurring now
- Arthur may have bandwidth for another task
- Joanna: TFC-87 may be a good one to work on. Currently the team
performs a work-around.
- Moved TFC-87 to SC Approved from Backlog
- *Action: Anton* to evaluate moving TFC-173 to SC Approved.
Don
Attendees: Riku, Don, Glen, Matteo, Shebu, Anton, Joanna, Dave Rodgman
Minutes:
- TFC-20: Git performance - infrastructure changes happened including
Leonardo infrastructure changes. Tested on stage. Should keep an eye on it
for next few days. New machine already added back in. Better performance
will be seen as well since more jobs can focus on builds and not clones.
- Expect scripts: Should wrap up this week
- Joanna: Brought up LAVA timeouts TFC-176. Pass on 2nd or 3rd
attempts. Initial analysis is too many parallel LAVA jobs. Starting out by
increasing timeout. Would prevent re-running jobs.
- Riku: Should add LAVA lab folks to this ticket since adjusting
timeouts
- Boards status:
- Chromebooks still dealing w/ import issues.
- *ACTION: Don* ask Julius to reject and resend the 3 boards to the
Cambridge lab
- Cypress and NXP platforms now in the lab
- LAVA team will be updating to latest release with the new board
configs included. Want this done in the next couple of weeks
- Arthur will be available for some other tasks.
- MBedTLS
- Dave: AMI images almost ready. Expect it soon.
- Glen: Linaro support prepared to copy to AMI's when they are queued
up.
- Glen: Joanna's new list of issues
- LAVA timeout was one.
- TFC-87: Joanna's team reviewing that one. CI reporting ticket. Need
some guidance/access from Leonardo
- Glen will let him know
Thanks
Don
