Hi Gábor,
Congrats on fixing the previous issues and getting Travis to pass.
Unfortunately, the complete logs of the Jenkins part of the CI can only be accessed by Arm employees so far. We have plans to move to a fully-public CI system, but this won't happen before the second half of this year. In the meantime I'm afraid you'll have to ask a team member to get you the results.
continuous-integration/jenkins/pr-head — This commit cannot be built
continuous-integration/jenkins/pr-merge — This commit cannot be buil
Note: this can be ignore, these are the results of the parent jobs that spawn the other jobs.
PR-4117-head TLS Testing — Failures: all_sh-ubuntu-16.04-test_m32_o1 all_sh-ubuntu-16.04-test_memory_buffer_allocator
PR-4117-merge TLS Testing — Failures: ABI-API-checking all_sh-ubuntu-16.04-test_memory_buffer_allocator
Note: this is a partial list of failed components (truncated due to limits in the Github notifications API). Most of them as components of tests/scripts/all.sh - if you have access to a Linux/Unix machine with the proper dependencies, you can run them locally, for example here the first one would be tests/scripts/all.sh test_m32_o1, the second one tests/scripts/all.sh test_memory_buffer_allocator.
Regarding your question about coverage: I _think_ it should work on Windows if you exclude ssl-opt.sh and compat.sh. As your work is related to X.509, excluding those SSL-related scripts should not affect coverage of the areas of interest. I'm not sure if lcov (the programs that turns gcov's raw data to human-readable form) works on Windows, I'm sure your favourite search engine will have more information than me about this 🙂
Note however that most of the development/testing tools are made with Linux in mind, so you might have an easier time running a Linux VM to use them.
As a last resort, one of the reviewers of the PRs can measure coverage on their machine - that's something I often do for PRs adding features, especially when some form of parsing is involved. Thanks for paying attention to coverage, by the way, that's really appreciated!
Best regards, Manuel.
________________________________ From: mbed-tls mbed-tls-bounces@lists.trustedfirmware.org on behalf of Gábor Tóth via mbed-tls mbed-tls@lists.trustedfirmware.org Sent: 11 February 2021 11:39 To: mbed-tls@lists.trustedfirmware.org mbed-tls@lists.trustedfirmware.org Subject: [mbed-tls] Failing CI tests, Generating coverage report
Hello Guys!
I have a branch that has been successfully built, but 4/10 tests are failing. Unfortunately I am not even able to read a log, because if I click on the details button I am redirected to a "This site can't be reached" page.
Could you help me how to check the logs regarding the Jenkins CI builds? This is the pull request https://github.com/ARMmbed/mbedtls/pull/4117 These are the failing tests:
continuous-integration/jenkins/pr-head — This commit cannot be built
continuous-integration/jenkins/pr-merge — This commit cannot be buil
PR-4117-head TLS Testing — Failures: all_sh-ubuntu-16.04-test_m32_o1 all_sh-ubuntu-16.04-test_memory_buffer_allocator
PR-4117-merge TLS Testing — Failures: ABI-API-checking all_sh-ubuntu-16.04-test_memory_buffer_allocator
I am also working on tests to cover the new functionality, but can not run coverage report. In the makefile of the base directory of mbedtls I have found this part:
ifndef WINDOWS # note: for coverage testing, build with: # make CFLAGS='--coverage -g3 -O0' covtest: $(MAKE) check programs/test/selftest tests/compat.sh tests/ssl-opt.sh
So am I right, that coverage check is only supported ot linux platform? Do I need to use that one or are there any solutions on windows?
Thank you in advance!
BR,
Gábor