Hello Manuel!
I have been thinking of making a VM or a partition for it, to get some experience, but haven't had enough free time yet. Right now I am working on a company topic (we started using MBEDTLS for security in automotive environment, but we need this functionality I am implementing currently) and cannot install it on this laptop unfortunately.
Could you send me the Jenkins logs that will help me find out what's wrong with those tests? I would like to correct it asap to be able to finish this task, so it can be part of the library in the next release. After that I am planning to create a linux machine with all the tools necessary and to dig a bit into the scripts of MBEDTLS.
Until then I will try out the Coverage solution you mentioned. The sh scripts are failing now, but it seems that I have something with my toolchain.
Thanks, Gábor
Manuel Pegourie-Gonnard Manuel.Pegourie-Gonnard@arm.com ezt írta (időpont: 2021. febr. 11., Cs, 13:25):
Hi Gábor,
Congrats on fixing the previous issues and getting Travis to pass.
Unfortunately, the complete logs of the Jenkins part of the CI can only be accessed by Arm employees so far. We have plans to move to a fully-public CI system, but this won't happen before the second half of this year. In the meantime I'm afraid you'll have to ask a team member to get you the results.
*> continuous-integration/jenkins/pr-head *— This commit cannot be built
*> continuous-integration/jenkins/pr-merge *— This commit cannot be buil
Note: this can be ignore, these are the results of the parent jobs that spawn the other jobs.
*> PR-4117-head TLS Testing *— Failures: all_sh-ubuntu-16.04-test_m32_o1 all_sh-ubuntu-16.04-test_memory_buffer_allocator
*> PR-4117-merge TLS Testing *— Failures: ABI-API-checking all_sh-ubuntu-16.04-test_memory_buffer_allocator
Note: this is a partial list of failed components (truncated due to limits in the Github notifications API). Most of them as components of tests/scripts/all.sh - if you have access to a Linux/Unix machine with the proper dependencies, you can run them locally, for example here the first one would be tests/scripts/all.sh test_m32_o1, the second one tests/scripts/all.sh test_memory_buffer_allocator.
Regarding your question about coverage: I _think_ it should work on Windows if you exclude ssl-opt.sh and compat.sh. As your work is related to X.509, excluding those SSL-related scripts should not affect coverage of the areas of interest. I'm not sure if lcov (the programs that turns gcov's raw data to human-readable form) works on Windows, I'm sure your favourite search engine will have more information than me about this 🙂
Note however that most of the development/testing tools are made with Linux in mind, so you might have an easier time running a Linux VM to use them.
As a last resort, one of the reviewers of the PRs can measure coverage on their machine - that's something I often do for PRs adding features, especially when some form of parsing is involved. Thanks for paying attention to coverage, by the way, that's really appreciated!
Best regards, Manuel.
*From:* mbed-tls mbed-tls-bounces@lists.trustedfirmware.org on behalf of Gábor Tóth via mbed-tls mbed-tls@lists.trustedfirmware.org *Sent:* 11 February 2021 11:39 *To:* mbed-tls@lists.trustedfirmware.org < mbed-tls@lists.trustedfirmware.org> *Subject:* [mbed-tls] Failing CI tests, Generating coverage report
Hello Guys!
I have a branch that has been successfully built, but 4/10 tests are failing. Unfortunately I am not even able to read a log, because if I click on the details button I am redirected to a "This site can't be reached" page.
Could you help me how to check the logs regarding the Jenkins CI builds? This is the pull request https://github.com/ARMmbed/mbedtls/pull/4117 These are the failing tests:
*continuous-integration/jenkins/pr-head *— This commit cannot be built
*continuous-integration/jenkins/pr-merge *— This commit cannot be buil
*PR-4117-head TLS Testing *— Failures: all_sh-ubuntu-16.04-test_m32_o1 all_sh-ubuntu-16.04-test_memory_buffer_allocator
*PR-4117-merge TLS Testing *— Failures: ABI-API-checking all_sh-ubuntu-16.04-test_memory_buffer_allocator
I am also working on tests to cover the new functionality, but can not run coverage report. In the makefile of the base directory of mbedtls I have found this part:
ifndef WINDOWS # note: for coverage testing, build with: # make CFLAGS='--coverage -g3 -O0' covtest: $(MAKE) check programs/test/selftest tests/compat.sh tests/ssl-opt.sh
So am I right, that coverage check is only supported ot linux platform? Do I need to use that one or are there any solutions on windows?
Thank you in advance!
BR,
Gábor