Hi Gábor,

> Could you send me the Jenkins logs that will help me find out what's wrong with those tests?

Sure, sorry, I meant to do that yesterday right after replying to your email, but apparently I got distracted. I just posted the details on github.

> would like to correct it asap to be able to finish this task, so it can be part of the library in the next release. 

Just to set expectations: in addition to passing the CI, each PR needs to be reviewed and approved by two reviewers from the project. As we're receiving fairly large number of contributions, review bandwidth sometimes turns out to be limiting, especially for non-trivial PRs. All that to say that we'll do our best but can't really guarantee how much time it will take to get your PR merged.

But the first step is obviously to get the CI to pass, so let's start with that!

Best regards,
Manuel.

From: Gábor Tóth <toth92g@gmail.com>
Sent: 11 February 2021 14:25
To: Manuel Pegourie-Gonnard <Manuel.Pegourie-Gonnard@arm.com>
Cc: mbed-tls@lists.trustedfirmware.org <mbed-tls@lists.trustedfirmware.org>
Subject: Re: [mbed-tls] Failing CI tests, Generating coverage report
 
Hello Manuel!

I have been thinking of making a VM or a partition for it, to get some experience, but haven't had enough free time yet. Right now I am working on a company topic (we started using MBEDTLS for security in automotive environment, but we need this functionality I am implementing currently) and cannot install it on this laptop unfortunately.

Could you send me the Jenkins logs that will help me find out what's wrong with those tests? I would like to correct it asap to be able to finish this task, so it can be part of the library in the next release. 
After that I am planning to create a linux machine with all the tools necessary and to dig a bit into the scripts of MBEDTLS.

Until then I will try out the Coverage solution you mentioned. The sh scripts are failing now, but it seems that I have something with my toolchain.

Thanks,
Gábor

Manuel Pegourie-Gonnard <Manuel.Pegourie-Gonnard@arm.com> ezt írta (időpont: 2021. febr. 11., Cs, 13:25):
Hi Gábor,

Congrats on fixing the previous issues and getting Travis to pass.

Unfortunately, the complete logs of the Jenkins part of the CI can only be accessed by Arm employees so far. We have plans to move to a fully-public CI system, but this won't happen before the second half of this year. In the meantime I'm afraid you'll have to ask a team member to get you the results.

> continuous-integration/jenkins/pr-head — This commit cannot be built

> continuous-integration/jenkins/pr-merge — This commit cannot be buil


Note: this can be ignore, these are the results of the parent jobs that spawn the other jobs.


> PR-4117-head TLS Testing — Failures: all_sh-ubuntu-16.04-test_m32_o1 all_sh-ubuntu-16.04-test_memory_buffer_allocator

> PR-4117-merge TLS Testing — Failures: ABI-API-checking all_sh-ubuntu-16.04-test_memory_buffer_allocator


Note: this is a partial list of failed components (truncated due to limits in the Github notifications API). Most of them as components of tests/scripts/all.sh - if you have access to a Linux/Unix machine with the proper dependencies, you can run them locally, for example here the first one would be tests/scripts/all.sh test_m32_o1, the second one tests/scripts/all.sh test_memory_buffer_allocator.

Regarding your question about coverage: I _think_ it should work on Windows if you exclude ssl-opt.sh and compat.sh. As your work is related to X.509, excluding those SSL-related scripts should not affect coverage of the areas of interest. I'm not sure if lcov (the programs that turns gcov's raw data to human-readable form) works on Windows, I'm sure your favourite search engine will have more information than me about this 🙂

Note however that most of the development/testing tools are made with Linux in mind, so you might have an easier time running a Linux VM to use them.

As a last resort, one of the reviewers of the PRs can measure coverage on their machine - that's something I often do for PRs adding features, especially when some form of parsing is involved. Thanks for paying attention to coverage, by the way, that's really appreciated!

Best regards,
Manuel.

From: mbed-tls <mbed-tls-bounces@lists.trustedfirmware.org> on behalf of Gábor Tóth via mbed-tls <mbed-tls@lists.trustedfirmware.org>
Sent: 11 February 2021 11:39
To: mbed-tls@lists.trustedfirmware.org <mbed-tls@lists.trustedfirmware.org>
Subject: [mbed-tls] Failing CI tests, Generating coverage report
 
Hello Guys!

I have a branch that has been successfully built, but 4/10 tests are failing. Unfortunately I am not even able to read a log, because if I click on the details button I am redirected to a "This site can't be reached" page.

Could you help me how to check the logs regarding the Jenkins CI builds?
These are the failing tests:

continuous-integration/jenkins/pr-head — This commit cannot be built

continuous-integration/jenkins/pr-merge — This commit cannot be buil

PR-4117-head TLS Testing — Failures: all_sh-ubuntu-16.04-test_m32_o1 all_sh-ubuntu-16.04-test_memory_buffer_allocator

PR-4117-merge TLS Testing — Failures: ABI-API-checking all_sh-ubuntu-16.04-test_memory_buffer_allocator

 

I am also working on tests to cover the new functionality, but can not run coverage report. In the makefile of the base directory of mbedtls I have found this part:

ifndef WINDOWS
# note: for coverage testing, build with:
# make CFLAGS='--coverage -g3 -O0'
covtest:
$(MAKE) check
programs/test/selftest
tests/compat.sh
tests/ssl-opt.sh


So am I right, that coverage check is only supported ot linux platform? Do I need to use that one or are there any solutions on windows?


Thank you in advance!


BR,

Gábor