Hi all,
I'd like to know if there is some way to retrieve the currently available number of bytes of application data without calling mbedtls_ssl_read()?
I'm writing a "TLS socket" for higher layers to use and would like to notify them when new application data is available, tell them how much it is, but leave it up to them when and how much to retrieve.
I'd like to prevent having to buffer all application data inside my TLS socket, because that would mean copying it once from mbedtls' buffer to my socket and then again from there to the application whenever it actually requests the data.
After a quick look into the sources, it seems like, if at all, this might be possible for single records. But all related fields are private and I could not find any API for this.
Issue #551 [1] seems related, but is more about peeking into the application data, while I would be fine with knowing just the size of available application data.
Thanks for any hints on how I could achieve this.
Best regards,
Jan
[1] https://github.com/Mbed-TLS/mbedtls/issues/551
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.
Hello,
Summary: I am soliciting feedback about builds of Mbed TLS where *there
is both a PSA implementation and the built-in software implementation of
the same algorithm, by design, *and the PSA implementation isn't just
calling the built-in implementation because the two have different
characteristics, each desirable in some context. Please read this
message if you are doing this or considering it. Feel free to ignore
otherwise.
For example, both MBEDTLS_SHA256_C and MBEDTLS_PSA_ACCEL_ALG_SHA_256 are
enabled, so that calls to mbedtls_md_xxx() use the software
implementation and calls to psa_hash_xxx() use the hardware accelerator
via the PSA driver. And this is deliberate: there is a reason why
mbedtls_md_xxx() must not call the PSA driver.
In most cases, this is not desirable: if there's an accelerator, why not
use it? And we're working to allow getting rid of the software
implementation in more and more cases. Ultimately, PSA will be the only
crypto interface in Mbed TLS, so all interfaces to calculate SHA-256
will go through psa_hash_xxx() and therefore will dispatch the call to
the driver if there is one. This will be an API break, since it will
require calling psa_crypto_init() before performing any cryptography.
Currently we are planning to introduce this requirement in Mbed TLS 4.0.
But it is currently possible to have dual algorithm support, and I can
think of unusual cases where it might desirable.
Scenario 1, with accelerator drivers: there is a driver, but it can only
be used after some initialization. The application needs to use the
algorithm before calling psa_crypto_init(), so it calls the legacy
interface. After psa_crypto_init() has been called, the application
would like to use the driver as much as possible. A typical use case is
a bootloader which wants to verify a signature before initializing the
random generator, so it calls mbedtls_md_xxx() and mbedtls_rsa_xxx().
Scenario 1 is clearly desirable, and for that we have a planned
solution, which is staged initialization. The bootloader will be able to
(1) initialize drivers, (2) perform a hash calculation, (3) initialize
the keystore, (4) verify a signature, all without initializing the RNG.
We won't make psa_crypto_init() mandatory until this feature is implemented.
Scenario 2: with a cryptography service. This is a build of Mbed TLS
with MBEDTLS_PSA_CRYPTO_CLIENT, so all psa_xxx() calls call the service.
But, for some reason, there is also a local implementation of some
cryptography algorithms. So you can call mbedtls_md() to calculate a
hash even before the connection to the service has been established. Or
maybe you want to call mbedtls_md() for short messages and
psa_hash_compute() for long messages, because the crypto service has a
faster implementation but the communication overhead offsets the gain
for short messages.
We are currently working on improving support for PSA drivers, and in
particular, saving code size by eliminating more unnecessary code when a
driver is present, and increasing the set of APIs that benefit from
drivers. The obvious way to do that is to make all cryptography calls
(especially from X.509 and TLS) go via the PSA interface, but we can't
do that yet due to the need to have initialized the keystore and RNG. We
are considering routing /certain/ crypto calls via PSA, in a way that
would break scenario 2, and would also break scenario 1 in some cases,
but not for hashes or signature verification.
If, for example, in Mbed TLS 3.4, mbedtls_md() starts calling
psa_hash_(), would this break your code? Are you in scenario 1, scenario
2, or some other variant I haven't thought of?
If so, *please reply to this message and let us know what your needs
are*. Feel free to reply to me in private if you don't want to discuss
this publicly (I won't share directly outside Arm, but the eventual
design might leak information about the unusual scenario).
If we don't hear objections, there is a chance that a future Mbed TLS
3.x will break scenarios 1 and 2. If we do hear objections, we'll work
to keep the current behavior or arrange a migration path.
Best regards,
--
Gilles Peskine
Mbed TLS developer
I have port mbedtls 2.28.0 into my platform and am able to connect to a few websites in PKI mode. But after enabling MBEDTLS_USE_PSA_CRYPTO & MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG, with everything else stay the same, I can't connect to the same websites anymore. in the working case (no MBEDTLS_USE_PSA_CRYPTO), after the client (my app) sends "Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message", the server responds with "Change Cipher Spec, Encrypted Handshake Message". In the broken case (with MBEDTLS_USE_PSA_CRYPTO), after the client (my app) sends "Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message", the server doesn't respond with "Change Cipher Spec, Encrypted Handshake Message". I don't know how to debug the issue, any suggestion what my cause the server to drop off at the last step?
I may need TLS 1.3 support which I believe arrived in 2.28, or maybe a
bit later.
I don't want to change to TLS 3 just yet. It looks like many changes.
My target is "OK" on FLASH (150k of the 350k total code size) but is
tight on RAM (after allocating 50k for the MbedTLS heap, we have just
20k RAM left).
Ideally I would like the very last version of v2.
The problem just found is that Cloudflare is asking for TLS 1.3 which
MbedTLS 2.16 does not support. But it may be that Cloudflare can fall
back and the problem is elsewhere.
Many thanks for any input.
Hi,
This is an updated post from https://github.com/Mbed-TLS/mbedtls/issues/6464,
which should be posted in mbedtls mail list.
My question is how to significantly improve SHA256 performance on big files
(regardless of architectures).
*=== Updates*
I use same code with mbedtls-3.1.0 to run tests in x86, and performance is
still downgraded.
Mbed TLS version (number or commit id): *3.1.0*
Operating system and version: * Centos-8.5, CPU 11900K*
Configuration (if not default, please attach mbedtls_config.h):
Compiler and options (if you used a pre-built binary, please indicate how
you obtained it): *gcc/g++ 8.5*
Additional environment information:
*Test files and performance*
CentOS-8.5.2111-x86_64-boot.iso (827.3 MB): sha256 *5 sec*
CentOS-8.5.2111-x86_64-boot.iso (10.79 GB): sha256 *66 sec*
Also, as advised I try to turn on "MBEDTLS_SHA256_USE_A64_CRYPTO_IF_PRESENT
" and "MBEDTLS_SHA512_USE_A64_CRYPTO_IF_PRESENT" using mbedtls-3.2.0 in M1,
but compiler reported the following error:
CMake Error at library/CMakeLists.txt:257 (add_library):
Cannot find source file:
psa_crypto_driver_wrappers.c
Tried extensions .c .C .c++ .cc .cpp .cxx .cu .mpp .m .M .mm .ixx .cppm .h
.hh .h++ .hm .hpp .hxx .in .txx .f .F .for .f77 .f90 .f95 .f03 .hip .ispc
CMake Error at library/CMakeLists.txt:257 (add_library):
No SOURCES given to target: mbedcrypto
Thanks for your help.
*=== Original message at github*
Summary
sha256() and sha1() incurs significant overhead on big files(~1G above). *This
might not be an issue*, and I'm looking for an efficient way to calculate
hash on big files.
System information
Mbed TLS version (number or commit id): 3.1.0
Operating system and version: M1 OSX
Configuration (if not default, please attach mbedtls_config.h):
Compiler and options (if you used a pre-built binary, please indicate how
you obtained it): Clang++
Additional environment information:
Expected behavior
Fast calculation of big files in less than 1 second
Actual behavior
Test files:
CentOS-8.5.2111-x86_64-boot.iso (827.3 MB): sha1 *3.3 sec*, sha256 *5.9
sec*
CentOS-8.5.2111-x86_64-boot.iso (10.79 GB): sha1 *40 sec*, sha256 *78
sec*
Steps to reproduce
ISO files can be downloaded at:
http://ftp.iij.ad.jp/pub/linux/centos-vault/8.5.2111/isos/x86_64/
Make sure use fast disk, say nvme, to store ISO files, or else loading big
files could take lots of time. Also use user from time command to measure
performance.
Workable code of sha256:
string test_sha256(string file_path)
{
mbedtls_sha256_context ctx;
FILE *fp;
string output;
int BUFFER_SIZE = 4096;
uint8_t buffer[BUFFER_SIZE];
size_t read, k_bytes;
uint8_t hash[32];
mbedtls_sha256_init(&ctx);
mbedtls_sha256_starts(&ctx, 0);
fp = fopen(file_path.c_str(), "r");
if (fp == NULL)
{
mbedtls_sha256_free(&ctx);
return output;
}
while ((read = fread(buffer, 1, BUFFER_SIZE, fp)))
{
mbedtls_sha256_update(&ctx, buffer, read);
}
mbedtls_sha256_finish(&ctx, hash);
mbedtls_sha256_free(&ctx);
fclose(fp);
// update hash string, omit here
return output;
}
Hi All,
There are machines out there for testing servers but I don't know of
one which can be used for testing a client.
This is a tricky area. For example I have a board running, LWIP and
MbedTLS, uploading little test files to two sites.
One was running EC and AES256. It worked fine.
The other was running RSA and AES256 but didn't work, and after some
work it was found that its certificate chain was running SHA-1 on the
top level certificate, dated 2006. This is actually a major name on
the internet! And we didn't have SHA-1 enabled because it is supposed
to be deprecated.
I wonder if there is some practical way to test out all this. We can
probably enable all the MbedTLS crypto options (TLS is taking up 150k
out of 350k of code for the whole product, but we can probably throw
in some more) but testing them is something else.
There is a test suite in TLS but it needs to be embedded in the
product itself. Has someone implemented that code on a server
somewhere?
Thank you in advance for any pointers.
Peter
I meet a problem when I call function `psa_crypto_init`, it return error code -148 that was PSA_ERROR_INSUFFICIENT_ENTROPY.
I track this function step by step and found it caused by MBEDTLS_ERR_ENTROPY_NO_SOURCES_DEFINED. Code in entropy.c, if( ctx->source_count == 0 ), return this error.
My question:
I run code on Ubuntu, it runs well. But in some arm board, it returns this error. Why this count will be 0 sometimes? What is the root cause of this error.
Hi All,
A gentle reminder that the US-Europe timezone-friendly MBed TLS Tech forum
is next Monday at 4:30 PM UK time. Invite details can be found on the
online calendar here <https://www.trustedfirmware.org/meetings/>.
If you have any topics, please let Dave Rodgman know. :)
Best regards,
Don Harbin
TrustedFirmware Community Manager
don.harbin(a)linaro.org
Hi,
I have developed a TLS and EST client application. Where the EST server issues the certificate in PKCS#7 format. It seems mbedtls library not support certificates in PKCS#7 format.
Is there any plan to support PKCS#7 in future?
Thanks,
Gopi Krishnan
Hi All,
A gentle reminder that the Asia-Europe timezone-friendly MBed TLS Tech
forum is next *Monday, Oct 10 at 10:00am UK time*. Invite details can be
found on the online calendar here
<https://www.trustedfirmware.org/meetings/>.
If you have any topics, please let Dave Rodgman, cc'd, know. :)
Best regards,
Don Harbin
TrustedFirmware Community Manager
don.harbin(a)linaro.org
