- mbed-tls - lists.trustedfirmware.org

Could memory leak or heap fragmentation be the issue?

by Alan Chen

I posted the question on the mbedTLS forum, only to realized that mbedTLS is now maintained by the project's mailing list. Here is the copy of what I wrote: *Occasionally* I am getting MBEDTLS_ERR_SSL_ALLOC_FAILED from mbedtls_ssl_setup() during repeated HTTP partial content download. Since this problem happens very rarely, it is a bit difficult to troubleshoot. I am running mbedTLS on a Microchip PIC32MZ MCU, connected to a LTE-M/NB-IoT modem. I have 128K static memory reserved for the library with MBEDTLS_PLATFORM_MEMORY defined in the config.h file. The MCU runs two main tasks - MQTT client, talking to the AWS MQTT broker, and HTTPS client, for downloading new firmware image from the AWS S3 bucket over the air. Due to the slowness and limited bandwidth of the LTE-M and NB-IoT technologies, the HTTP file download has to use the partial content GET, basically 2KB per request, until all ~700KB of data are received. During the course of the file download, one can see as many as 30 disconnect and reconnect, and each time the TLS session would close down and re-open once the cell network is established. Here are some of my functions: ``` #define MBEDTLS_MAX_MEMORY_ALLOCATED (1024 * 128) static uint8_t tls_memory_buf[MBEDTLS_MAX_MEMORY_ALLOCATED]; // called in main() void mbedtls_mem_init(void) { mbedtls_memory_buffer_alloc_init(tls_memory_buf, sizeof tls_memory_buf); } void HTTPS_TLS_CLOSE(void) { if (server_fd_https.fd != -1) { mbedtls_entropy_free(&entropy_https); mbedtls_x509_crt_free(&cacert_https); mbedtls_ctr_drbg_free(&ctr_drbg_https); mbedtls_ssl_config_free(&conf_https); mbedtls_ssl_free(&ssl_https); server_fd_https.fd = -1; } } bool HTTPS_TLS_OPEN(void) { int ret; const char *pers = "https_tls_wrapper"; server_fd_https.fd = 1; mbedtls_debug_set_threshold(1); mbedtls_ssl_init(&ssl_https); mbedtls_ssl_config_init(&conf_https); mbedtls_ctr_drbg_init(&ctr_drbg_https); mbedtls_x509_crt_init(&cacert_https); mbedtls_entropy_init(&entropy_https); mbedtls_entropy_add_source(&entropy_https, my_https_entropy, NULL, sizeof my_https_random, MBEDTLS_ENTROPY_SOURCE_STRONG); ret = mbedtls_ctr_drbg_seed(&ctr_drbg_https, mbedtls_entropy_func, &entropy_https, (const unsigned char *)pers, strlen(pers)); if (ret != 0) { printf("%s: mbedtls_ctr_drbg_seed ERROR -0x%x\r\n", __FUNCTION__, -ret); return false; } ret = mbedtls_x509_crt_parse(&cacert_https, TRUSTED_ROOT_CA, TRUSTED_ROOT_CA_SIZE); if (ret != 0) { printf("%s: mbedtls_x509_crt_parse cacert ERROR -0x%x\r\n", __FUNCTION__, -ret); return false; } ret = mbedtls_ssl_config_defaults(&conf_https, MBEDTLS_SSL_IS_CLIENT, MBEDTLS_SSL_TRANSPORT_STREAM, MBEDTLS_SSL_PRESET_DEFAULT); if (ret != 0) { printf("%s: mbedtls_ssl_config_defaults ERROR -0x%x\r\n", __FUNCTION__, -ret); return false; } mbedtls_ssl_conf_verify(&conf_https, NULL, NULL); mbedtls_ssl_conf_authmode(&conf_https, MBEDTLS_SSL_VERIFY_REQUIRED); mbedtls_ssl_conf_rng(&conf_https, mbedtls_ctr_drbg_random, &ctr_drbg_https); mbedtls_ssl_conf_dbg(&conf_https, my_https_debug, stdout); mbedtls_ssl_conf_ca_chain(&conf_https, &cacert_https, NULL); mbedtls_ssl_conf_read_timeout(&conf_https, TLS_TIMEOUT_MS); HTTPS_SetHostname(); /* calling mbedtls_ssl_set_hostname */ ret = mbedtls_ssl_setup(&ssl_https, &conf_https); if (ret != 0) { printf("%s: mbedtls_ssl_setup ERROR -0x%x\r\n", __FUNCTION__, -ret); return false; } mbedtls_ssl_set_bio(&ssl_https, &server_fd_https, mbedtls_https_send, mbedtls_https_recv, NULL); return true; } ``` Can someone please tell me if I am doing something inappropriate here? I am speculating that perhaps there is a memory leak or the heap becomes so fragmented that it fails on mbedtls_calloc(). The exact error message in my case is: > ../mbedtls_lib/ssl_tls.c:5661: alloc(16717 bytes) failed Thanks. Alan Chen ________________________________ [https://secureimages.mcafee.com/common/affiliateImages/mfe/logo.png]<https://home.mcafee.com/utm_medium=email&utm_source=link&utm_campaign=sig-e…> Scanned by McAfee<https://home.mcafee.com/utm_medium=email&utm_source=link&utm_campaign=sig-e…> and confirmed virus-free.

5 years

1
0
0 0

Re: [mbed-tls] Mbed TLS: long term support versions

by Dave Rodgman

Sorry, just realised that myself! Gilles is correct, I should have said 2.28. Thanks Dave On 29/07/2021, 14:25, "mbed-tls on behalf of Gilles Peskine via mbed-tls" <mbed-tls-bounces(a)lists.trustedfirmware.org on behalf of mbed-tls(a)lists.trustedfirmware.org> wrote: Off-by-one error! The current 2.x release is 2.27.0. Most development work is happening on 3.x but there will be at least one more 2.x release: 2.28.0. The last 2.x release will become an LTS. -- Gilles Peskine Mbed TLS developer On 29/07/2021 15:05, Dave Rodgman via mbed-tls wrote: > > Hi Matteo, > > > > We expect to release an LTS later this year. It’s likely to be 2.27, > and very likely will be supported for the usual LTS period of 3 years. > > > > So if you are considering updating to a new LTS, you could use 2.26 > for prototyping in the short term until the LTS becomes available. The > upcoming LTS will be API-compatible with 2.26. > > > > Hope this helps, > > > > Dave > > > > *From: *mbed-tls <mbed-tls-bounces(a)lists.trustedfirmware.org> on > behalf of "matteo.cogi--- via mbed-tls" > <mbed-tls(a)lists.trustedfirmware.org> > *Reply to: *"matteo.cogi(a)alice.it" <matteo.cogi(a)alice.it> > *Date: *Tuesday, 27 July 2021 at 07:44 > *To: *"mbed-tls(a)lists.trustedfirmware.org" > <mbed-tls(a)lists.trustedfirmware.org> > *Subject: *[mbed-tls] Mbed TLS: long term support versions > > > > Dear all, > I wish to know which are the future LTS (Long Term Support) versions > for Mbed TLS. > In these last years I have been working with the 2.16.x, but I read it > will be maintained until at least the end of 2021 ( > https://github.com/ARMmbed/mbedtls/blob/development/BRANCHES.md#current-bra… > <https://github.com/ARMmbed/mbedtls/blob/development/BRANCHES.md#current-bra…> > ), so I am considering an update to a newer LTS version. > However I don’t find which are the next LTS version and for how much > time they will be maintained. > Thanks in advance. > Regards, > Matteo > > -- mbed-tls mailing list mbed-tls(a)lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/mbed-tls

5 years

1
0
0 0

Re: [mbed-tls] Mbed TLS: long term support versions

by Gilles Peskine

Off-by-one error! The current 2.x release is 2.27.0. Most development work is happening on 3.x but there will be at least one more 2.x release: 2.28.0. The last 2.x release will become an LTS. -- Gilles Peskine Mbed TLS developer On 29/07/2021 15:05, Dave Rodgman via mbed-tls wrote: > > Hi Matteo, > > > > We expect to release an LTS later this year. It’s likely to be 2.27, > and very likely will be supported for the usual LTS period of 3 years. > > > > So if you are considering updating to a new LTS, you could use 2.26 > for prototyping in the short term until the LTS becomes available. The > upcoming LTS will be API-compatible with 2.26. > > > > Hope this helps, > > > > Dave > > > > *From: *mbed-tls <mbed-tls-bounces(a)lists.trustedfirmware.org> on > behalf of "matteo.cogi--- via mbed-tls" > <mbed-tls(a)lists.trustedfirmware.org> > *Reply to: *"matteo.cogi(a)alice.it" <matteo.cogi(a)alice.it> > *Date: *Tuesday, 27 July 2021 at 07:44 > *To: *"mbed-tls(a)lists.trustedfirmware.org" > <mbed-tls(a)lists.trustedfirmware.org> > *Subject: *[mbed-tls] Mbed TLS: long term support versions > > > > Dear all, > I wish to know which are the future LTS (Long Term Support) versions > for Mbed TLS. > In these last years I have been working with the 2.16.x, but I read it > will be maintained until at least the end of 2021 ( > https://github.com/ARMmbed/mbedtls/blob/development/BRANCHES.md#current-bra… > <https://github.com/ARMmbed/mbedtls/blob/development/BRANCHES.md#current-bra…> > ), so I am considering an update to a newer LTS version. > However I don’t find which are the next LTS version and for how much > time they will be maintained. > Thanks in advance. > Regards, > Matteo > >

5 years

1
0
0 0

Re: [mbed-tls] Mbed TLS: long term support versions

by Dave Rodgman

Hi Matteo, We expect to release an LTS later this year. It’s likely to be 2.27, and very likely will be supported for the usual LTS period of 3 years. So if you are considering updating to a new LTS, you could use 2.26 for prototyping in the short term until the LTS becomes available. The upcoming LTS will be API-compatible with 2.26. Hope this helps, Dave From: mbed-tls <mbed-tls-bounces(a)lists.trustedfirmware.org> on behalf of "matteo.cogi--- via mbed-tls" <mbed-tls(a)lists.trustedfirmware.org> Reply to: "matteo.cogi(a)alice.it" <matteo.cogi(a)alice.it> Date: Tuesday, 27 July 2021 at 07:44 To: "mbed-tls(a)lists.trustedfirmware.org" <mbed-tls(a)lists.trustedfirmware.org> Subject: [mbed-tls] Mbed TLS: long term support versions Dear all, I wish to know which are the future LTS (Long Term Support) versions for Mbed TLS. In these last years I have been working with the 2.16.x, but I read it will be maintained until at least the end of 2021 ( https://github.com/ARMmbed/mbedtls/blob/development/BRANCHES.md#current-bra… ), so I am considering an update to a newer LTS version. However I don’t find which are the next LTS version and for how much time they will be maintained. Thanks in advance. Regards, Matteo

5 years

1
0
0 0

Mbed TLS: long term support versions

by matteo.cogi＠alice.it

Dear all, I wish to know which are the future LTS (Long Term Support) versions for Mbed TLS. In these last years I have been working with the 2.16.x, but I read it will be maintained until at least the end of 2021 ( https://github.com/ARMmbed/mbedtls/blob/development/BRANCHES.md#current-bra… <https://github.com/ARMmbed/mbedtls/blob/development/BRANCHES.md#current-bra…> ), so I am considering an update to a newer LTS version. However I don’t find which are the next LTS version and for how much time they will be maintained. Thanks in advance. Regards, Matteo

5 years

1
0
0 0

Re: [mbed-tls] May I know why SHA224 is mandatory with SH256?

by Gilles Peskine

Hello, Mbed TLS has never supported a build with SHA-256 but not SHA-224. In Mbed TLS 2.x, enabling MBEDTLS_SHA256_C enables both SHA-256 and SHA-224. Likewise, MBEDTLS_SHA512_C enables both SHA-512 and SHA-384. The reason for this design is that SHA-256 and SHA-224 have essentially the same code but different constants, and likewise for SHA-512 and SHA-384. What changed in Mbed TLS 3.0 is that there are now separate configuration options for each of the four SHA2 variants. It is not possible yet to enable SHA-384 without SHA-512, SHA-224 without SHA-256 or SHA-256 without SHA-224. These are implementation limitations due to missing #ifdef in various places. We expect to lift these limitations in one of the next 3.x releases. Best regards, -- Gilles Peskine Mbed TLS developer On 19/07/2021 14:50, David Hu via mbed-tls wrote: > > Hi, > > ï¿½ > > It seems that SHA224 is mandatory if SHA256 is selected, in Mbed TLS > latest version, according to this new check below: > > ï¿½ > > #if defined(MBEDTLS_SHA256_C) && !defined(MBEDTLS_SHA224_C) > > #error "MBEDTLS_SHA256_C defined without MBEDTLS_SHA224_C" > > #endif > > ï¿½ > > May I know why SHA224 must be enabled with SHA256? > > Could you please point me to any reference/document? > > ï¿½ > > Best regards, > > Hu Ziji > >

5 years

2
3
0 0

Re: [mbed-tls] How to setup config.h to make memory_buffer_alloc_init() thread-safe?

by Shariful Alam

Hi Gilles, Thank you very much, for your reply. Sorry to bother you again. I am trying to follow your instructions. I have a question regarding your suggestions. You said "applications call mbedtls_memory_buffer_alloc_init() in the startup code of the initial thread, before creating other threads. The alloc/free functions are thread-safe, but the initialization and deinitialization functions aren't." I'm a little confused here. Say, if I call mbedtls_memory_buffer_alloc_init() with a buffer in the main thread, how did all other threads use this memory (or how do all the threads know which memory block to use)? After calling mbedtls_memory_buffer_alloc_init() in the main thread, I can get the address of the buffer and pass it to the threads, do I have to call mbedtls_memory_buffer_alloc_init() again inside each thread? Thanks, Shariful On Mon, Jul 19, 2021 at 3:48 AM Gilles Peskine via mbed-tls < mbed-tls(a)lists.trustedfirmware.org> wrote: > Hi Shariful, > > First, please note that the library called PolarSSL with functions like > rsa_private() and memory_buffer_alloc_init() has not been supported for > several years. You should upgrade to Mbed TLS, with functions like > mbedtls_rsa_private() and mbedtls_memory_buffer_alloc_init(). That being > said, the memory_buffer_alloc module works in the same way. > > Normally, applications call mbedtls_memory_buffer_alloc_init() in the > startup code of the initial thread, before creating other threads. The > alloc/free functions are thread-safe, but the initialization and > deinitialization functions aren't. If you must call > mbedtls_memory_buffer_alloc_init() after creating other threads, make > sure that no thread calls mbedtls_calloc until > mbedtls_memory_buffer_alloc_init() has returned. > > The same principle applies to other parts of Mbed TLS that are > thread-safe. For example, only the RSA operations (encryption, > decryption, signature, verification, and also the low-level functions > mbedtls_rsa_public() and mbedtls_rsa_private()) are protected. So you > must finish setting up the RSA key inside one thread before you pass a > pointer to other threads. Similarly, only mbedtls_xxx_drbg_random() is > thread-safe, and the RNG setup (including mbedtls_xxx_drgb_seed()) > should be done as part of the initial application startup. > > Finally, note that mbedtls_rsa_private() alone cannot decrypt a message: > all it does it to apply the private key operation. To decrypt a simple > message encrypted with RSA-OAEP, call mbedtls_rsa_rsaes_oaep_decrypt() > or mbedtls_rsa_pkcs1_decrypt() with a key set up for > MBEDTLS_RSA_PKCS_V21 encoding. To use the legacy PKCS#1v1.5 mechanism, > call mbedtls_rsa_rsaes_pkcs1_v15_decrypt() or > mbedtls_rsa_pkcs1_decrypt() with a key set up for .MBEDTLS_RSA_PKCS_V15. > To decrypt a message using a RSA FDH hybrid scheme, you do need to call > mbedtls_rsa_private() since Mbed TLS doesn't support it natively, but > what this gives you is the intermediate secret from which you then need > to derive a symmetric key, not the message itself. > > Best regards, > > -- > Gilles Peskine > Mbed TLS developer > > On 18/07/2021 07:16, Shariful Alam via mbed-tls wrote: > > Hello, > > I have a simple example code to decrypt an encrypted message using > > *rsa_private()*. I use *memory_buffer_alloc_init(), *in order to use > > a static memory for the computation. I want to run my code > > concurrently. My code works with a single pthread. However, when I try > > to run more than one thread my program fails to decrypt. > > > > ** I check the same code without *memory_buffer_alloc_init(), *it > > works concurrently, without any issues at all. > > > > Therefore, I believe, the issue that I'm facing is coming from the use > > of static memory(e.g. *memory_buffer_alloc_init()*). The documentation > > of memorry_buffer_alloc.h shows, > > > > /** > > > > * \brief Initialize use of stack-based memory allocator. > > > > * The stack-based allocator does memory management > > inside the > > > > * presented buffer and does not call malloc() and free(). > > > > * It sets the global polarssl_malloc() and > > polarssl_free() pointers > > > > * to its own functions. > > > > * (Provided polarssl_malloc() and polarssl_free() are > > thread-safe if > > > > * POLARSSL_THREADING_C is defined) > > > > * > > > > * \note This code is not optimized and provides a > straight-forward > > > > * implementation of a stack-based memory allocator. > > > > * > > > > * \param buf buffer to use as heap > > > > * \param len size of the buffer > > > > * > > > > * \return 0 if successful > > > > */ > > > > > > So, I added the following configuration to the *config.h* file > > > > 1. #define POLARSSL_THREADING_PTHREAD > > 2. #define POLARSSL_THREADING_C > > > > But I'm still getting errors while decrypting. Any help on how to fix > > this? or what else should I add into the config.h file to > > make *memory_buffer_alloc_init() *thread-safe? Here is my sample > > code: https://pastebin.com/uyW3vknt <https://pastebin.com/uyW3vknt> > > > > Thanks, > > Shariful > > > > -- > mbed-tls mailing list > mbed-tls(a)lists.trustedfirmware.org > https://lists.trustedfirmware.org/mailman/listinfo/mbed-tls >

5 years

2
1
0 0

May I know why SHA224 is mandatory with SH256?

by David Hu

Hi, It seems that SHA224 is mandatory if SHA256 is selected, in Mbed TLS latest version, according to this new check below: #if defined(MBEDTLS_SHA256_C) && !defined(MBEDTLS_SHA224_C) #error "MBEDTLS_SHA256_C defined without MBEDTLS_SHA224_C" #endif May I know why SHA224 must be enabled with SHA256? Could you please point me to any reference/document? Best regards, Hu Ziji

5 years, 1 month

1
0
0 0

Re: [mbed-tls] How to setup config.h to make memory_buffer_alloc_init() thread-safe?

by Gilles Peskine

Hi Shariful, First, please note that the library called PolarSSL with functions like rsa_private() and memory_buffer_alloc_init() has not been supported for several years. You should upgrade to Mbed TLS, with functions like mbedtls_rsa_private() and mbedtls_memory_buffer_alloc_init(). That being said, the memory_buffer_alloc module works in the same way. Normally, applications call mbedtls_memory_buffer_alloc_init() in the startup code of the initial thread, before creating other threads. The alloc/free functions are thread-safe, but the initialization and deinitialization functions aren't. If you must call mbedtls_memory_buffer_alloc_init() after creating other threads, make sure that no thread calls mbedtls_calloc until mbedtls_memory_buffer_alloc_init() has returned. The same principle applies to other parts of Mbed TLS that are thread-safe. For example, only the RSA operations (encryption, decryption, signature, verification, and also the low-level functions mbedtls_rsa_public() and mbedtls_rsa_private()) are protected. So you must finish setting up the RSA key inside one thread before you pass a pointer to other threads. Similarly, only mbedtls_xxx_drbg_random() is thread-safe, and the RNG setup (including mbedtls_xxx_drgb_seed()) should be done as part of the initial application startup. Finally, note that mbedtls_rsa_private() alone cannot decrypt a message: all it does it to apply the private key operation. To decrypt a simple message encrypted with RSA-OAEP, call mbedtls_rsa_rsaes_oaep_decrypt() or mbedtls_rsa_pkcs1_decrypt() with a key set up for MBEDTLS_RSA_PKCS_V21 encoding. To use the legacy PKCS#1v1.5 mechanism, call mbedtls_rsa_rsaes_pkcs1_v15_decrypt() or mbedtls_rsa_pkcs1_decrypt() with a key set up for .MBEDTLS_RSA_PKCS_V15. To decrypt a message using a RSA FDH hybrid scheme, you do need to call mbedtls_rsa_private() since Mbed TLS doesn't support it natively, but what this gives you is the intermediate secret from which you then need to derive a symmetric key, not the message itself. Best regards, -- Gilles Peskine Mbed TLS developer On 18/07/2021 07:16, Shariful Alam via mbed-tls wrote: > Hello, > I have a simple example code to decrypt an encrypted message using > *rsa_private()*. I use *memory_buffer_alloc_init(), *in order to use > a static memory for the computation. I want to run my code > concurrently. My code works with a single pthread. However, when I try > to run more than one thread my program fails to decrypt. > > ** I check the same code without *memory_buffer_alloc_init(), *it > works concurrently, without any issues at all. > > Therefore, I believe, the issue that I'm facing is coming from the use > of static memory(e.g. *memory_buffer_alloc_init()*). The documentation > of memorry_buffer_alloc.h shows, > > /** > > * \brief Initialize use of stack-based memory allocator. > > * The stack-based allocator does memory management > inside the > > * presented buffer and does not call malloc() and free(). > > * It sets the global polarssl_malloc() and > polarssl_free() pointers > > * to its own functions. > > * (Provided polarssl_malloc() and polarssl_free() are > thread-safe if > > * POLARSSL_THREADING_C is defined) > > * > > * \note This code is not optimized and provides a straight-forward > > * implementation of a stack-based memory allocator. > > * > > * \param buf buffer to use as heap > > * \param len size of the buffer > > * > > * \return 0 if successful > > */ > > > So, I added the following configuration to the *config.h* file > > 1. #define POLARSSL_THREADING_PTHREAD > 2. #define POLARSSL_THREADING_C > > But I'm still getting errors while decrypting. Any help on how to fix > this? or what else should I add into the config.h file to > make *memory_buffer_alloc_init() *thread-safe? Here is my sample > code: https://pastebin.com/uyW3vknt <https://pastebin.com/uyW3vknt> > > Thanks, > Shariful >

5 years, 1 month

1
0
0 0

How to setup config.h to make memory_buffer_alloc_init() thread-safe?

by Shariful Alam

Hello, I have a simple example code to decrypt an encrypted message using *rsa_private()*. I use *memory_buffer_alloc_init(), *in order to use a static memory for the computation. I want to run my code concurrently. My code works with a single pthread. However, when I try to run more than one thread my program fails to decrypt. ** I check the same code without *memory_buffer_alloc_init(), *it works concurrently, without any issues at all. Therefore, I believe, the issue that I'm facing is coming from the use of static memory(e.g. *memory_buffer_alloc_init()*). The documentation of memorry_buffer_alloc.h shows, /** * \brief Initialize use of stack-based memory allocator. * The stack-based allocator does memory management inside the * presented buffer and does not call malloc() and free(). * It sets the global polarssl_malloc() and polarssl_free() > pointers * to its own functions. * (Provided polarssl_malloc() and polarssl_free() are thread-safe > if * POLARSSL_THREADING_C is defined) * * \note This code is not optimized and provides a straight-forward * implementation of a stack-based memory allocator. * * \param buf buffer to use as heap * \param len size of the buffer * * \return 0 if successful */ So, I added the following configuration to the *config.h* file 1. #define POLARSSL_THREADING_PTHREAD 2. #define POLARSSL_THREADING_C But I'm still getting errors while decrypting. Any help on how to fix this? or what else should I add into the config.h file to make *memory_buffer_alloc_init() *thread-safe? Here is my sample code: https://pastebin.com/uyW3vknt Thanks, Shariful

5 years, 1 month

1
1
0 0

2026

2025

2024

2023

2022

2021

2020

mbed-tls