Hello,

I don't see anything obviously wrong with the code. Heap fragmentation is a possibility. A memory leak is also a possibility; we do fairly extensive testing for memory leaks in unit tests, but this doesn't catch unusual conditions (especially not recovering after a low-memory condition).

I think the first tool to investigate is to enable MBEDTLS_MEMORY_DEBUG. This has a small cost in code size and gives you access to some introspection functions mbedtls_memory_buffer_alloc_status() and mbedtls_memory_buffer_alloc_max_get(). Call these functions on an error, and also perhaps at other times for comparison.

There's also an option MBEDTLS_MEMORY_BACKTRACE which creates extremely verbose logs. Those logs might be helpful, but they're so verbose that they often aren't practical in a real-world application.

I notice that the allocation is for a little over 16kB. The default size of the SSL input/output buffers is 16kB because that's the maximum size of a message according to the specification. However you can usually get away with a lot less, especially on IoT networks where the infrastructure is geared towards much smaller messages. See https://tls.mbed.org/kb/how-to/controlling_package_size for more information on message sizes and buffer sizes. If the problem is a memory leak, smaller buffers will only delay the failure. But if the problem is that the application just needs a bit more heap space, this could solve the problem.

Concretely, try setting MBEDTLS_SSL_IN_CONTENT_LEN and MBEDTLS_SSL_OUT_CONTENT_LEN in the compile-time configuration to much lower values. MBEDTLS_SSL_OUT_CONTENT_LEN can be as low as you like as long as the handshake messages fit. MBEDTLS_SSL_IN_CONTENT_LEN has to be large enough for the messages that you're sent.

Alternatively, enable the option MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH, which results in smaller buffers but can make fragmentation worse due to reallocations.

Hope this helps.

--
Gilles Peskine
Mbed TLS developer

On 01/08/2021 22:40, Alan Chen via mbed-tls wrote:

I posted the question on the mbedTLS forum, only to realized that mbedTLS is now maintained by the project�s mailing list. Here is the copy of what I wrote:

 

*Occasionally*  I am getting MBEDTLS_ERR_SSL_ALLOC_FAILED from mbedtls_ssl_setup() during repeated HTTP partial content download. Since this problem happens very rarely, it is a bit difficult to troubleshoot.

 

I am running mbedTLS on a Microchip PIC32MZ MCU, connected to a LTE-M/NB-IoT modem. I have 128K static memory reserved for the library with MBEDTLS_PLATFORM_MEMORY defined in the config.h file. The MCU runs two main tasks - MQTT client, talking to the AWS MQTT broker, and HTTPS client, for downloading new firmware image from the AWS S3 bucket over the air.

 

Due to the slowness and limited bandwidth of the LTE-M and NB-IoT technologies, the HTTP file download has to use the partial content GET, basically 2KB per request, until all ~700KB of data are received. During the course of the file download, one can see as many as 30 disconnect and reconnect, and each time the TLS session would close down and re-open once the cell network is established. Here are some of my functions:

 

```

#define MBEDTLS_MAX_MEMORY_ALLOCATED    (1024 * 128)

static uint8_t tls_memory_buf[MBEDTLS_MAX_MEMORY_ALLOCATED];

 

// called in main()

void mbedtls_mem_init(void)

{

    mbedtls_memory_buffer_alloc_init(tls_memory_buf, sizeof tls_memory_buf);

}

 

void HTTPS_TLS_CLOSE(void)

{

    if (server_fd_https.fd != -1)

    {       

        mbedtls_entropy_free(&entropy_https);

        mbedtls_x509_crt_free(&cacert_https);

        mbedtls_ctr_drbg_free(&ctr_drbg_https);

        mbedtls_ssl_config_free(&conf_https);

        mbedtls_ssl_free(&ssl_https);

 

        server_fd_https.fd = -1;

    }

}

 

bool HTTPS_TLS_OPEN(void)

{

    int ret;

    const char *pers = "https_tls_wrapper";

 

 

    server_fd_https.fd = 1;

    mbedtls_debug_set_threshold(1);

 

    mbedtls_ssl_init(&ssl_https);

    mbedtls_ssl_config_init(&conf_https);

    mbedtls_ctr_drbg_init(&ctr_drbg_https);

    mbedtls_x509_crt_init(&cacert_https);

   

    mbedtls_entropy_init(&entropy_https);   

    mbedtls_entropy_add_source(&entropy_https, my_https_entropy, NULL, sizeof my_https_random, MBEDTLS_ENTROPY_SOURCE_STRONG);

   

    ret = mbedtls_ctr_drbg_seed(&ctr_drbg_https, mbedtls_entropy_func, &entropy_https, (const unsigned char *)pers, strlen(pers));

    if (ret != 0)

    {

        printf("%s: mbedtls_ctr_drbg_seed ERROR -0x%x\r\n", __FUNCTION__, -ret);

        return false;

    }

 

    ret = mbedtls_x509_crt_parse(&cacert_https, TRUSTED_ROOT_CA, TRUSTED_ROOT_CA_SIZE);

    if (ret != 0)

    {

        printf("%s: mbedtls_x509_crt_parse cacert ERROR -0x%x\r\n", __FUNCTION__, -ret);

        return false;

    }

   

    ret = mbedtls_ssl_config_defaults(&conf_https, MBEDTLS_SSL_IS_CLIENT, MBEDTLS_SSL_TRANSPORT_STREAM, MBEDTLS_SSL_PRESET_DEFAULT);

    if (ret != 0)

    {

        printf("%s: mbedtls_ssl_config_defaults ERROR -0x%x\r\n", __FUNCTION__, -ret);

        return false;

    }

 

    mbedtls_ssl_conf_verify(&conf_https, NULL, NULL);

    mbedtls_ssl_conf_authmode(&conf_https, MBEDTLS_SSL_VERIFY_REQUIRED);

    mbedtls_ssl_conf_rng(&conf_https, mbedtls_ctr_drbg_random, &ctr_drbg_https);

    mbedtls_ssl_conf_dbg(&conf_https, my_https_debug, stdout);

    mbedtls_ssl_conf_ca_chain(&conf_https, &cacert_https, NULL);

       

    mbedtls_ssl_conf_read_timeout(&conf_https, TLS_TIMEOUT_MS);

 

    HTTPS_SetHostname(); /* calling mbedtls_ssl_set_hostname */

   

    ret = mbedtls_ssl_setup(&ssl_https, &conf_https);

    if (ret != 0)

    {

        printf("%s: mbedtls_ssl_setup ERROR -0x%x\r\n", __FUNCTION__, -ret);

        return false;

    }

 

    mbedtls_ssl_set_bio(&ssl_https, &server_fd_https, mbedtls_https_send, mbedtls_https_recv, NULL);

 

    return true;

}

```

Can someone please tell me if I am doing something inappropriate here? I am speculating that perhaps there is a memory leak or the heap becomes so fragmented that it fails on mbedtls_calloc(). The exact error message in my case is:

 

> ../mbedtls_lib/ssl_tls.c:5661: alloc(16717 bytes) failed

 

Thanks.

 

Alan Chen



Scanned by McAfee and confirmed virus-free.