I have ported mbedtls library on am embedded platform developed to encrypt / decrypt messages using AES GCM 256 key. After several hours of running, we are experiencing error MBEDTLS_ERR_CIPHER_ALLOC_FAILED 0x6180 and malloc functions fails as the heap seems to be piled-up.
How could I is using correct free function and the actual allocated memory is freed?
Thanks, Gopi Krishnan
Please make sure that you're correctly freeing all contexts. In particular, note that each call to malloc() or calloc() must have a corresponding call to free(), AND each call to mbedtls_cipher_init() must have a corresponding call to mbedtls_cipher_free(). Despite the name, mbedtls_xxx_free() functions do not free the memory used by their argument itself (and it can be on the stack or a global variable), they only free resources used by their argument.
If that's not the problem, I recommend doing static analysis or doing stress tests under a memory allocation framework. For Mbed TLS, we do static analysis with Coverity (expensive but nice), and runtime testing with AddressSanitizer and Valgrind.