(Note: I'm replying to the list. Please keep list conversations on the list.)

Writing memory allocators is a specialty. It's not my specialty, and apparently it's not your specialty either, so if we design a memory allocator we're likely to end up with poor performance. On most systems, the standard library malloc is designed by experts, and often fine-tuned for the type of platform and usage (e.g. large or small pools, with or without MMU, with or without cache).

memory_buffer_alloc uses an array to implement malloc.

One possible explanation for poor performance is if your allocator doesn't align data nicely. Some platforms supports unaligned accesses (e.g. accessing a 4-byte value at an address that isn't a multiple of 4) but with a significant performance penalty (most others just crash). If there's a cache, alignment with cache line boundaries can also help. Another potential reason in a multithreaded program is not doing synchronization efficiently, or doing it too often. Note that you do NOT need synchronization if each thread has its own memory pool and no thread ever modifies the data of another thread. Note, however, that Mbed TLS caches some data in public/private key objects, so doing an operation with a key modifies the key object, therefore each thread would need to have its own copy of the key. There are undoubtedly other likely reasons that I'm not thinking of.

Best regards,

--
Gilles Peskine
Mbed TLS developer

On 29/07/2021 22:38, Shariful Alam wrote:
Hi Gilles,
Hope you are well. The project that I'm currently working on requires static memory allocation instead of dynamic memory allocation. I was trying to use  "memory_buffer_alloc()". Since memory_buffer_alloc() doesn't support multiple memory pools, seems like I can't use this function in my project. 

I have a working version of a modified bignum.c & bignum.h, where malloc is replaced with an array. The library is working. However, the performance is poor (~4 times slower). I was wondering what could cause this slow performance. I mean, I understand it will be slow, but I did not expect it to be 4 times slower. 

Is there any version, where an array was used instead of malloc? Or if you could point out some of the reasons for this library to slow down while using an array, I will be very grateful.

Best regards,
Shariful
   

On Tue, Jul 20, 2021 at 2:22 PM Gilles Peskine <gilles.peskine@arm.com> wrote:
Hi Shariful,

You just call mbedtls_calloc() (or let other functions call it). It will use the memory pool set by mbedtls_memory_buffer_alloc_init().

The memory_buffer_alloc module does not support multiple memory pools. If you want a separate pool for each thread's allocation for performance reasons on a multicore system, you'd better rely on your platform's built-in calloc(). It's likely to be fined-tuned for multicore operation. The built-in allocator in Mbed TLS is intended for highly resource-constrained systems where the basic platform doesn't even include an allocator.

Best regards,

--
Gilles Peskine
Mbed TLS developer

On 20/07/2021 22:10, Shariful Alam wrote:
Hi Gilles,
Thank you very much, for your reply. Sorry to bother you again. I am trying to follow your instructions. I have a question regarding your suggestions.

You said "applications call mbedtls_memory_buffer_alloc_init() in the startup code of the initial thread, before creating other threads. The alloc/free functions are thread-safe, but the initialization and deinitialization functions aren't." 

I'm a little confused here. Say, if I call mbedtls_memory_buffer_alloc_init() with a buffer in the main thread, how did all other threads use this memory (or how do all the threads know which memory block to use)?

After calling mbedtls_memory_buffer_alloc_init() in the main thread, I can get the address of the buffer and pass it to the threads, do I have to call mbedtls_memory_buffer_alloc_init() again inside each thread?   

Thanks,
Shariful

On Mon, Jul 19, 2021 at 3:48 AM Gilles Peskine via mbed-tls <mbed-tls@lists.trustedfirmware.org> wrote:
Hi Shariful,

First, please note that the library called PolarSSL with functions like
rsa_private() and memory_buffer_alloc_init() has not been supported for
several years. You should upgrade to Mbed TLS, with functions like
mbedtls_rsa_private() and mbedtls_memory_buffer_alloc_init(). That being
said, the memory_buffer_alloc module works in the same way.

Normally, applications call mbedtls_memory_buffer_alloc_init() in the
startup code of the initial thread, before creating other threads. The
alloc/free functions are thread-safe, but the initialization and
deinitialization functions aren't. If you must call
mbedtls_memory_buffer_alloc_init() after creating other threads, make
sure that no thread calls mbedtls_calloc until
mbedtls_memory_buffer_alloc_init() has returned.

The same principle applies to other parts of Mbed TLS that are
thread-safe. For example, only the RSA operations (encryption,
decryption, signature, verification, and also the low-level functions
mbedtls_rsa_public() and mbedtls_rsa_private()) are protected. So you
must finish setting up the RSA key inside one thread before you pass a
pointer to other threads. Similarly, only mbedtls_xxx_drbg_random() is
thread-safe, and the RNG setup (including mbedtls_xxx_drgb_seed())
should be done as part of the initial application startup.

Finally, note that mbedtls_rsa_private() alone cannot decrypt a message:
all it does it to apply the private key operation. To decrypt a simple
message encrypted with RSA-OAEP, call mbedtls_rsa_rsaes_oaep_decrypt()
or mbedtls_rsa_pkcs1_decrypt() with a key set up for
MBEDTLS_RSA_PKCS_V21 encoding. To use the legacy PKCS#1v1.5 mechanism,
call mbedtls_rsa_rsaes_pkcs1_v15_decrypt() or
mbedtls_rsa_pkcs1_decrypt() with a key set up for .MBEDTLS_RSA_PKCS_V15.
To decrypt a message using a RSA FDH hybrid scheme, you do need to call
mbedtls_rsa_private() since Mbed TLS doesn't support it natively, but
what this gives you is the intermediate secret from which you then need
to derive a symmetric key, not the message itself.

Best regards,

--
Gilles Peskine
Mbed TLS developer

On 18/07/2021 07:16, Shariful Alam via mbed-tls wrote:
> Hello,
> I have a simple example code to decrypt an encrypted message using
> *rsa_private()*. I use *memory_buffer_alloc_init(),  *in order to use
> a static memory for the computation. I want to run my code
> concurrently. My code works with a single pthread. However, when I try
> to run more than one thread my program fails to decrypt. 
>
> ** I check the same code without *memory_buffer_alloc_init(), *it
> works concurrently, without any issues at all. 
>
> Therefore, I believe, the issue that I'm facing is coming from the use
> of static memory(e.g. *memory_buffer_alloc_init()*). The documentation
> of memorry_buffer_alloc.h shows,
>
>     /** 
>
>     * \brief   Initialize use of stack-based memory allocator. 
>
>     *           The stack-based allocator does memory management
>     inside the 
>
>     *          presented buffer and does not call malloc() and free(). 
>
>     *          It sets the global polarssl_malloc() and
>     polarssl_free() pointers 
>
>     *          to its own functions. 
>
>     *          (Provided polarssl_malloc() and polarssl_free() are
>     thread-safe if 
>
>     *           POLARSSL_THREADING_C is defined)
>
>     * 
>
>     * \note    This code is not optimized and provides a straight-forward 
>
>     *          implementation of a stack-based memory allocator. 
>
>     * 
>
>     * \param buf   buffer to use as heap 
>
>     * \param len   size of the buffer 
>
>     * 
>
>     * \return              0 if successful 
>
>     */   
>
>
> So, I added the following configuration to the *config.h* file  
>
>     1. #define POLARSSL_THREADING_PTHREAD
>     2. #define POLARSSL_THREADING_C
>
> But I'm still getting errors while decrypting. Any help on how to fix
> this? or what else should I add into the config.h file to
> make *memory_buffer_alloc_init() *thread-safe? Here is my sample
> code: https://pastebin.com/uyW3vknt <https://pastebin.com/uyW3vknt>
>
> Thanks,
> Shariful
>

--
mbed-tls mailing list
mbed-tls@lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/mbed-tls