(Note: I'm replying to the list. Please keep list conversations on the
list.)
Writing memory allocators is a specialty. It's not my specialty, and
apparently it's not your specialty either, so if we design a memory
allocator we're likely to end up with poor performance. On most systems,
the standard library malloc is designed by experts, and often fine-tuned
for the type of platform and usage (e.g. large or small pools, with or
without MMU, with or without cache).
memory_buffer_alloc uses an array to implement malloc.
One possible explanation for poor performance is if your allocator
doesn't align data nicely. Some platforms supports unaligned accesses
(e.g. accessing a 4-byte value at an address that isn't a multiple of 4)
but with a significant performance penalty (most others just crash). If
there's a cache, alignment with cache line boundaries can also help.
Another potential reason in a multithreaded program is not doing
synchronization efficiently, or doing it too often. Note that you do NOT
need synchronization if each thread has its own memory pool and no
thread ever modifies the data of another thread. Note, however, that
Mbed TLS caches some data in public/private key objects, so doing an
operation with a key modifies the key object, therefore each thread
would need to have its own copy of the key. There are undoubtedly other
likely reasons that I'm not thinking of.
--
Gilles Peskine
Mbed TLS developer
On 29/07/2021 22:38, Shariful Alam wrote:
> Hi Gilles,
> Hope you are well. The project that I'm currently working on requires
> static memory allocation instead of dynamic memory allocation. I was
> trying to use "memory_buffer_alloc()". Since memory_buffer_alloc()
> doesn't support multiple memory pools, seems like I can't use this
> function in my project.
>
> I have a working version of a modified bignum.c & bignum.h, where
> malloc is replaced with an array. The library is working. However, the
> performance is poor (~4 times slower). I was wondering what could
> cause this slow performance. I mean, I understand it will be slow, but
> I did not expect it to be 4 times slower.
>
> Is there any version, where an array was used instead of malloc? Or if
> you could point out some of the reasons for this library to slow down
> while using an array, I will be very grateful.
>
> Best regards,
> Shariful
>
>
> On Tue, Jul 20, 2021 at 2:22 PM Gilles Peskine <gilles.peskine@arm.com
>
mailto:gilles.peskine@arm.com> wrote:
>
> Hi Shariful,
>
> You just call mbedtls_calloc() (or let other functions call it).
> It will use the memory pool set by mbedtls_memory_buffer_alloc_init().
>
> The memory_buffer_alloc module does not support multiple memory
> pools. If you want a separate pool for each thread's allocation
> for performance reasons on a multicore system, you'd better rely
> on your platform's built-in calloc(). It's likely to be
> fined-tuned for multicore operation. The built-in allocator in
> Mbed TLS is intended for highly resource-constrained systems where
> the basic platform doesn't even include an allocator.
>
> Best regards,
>
> --
> Gilles Peskine
> Mbed TLS developer
>
> On 20/07/2021 22:10, Shariful Alam wrote:
>> Hi Gilles,
>> Thank you very much, for your reply. Sorry to bother you again. I
>> am trying to follow your instructions. I have a question
>> regarding your suggestions.
>>
>> You said "applications call mbedtls_memory_buffer_alloc_init() in
>> the startup code of the initial thread, before creating other
>> threads. The alloc/free functions are thread-safe, but the
>> initialization and deinitialization functions aren't."
>>
>> I'm a little confused here. Say, if I call
>> mbedtls_memory_buffer_alloc_init() with a buffer in the main
>> thread, how did all other threads use this memory (or how do all
>> the threads know which memory block to use)?
>>
>> After calling mbedtls_memory_buffer_alloc_init() in the main
>> thread, I can get the address of the buffer and pass it to the
>> threads, do I have to call mbedtls_memory_buffer_alloc_init()
>> again inside each thread?
>>
>> Thanks,
>> Shariful
>>
>> On Mon, Jul 19, 2021 at 3:48 AM Gilles Peskine via mbed-tls
>> <mbed-tls@lists.trustedfirmware.org
>>
mailto:mbed-tls@lists.trustedfirmware.org> wrote:
>>
>> Hi Shariful,
>>
>> First, please note that the library called PolarSSL with
>> functions like
>> rsa_private() and memory_buffer_alloc_init() has not been
>> supported for
>> several years. You should upgrade to Mbed TLS, with functions
>> like
>> mbedtls_rsa_private() and mbedtls_memory_buffer_alloc_init().
>> That being
>> said, the memory_buffer_alloc module works in the same way.
>>
>> Normally, applications call
>> mbedtls_memory_buffer_alloc_init() in the
>> startup code of the initial thread, before creating other
>> threads. The
>> alloc/free functions are thread-safe, but the initialization and
>> deinitialization functions aren't. If you must call
>> mbedtls_memory_buffer_alloc_init() after creating other
>> threads, make
>> sure that no thread calls mbedtls_calloc until
>> mbedtls_memory_buffer_alloc_init() has returned.
>>
>> The same principle applies to other parts of Mbed TLS that are
>> thread-safe. For example, only the RSA operations (encryption,
>> decryption, signature, verification, and also the low-level
>> functions
>> mbedtls_rsa_public() and mbedtls_rsa_private()) are
>> protected. So you
>> must finish setting up the RSA key inside one thread before
>> you pass a
>> pointer to other threads. Similarly, only
>> mbedtls_xxx_drbg_random() is
>> thread-safe, and the RNG setup (including
>> mbedtls_xxx_drgb_seed())
>> should be done as part of the initial application startup.
>>
>> Finally, note that mbedtls_rsa_private() alone cannot decrypt
>> a message:
>> all it does it to apply the private key operation. To decrypt
>> a simple
>> message encrypted with RSA-OAEP, call
>> mbedtls_rsa_rsaes_oaep_decrypt()
>> or mbedtls_rsa_pkcs1_decrypt() with a key set up for
>> MBEDTLS_RSA_PKCS_V21 encoding. To use the legacy PKCS#1v1.5
>> mechanism,
>> call mbedtls_rsa_rsaes_pkcs1_v15_decrypt() or
>> mbedtls_rsa_pkcs1_decrypt() with a key set up for
>> .MBEDTLS_RSA_PKCS_V15.
>> To decrypt a message using a RSA FDH hybrid scheme, you do
>> need to call
>> mbedtls_rsa_private() since Mbed TLS doesn't support it
>> natively, but
>> what this gives you is the intermediate secret from which you
>> then need
>> to derive a symmetric key, not the message itself.
>>
>> Best regards,
>>
>> --
>> Gilles Peskine
>> Mbed TLS developer
>>
>> On 18/07/2021 07:16, Shariful Alam via mbed-tls wrote:
>> > Hello,
>> > I have a simple example code to decrypt an
>> encrypted message using
>> > *rsa_private()*. I use *memory_buffer_alloc_init(), *in
>> order to use
>> > a static memory for the computation. I want to run my code
>> > concurrently. My code works with a single pthread. However,
>> when I try
>> > to run more than one thread my program fails to decrypt.
>> >
>> > ** I check the same code
>> without *memory_buffer_alloc_init(), *it
>> > works concurrently, without any issues at all.
>> >
>> > Therefore, I believe, the issue that I'm facing is
>> coming from the use
>> > of static memory(e.g. *memory_buffer_alloc_init()*). The
>> documentation
>> > of memorry_buffer_alloc.h shows,
>> >
>> > /**
>> >
>> > * \brief Initialize use of stack-based memory allocator.
>> >
>> > * The stack-based allocator does memory
>> management
>> > inside the
>> >
>> > * presented buffer and does not call malloc()
>> and free().
>> >
>> > * It sets the global polarssl_malloc() and
>> > polarssl_free() pointers
>> >
>> > * to its own functions.
>> >
>> > * (Provided polarssl_malloc() and
>> polarssl_free() are
>> > thread-safe if
>> >
>> > * POLARSSL_THREADING_C is defined)
>> >
>> > *
>> >
>> > * \note This code is not optimized and provides a
>> straight-forward
>> >
>> > * implementation of a stack-based memory
>> allocator.
>> >
>> > *
>> >
>> > * \param buf buffer to use as heap
>> >
>> > * \param len size of the buffer
>> >
>> > *
>> >
>> > * \return 0 if successful
>> >
>> > */
>> >
>> >
>> > So, I added the following configuration to the *config.h*
>> file
>> >
>> > 1. #define POLARSSL_THREADING_PTHREAD
>> > 2. #define POLARSSL_THREADING_C
>> >
>> > But I'm still getting errors while decrypting. Any help on
>> how to fix
>> > this? or what else should I add into the config.h file to
>> > make *memory_buffer_alloc_init() *thread-safe? Here is my
>> sample
>> > code:
https://pastebin.com/uyW3vknt
>>
https://pastebin.com/uyW3vknt
>> <
https://pastebin.com/uyW3vknt https://pastebin.com/uyW3vknt>
>> >
>> > Thanks,
>> > Shariful
>> >
>>
>> --
>> mbed-tls mailing list
>> mbed-tls@lists.trustedfirmware.org
>>
mailto:mbed-tls@lists.trustedfirmware.org
>>
https://lists.trustedfirmware.org/mailman/listinfo/mbed-tls
>>
https://lists.trustedfirmware.org/mailman/listinfo/mbed-tls
>>
>