When `mbedtls_pk_verify` is used to verify digital signatures generated by openssl, the MBEDTLS_ERR_RSA_VERIFY_FAILED error occurs, openssl Specifies the command used to generate a certificate: ```bash openssl md5 -sign private.key -out sign test.md ```
But when I use `mbedtls_pk_sign(&pk_pri_ctx, MBEDTLS_MD_MD5, md, 0, sign_info, sizeof(sign_info), &size, mbedtls_ctr_drbg_random, &ctrl_drbg)` Generating the signature and using `mbedtls_pk_verify` results are successful, Print the signatures generated by mbedtls are not found to be the same as those generated by openssl. Please help。
mbedtls version: ```c #define MBEDTLS_VERSION_STRING "3.4.0" #define MBEDTLS_VERSION_STRING_FULL "mbed TLS 3.4.0" ```
openssl version: ```c OpenSSL 1.1.1 11 Sep 2018 ```
Hello,
It's hard to say anything without seeing your code. But please make sure that you are doing the correct sequence of operations. The openssl command hashes the message, then performs an RSA or ECDSA signature on the hash. The function mbedtls_pk_sign only performs the RSA or ECDSA signature: md must be the hash of the message, e.g. obtained with mbedtls_md.
If you don't understand the data flow, I suggest separating the hashing step from the signing step with openssl: call `openssl md5 -binary` without -sign to calculate the hash, and call `openssl pkeyutl -pkeyopt digest:md5 -sign` to sign that hash. Compare the intermediate outputs with the ones from Mbed TLS.
Best regards,
mbed-tls@lists.trustedfirmware.org