Hi Michael,
I'm not sure I understand what library/program you're using on each side: you posted the source code of a server based on Mbed TLS, and an invocation of openssl as a server - 2 servers and no client, surely that's not your entire setup.
Also, even with the full details, this is a considerable amount of things to go over, so instead of trying to pinpoint the issue, I'd like to suggest a strategy that would help you debug things. For the side of the connection that's based on Mbed TLS, I suggest you start with one of the example programs located in programs/ssl? in our source tree. Hopefully those should work. If they don't, please come back to us with details on how you're invoking them and what's on the other side. If they work, I suggest you gradually modify them to adapt them to your needs, and after each change check if things still work. That way, if things break at one point, it should be easier to pinpoint the source of the issue.
Hope that will help!
Regards, Manuel.
PS: I had a quick look at the logs, and I'm not sure what software is on the client side, but its ClientHello.random doesn't look random. Also, the location of the errors suggests the the client and server didn't compute the same session keys - so something probably went wrong computing the pre-master secret or deriving the master secret and session keys from it. ________________________________ From: mbed-tls mbed-tls-bounces@lists.trustedfirmware.org on behalf of Михаил Азанов via mbed-tls mbed-tls@lists.trustedfirmware.org Sent: 21 November 2020 05:17 To: mbed-tls@lists.trustedfirmware.org mbed-tls@lists.trustedfirmware.org Subject: [mbed-tls] -0x7180 - SSL - Verification of the message MAC failed
Clients connect to my server over SSL and an error "-0x7180 - SSL - Verification of the message MAC failed" occurs when creating a secure connection. Linux commands give the following: uname -a Linux termv7 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
openssl s_server -key /etc/minpay/cert/TermV-terminals-server.key -cert /etc/minpay/cert/TermV-terminals-server.crt -accept 8080 -state Using default temp DH parameters ACCEPT SSL_accept:before SSL initialization SSL_accept:before SSL initialization SSL_accept:SSLv3/TLS read client hello SSL_accept:SSLv3/TLS write server hello SSL_accept:SSLv3/TLS write certificate SSL_accept:SSLv3/TLS write server done SSL_accept:SSLv3/TLS write server done SSL_accept:SSLv3/TLS read client key exchange SSL_accept:SSLv3/TLS read change cipher spec SSL_accept:SSLv3/TLS read finished SSL_accept:SSLv3/TLS write change cipher spec SSL_accept:SSLv3/TLS write finished -----BEGIN SSL SESSION PARAMETERS----- MHUCAQECAgMBBAIALwQgNbcRsh26hMujwXcoa5ZlBGihMY+GFhkTscR4Ds8fppwE MHAJs/h2ldK+C7Sm2fpNGbzYMycfUPGq4Ydg9PtnHPt1Mn9eH68tnQsT2nuTwGRq zqEGAgRftlMXogQCAhwgpAYEBAEAAAA= -----END SSL SESSION PARAMETERS----- Shared ciphers:AES128-SHA --- No server certificate CA names sent CIPHER is AES128-SHA Secure Renegotiation IS NOT supported
Source content:
File contents TermV-terminals-server.key: subject=/C=RU/ST=Sample/L=Sample/O=Sample/OU=Sample/CN=localhost issuer=/C=RU/ST=Sample/L=Sample/O=Sample/OU=Sample/CN=Sample -----BEGIN CERTIFICATE----- MIIDTjCCAjYCFBhhDBE3BeiTMD0dzFYOGILO8yq0MA0GCSqGSIb3DQEBCwUAMGIx CzAJBgNVBAYTAlJVMQ8wDQYDVQQIDAZTYW1wbGUxDzANBgNVBAcMBlNhbXBsZTEP MA0GA1UECgwGU2FtcGxlMQ8wDQYDVQQLDAZTYW1wbGUxDzANBgNVBAMMBlNhbXBs ZTAeFw0yMDAxMTgxODA5MDlaFw0yNDAxMTcxODA5MDlaMGUxCzAJBgNVBAYTAlJV MQ8wDQYDVQQIDAZTYW1wbGUxDzANBgNVBAcMBlNhbXBsZTEPMA0GA1UECgwGU2Ft cGxlMQ8wDQYDVQQLDAZTYW1wbGUxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJ KoZIhvcNAQEBBQADggEPADCCAQoCggEBALQSjObDf16fv02qH+3FtI4eX2tPfj+u XsSF8KK6OpcNNfHv17LB9NRUGbZdeikvL5mjB1qd8Qt5YBzSYvvNmuQqn/ZZhEhg 9zqDd+j6dsl03OSW42qSP59M22plqhHZv1ZpkbpHSnEbbwbIVhgx1GuJvMHbQKxw et3UobG9M7q0/6Z2hlvbiuQgZWpc6+vnGGFUzVFjJhsUMgewU1UwTtfahOVegv9v ocuj+AOHzzH2+l72ybpKIFKZ6eXjr3sVp1lIPXm+gUNLqkReNV2QVk4H/j2ow3xu hdHjiPQ+6o6dgbCHeCvbf7Qs35S8YlQH/Mn2bik2NIMKucdq7S35AEsCAwEAATAN BgkqhkiG9w0BAQsFAAOCAQEAhANjStGtYfPJ2Ibkbn6ct4bWLPe6sFBvbwYow3Fd lBz0oxBukhJDOrPa4OqJDpnWenXkEcUrulI2JG5sIylOg3QlwlmgPoJU9fkpmr+u xNVlTxBSh1BVp0sccC+LxNRyDMl4C89R1TjPpPjoRU4cciY9N0oygPjTj0c26qBU iqwtyJTeeJYJUdplCMvXFOt8deqi/NBpBGnEfo8P/IeM/iCWLD5VtloHX/R0C+au q6803qd1/Gd0pIdsELQKrR6mBz6sWukg2f25GheLQVV7Wz/elZD8MwYwjYLoCtIP 65Ae3Eb6S2KlsVPzMj09u6kQ+KPfZtKu8tiUSc+11hurRw== -----END CERTIFICATE-----
File contents TermV-terminals-server.crt: -----BEGIN PRIVATE KEY----- MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQC0Eozmw39en79N qh/txbSOHl9rT34/rl7EhfCiujqXDTXx79eywfTUVBm2XXopLy+ZowdanfELeWAc 0mL7zZrkKp/2WYRIYPc6g3fo+nbJdNzkluNqkj+fTNtqZaoR2b9WaZG6R0pxG28G yFYYMdRribzB20CscHrd1KGxvTO6tP+mdoZb24rkIGVqXOvr5xhhVM1RYyYbFDIH sFNVME7X2oTlXoL/b6HLo/gDh88x9vpe9sm6SiBSmenl4697FadZSD15voFDS6pE XjVdkFZOB/49qMN8boXR44j0PuqOnYGwh3gr23+0LN+UvGJUB/zJ9m4pNjSDCrnH au0t+QBLAgMBAAECggEBAKWbKdpQkSME53sVqqeR26uYY0jdos4OHzm9tMDzZE7m 52XJniXYGsHrVnpRHbginTQ+qtS7zKMwzSO0mTPas0iDqvl0+iNWighWQEETl4tO 3peb7Spltf1bQa9oMDCagU2HaW5Xgi7jfAhQ1l0J9sGvutjOO3xbNaQtmUdNKfXa VRaof94/YKcbTPIn588orxfumgeiYwQ8exhA7yg1WLl2iSy9jj+8WrxaKvy4vR45 nXrb2tXletRoZZY/FC/NZNMgXsGTmYHl6blTEZVz7Zu0XPHKJo42tHnKHW46/ccJ vkXQNjUsu0JN95+lWBA5Yne8xddhiLNKv3DZ3iwXRAECgYEA17YaIbtlgSLd5hJJ OuS9n9XNUCEx1YLhDRzhHggPNTZkVxSnxnWVGpbubtWN/OggRtJyk+mLFV+aHLCa R224ofSPWWE1HtoSdtLcL5oLgp6RDoM61h1p1glFS7BpbYzBegAcbMJa1IvAMspx 5lsYowBUJLNHAEcqUrXzQ88gu2ECgYEA1bRugqLGXqz1E0z6o/MuGXfmJaGQEmJg 833As5jzsyd6pyznsB0TKUAJkcf0RmZrr1EDVigA9aLcwq4qThjGEL2FpGkOvXPC bY0c2pLnsZJetNmXnky/w+W127V+LbDjjQaYl/7c9Fp4hoomoykt7d1YW7lRZ2Le wo8muiGF5ysCgYEAlYHahMysop926tJ7vPzzTMfT4JjRQGnQ79S3VqhBWiFT1GM1 kcDHUkGQCnOrUMHWNSABV/FDe9HiL8Zbd+xdTqsBe/J67eI5b+/fuoJrPeIHKebc rbB/PWD5jWc8+zfWlWdkTCE88RnXYZyc6wryfW9p4nH7YP7yH5eKftIdnqECgYBl aWY33/661uDF8/XM742k0F0K5oxz7PONGNPlZmPfVJDD3G9mB6YcISNpZrXo4pmf bJZkwD8UUeDpEbVJsj/rmcRdrO0twk01p41Vu/jvL0J6F/f3SvyFffC6/nmOPS7+ sW6gUnWQD466abzEGLqO8kcH3/1dTnHfagc6tMXSWQKBgQCZjWzyLtEUxvWI2PSe B6/osxlYDyCkXrJDIT1LuepM/54Kre9YMUy2UZBVC42DR6paEbkAAqHdTsLNlKhV s+6/JGnEDR1e/L7ks0dBY8jow04r0mKG1M91SMLr5OeBTk6SRMnR7GAKewjeLmJD ULgsqyHSTh/4r6tDQ7Gayci2vQ== -----END PRIVATE KEY-----
File contents ssl_server.h: #ifndef SSL_SERVER_H #define SSL_SERVER_H
#define DEBUG_LEVEL 5 #if DEBUG_LEVEL > 0 #include "mbedtls/debug.h" #endif //#define MBEDTLS_SSL_CIPHERSUITES MBEDTLS_TLS_DHE_PSK_WITH_AES_128_GCM_SHA256 #include "mbedtls/entropy.h" #include "mbedtls/ctr_drbg.h" #include "mbedtls/certs.h" #include "mbedtls/x509.h" #include "mbedtls/ssl.h" #include "mbedtls/net_sockets.h" #include "mbedtls/error.h" #include "mbedtls/ssl_cache.h" #include "packet_handler/packet_handler.h" #include "thread_pool_ssl.h" #include "queue_ssl.h" extern pthread_mutex_t listen_fd1_mutex; extern mbedtls_net_context listen_fd1; extern void* runSSLServer(void *arg); #endif // SSL_SERVER_H
File contents ssl_server.c: #include "ssl_server.h"
mbedtls_net_context listen_fd1; pthread_mutex_t listen_fd1_mutex=PTHREAD_MUTEX_INITIALIZER;
static mbedtls_ssl_config *config;
#if DEBUG_LEVEL > 0 static void my_debug( void *ctx, int level, const char *file, int line, const char *str ) { ((void) level); char s[1000]; snprintf(s,sizeof(s),"%s:%04d: %s", file, line, str ); trace(s); } #endif void * runSSLServer(void *arg) { (void)(arg); int ret; char s[100]; int n = atoi(config_list[SSL_SERVER_NUM_THREADS].value); createThreadPoolSSLArray(n); mbedtls_net_context listen_fd, client_socket_ssl; const char pers[] = "ssl_pthread_server"; mbedtls_entropy_context entropy; mbedtls_ctr_drbg_context ctr_drbg; mbedtls_ssl_config conf; mbedtls_x509_crt srvcert; mbedtls_x509_crt cachain; mbedtls_pk_context pkey; mbedtls_ssl_cache_context cache; mbedtls_ssl_cache_init( &cache ); mbedtls_x509_crt_init( &srvcert ); mbedtls_x509_crt_init( &cachain ); mbedtls_ssl_config_init( &conf ); mbedtls_ctr_drbg_init( &ctr_drbg ); mbedtls_net_init( &listen_fd ); mbedtls_net_init( &client_socket_ssl );
config = &conf; /* * We use only a single entropy source that is used in all the threads. */ mbedtls_entropy_init( &entropy );
#if DEBUG_LEVEL > 0 mbedtls_debug_set_threshold(DEBUG_LEVEL); #endif /* * 1. Load the certificates and private RSA key */ trace( "[SSL Server] Loading the server cert. and key..." ); ret = mbedtls_x509_crt_parse_file( &srvcert,"/etc/minpay/cert/TermV-terminals-server.crt"); if( ret != 0 ) { snprintf(s,sizeof(s), "[SSL Server] failed\n ! mbedtls_x509_crt_parse_file returned %d\n\n", ret ); trace(s); goto thread_exit; }
mbedtls_pk_init( &pkey ); ret = mbedtls_pk_parse_keyfile( &pkey,"/etc/minpay/cert/TermV-terminals-server.key" , NULL ); if( ret != 0 ) { snprintf(s,sizeof(s), "[SSL Server] failed\n ! mbedtls_pk_parse_keyfile returned %d\n\n", ret ); trace(s); goto thread_exit; } trace( "[SSL Server] ok\n" ); /* * 1b. Seed the random number generator */ trace( "Seeding the random number generator..." ); if( ( ret = mbedtls_ctr_drbg_seed( &ctr_drbg, mbedtls_entropy_func, &entropy, (const unsigned char *) pers, strlen( pers ) ) ) != 0 ) { snprintf(s,sizeof(s),"[SSL Server] failed: mbedtls_ctr_drbg_seed returned -0x%04x\n", -ret ); trace(s); goto thread_exit; } trace( "[SSL Server] ok\n" ); /* * 1c. Prepare SSL configuration */ trace( "[SSL Server] Setting up the SSL data...." ); if( ( ret = mbedtls_ssl_config_defaults( &conf, MBEDTLS_SSL_IS_SERVER, MBEDTLS_SSL_TRANSPORT_STREAM, MBEDTLS_SSL_PRESET_DEFAULT ) ) != 0 ) { snprintf(s,sizeof(s),"[SSL Server] failed: mbedtls_ssl_config_defaults returned -0x%04x\n", -ret ); trace(s); goto thread_exit; } mbedtls_ssl_conf_rng( &conf, mbedtls_ctr_drbg_random, &ctr_drbg ); #if DEBUG_LEVEL > 0 mbedtls_ssl_conf_dbg( &conf, my_debug, NULL ); #endif /* mbedtls_ssl_cache_get() and mbedtls_ssl_cache_set() are thread-safe if * MBEDTLS_THREADING_C is set. */ mbedtls_ssl_conf_session_cache( &conf, &cache, mbedtls_ssl_cache_get, mbedtls_ssl_cache_set );
mbedtls_ssl_conf_ca_chain( &conf, &cachain, NULL ); if( ( ret = mbedtls_ssl_conf_own_cert( &conf, &srvcert, &pkey ) ) != 0 ) { snprintf(s,sizeof(s),"[SSL Server] failed\n ! mbedtls_ssl_conf_own_cert returned %d\n\n", ret ); trace(s); goto thread_exit; } trace( "[SSL Server] ok\n" ); /* * 2. Setup the listening TCP socket */
snprintf(s,sizeof(s),"[SSL Server] Bind on localhost: %s ...","8080"); trace(s); fflush( stdout ); if( ( ret = mbedtls_net_bind( &listen_fd, NULL, "8080", MBEDTLS_NET_PROTO_TCP ) ) != 0 ) { snprintf(s,sizeof(s),"[SSL Server] failed\n ! mbedtls_net_bind returned %d\n\n", ret ); trace(s); goto thread_exit; } trace( "[SSL Server] ok\n" );
pthread_mutex_lock(&listen_fd1_mutex); listen_fd1 = listen_fd; pthread_mutex_unlock(&listen_fd1_mutex);
while(true) { //reset: if(ret!=0) { char error_buf[100]; mbedtls_strerror( ret, error_buf, 100 ); snprintf(s,sizeof(s),"[SSL Server] Last error was: -0x%04x - %s\n", -ret, error_buf ); trace(s); } /* * 3. Wait until a client connects */ trace( "[SSL Server] Waiting for a remote connection\n" ); int ret =mbedtls_net_accept(&listen_fd,&client_socket_ssl,NULL,0,NULL); if(ret != 0) { snprintf(s,sizeof(s), "[SSL Server] failed: mbedtls_net_accept returned -0x%04x\n", ret); trace(s); break; } trace( "[SSL Server] ok\n" ); //do whatever we do with connections. //put the connection somewhere so that an available thread //can find it.
mbedtls_net_context *pclient = malloc(sizeof(mbedtls_net_context)); *pclient = client_socket_ssl; pthread_mutex_lock(&client_socket_ssl_mutex); enqueue_ssl(pclient); pthread_cond_signal(&condition_var_ssl); pthread_mutex_unlock(&client_socket_ssl_mutex); ret = 0; } thread_exit: mbedtls_x509_crt_free( &srvcert ); mbedtls_pk_free( &pkey ); mbedtls_ssl_cache_free( &cache ); mbedtls_ctr_drbg_free( &ctr_drbg ); mbedtls_entropy_free( &entropy ); mbedtls_ssl_config_free( &conf ); mbedtls_net_free( &listen_fd ); deleteThreadPoolSSLArray(); return NULL; } static void *handleConnectionSSL(mbedtls_net_context *p_net_context) { int ret, len; char s[100]; struct packet_handler handler={}; handler.net_context = *p_net_context; free(p_net_context);// we really don't need this anymore unsigned char buf[1024]; /* Make sure memory references are valid */ mbedtls_ssl_init( &handler.ssl ); snprintf(s,sizeof(s),"[SSL client_socket=%d] Setting up SSL/TLS data\n", handler.net_context.fd ); trace(s);
/* * 4. Get the SSL context ready */ if( ( ret = mbedtls_ssl_setup( &handler.ssl, config ) ) != 0 ) { snprintf(s,sizeof(s),"[SSL client_socket=%d] failed: mbedtls_ssl_setup returned -0x%04x\n",handler.net_context.fd, -ret ); trace(s); goto thread_exit; } mbedtls_ssl_set_bio( &handler.ssl, (mbedtls_net_context*)&handler.net_context, mbedtls_net_send, mbedtls_net_recv, NULL ); /* * 5. Handshake */ snprintf(s,sizeof(s),"[SSL client_socket=%d] Performing the SSL/TLS handshake\n", handler.net_context.fd); trace(s); while( ( ret = mbedtls_ssl_handshake( &handler.ssl ) ) != 0 ) { if( ret != MBEDTLS_ERR_SSL_WANT_READ && ret != MBEDTLS_ERR_SSL_WANT_WRITE ) { snprintf(s,sizeof(s),"[SSL client_socket=%d] failed: mbedtls_ssl_handshake returned -0x%04x\n", handler.net_context.fd, -ret ); trace(s); goto thread_exit; } } snprintf(s,sizeof(s),"[SSL client_socket=%d] ok\n", handler.net_context.fd ); trace(s);
ret = runPacketHandler(&handler);
snprintf(s,sizeof(s), "[SSL client_socket=%d] . Closing the connection...", handler.net_context.fd ); trace(s); while( ( ret = mbedtls_ssl_close_notify( &handler.ssl ) ) < 0 ) { if( ret != MBEDTLS_ERR_SSL_WANT_READ && ret != MBEDTLS_ERR_SSL_WANT_WRITE ) { snprintf(s,sizeof(s), "[SSL client_socket=%d] failed: mbedtls_ssl_close_notify returned -0x%04x\n", handler.net_context.fd, ret); trace(s); goto thread_exit; } } trace( " ok\n" ); ret = 0; thread_exit: if( ret != 0 ) { char error_buf[100]; mbedtls_strerror( ret, error_buf, 100 ); snprintf(s,sizeof(s), "[SSL client_socket=%d] Last error was: -0x%04x - %s\n\n", handler.net_context.fd, -ret, error_buf ); trace(s); } mbedtls_net_free((mbedtls_net_context*)&handler.net_context); mbedtls_ssl_free(&handler.ssl); return NULL; } void *threadFunctionSSL(void *arg) { (void)(arg); while(true) { mbedtls_net_context *pclient; pthread_mutex_lock(&client_socket_ssl_mutex); if((pclient=dequeue_ssl())==NULL) { pthread_cond_wait((pthread_cond_t *)&pclient,&client_socket_ssl_mutex); //try again pclient = dequeue_ssl(); } pthread_mutex_unlock(&client_socket_ssl_mutex); if(pclient!=NULL) { //we have a connection handleConnectionSSL(pclient); } } }
File contents thread_pool_ssl.h: #ifndef SSL_THREAD_POOL_H #define SSL_THREAD_POOL_H #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <stdbool.h> #include <pthread.h> #include "mbedtls/net_sockets.h"
extern pthread_t *thread_pool_ssl; extern pthread_mutex_t client_socket_ssl_mutex; extern pthread_cond_t condition_var_ssl; void createThreadPoolSSLArray(int n); void deleteThreadPoolSSLArray(); extern void *handleSSLConnection(int* p_client_socket); extern void *threadFunctionSSL(void *arg); #endif // SSL_THREAD_POOL_H
File contents thread_pool_ssl.c: #include "thread_pool_ssl.h" pthread_t *thread_pool_ssl=NULL; pthread_mutex_t client_socket_ssl_mutex = PTHREAD_MUTEX_INITIALIZER; pthread_cond_t condition_var_ssl = PTHREAD_COND_INITIALIZER; void *threadFunctionSSL(void *arg); void createThreadPoolSSLArray(int n) { thread_pool_ssl = (pthread_t*)malloc(n); //first off create a bunch of threads to handle future connections for(int i=0; i<n;i++) { pthread_create(&thread_pool_ssl[i],NULL,threadFunctionSSL,NULL); } } void deleteThreadPoolSSLArray() { free(thread_pool_ssl); } File contents main.c: #include <stdio.h> #include "log.h" #include "ssl_server.h" #include <signal.h>
void catchSigterm(); void catchSigint(); void runPortListening(); int main() { catchSigterm(); catchSigint(); runPortListening(); return 0; }
void runPortListening() { trace("Starting port listening..."); pthread_t handleID; pthread_create(&handleID,NULL,runSSLServer,NULL); trace("Port listening started."); pthread_join(handleID,NULL); trace("Port listening complete."); } void closeServer() { trace("Shutdown SSL Server..."); pthread_mutex_lock(&listen_fd1_mutex); mbedtls_net_free(&listen_fd1); pthread_mutex_unlock(&listen_fd1_mutex); }
void sigTermHandler(int signum, siginfo_t *info, void *ptr) { (void)signum; (void)info; (void)ptr; closeServer(); } void catchSigterm() { static struct sigaction _sigact; memset(&_sigact, 0, sizeof(_sigact)); _sigact.sa_sigaction = sigTermHandler; _sigact.sa_flags = SA_SIGINFO; sigaction(SIGTERM, &_sigact, NULL); } void sigIntHandler(int signum, siginfo_t *info, void *ptr) { (void)signum; (void)info; (void)ptr; closeServer(); } void catchSigint() { static struct sigaction _sigact; memset(&_sigact, 0, sizeof(_sigact)); _sigact.sa_sigaction = sigIntHandler; _sigact.sa_flags = SA_SIGINFO; sigaction(SIGINT, &_sigact, NULL); }
Log files attached to the email.
Regards, Michael.
mbed-tls@lists.trustedfirmware.org