I am developing an AWS IoT MQTT client on STM32H723 using FreeRTOS 10.3.1, LwIP 2.1.2 and MbedTLS.

My first attempt used MbedTLS 2.16.2 and it mostly works except that sometimes, seemingly related to the timing of the network I’m using, it fails when my client fails to send out anything after the Client Key Exchange message and then the server responds with a Close Notify as follows:

“Time”,“Source”,“Destination”,“Protocol”,“Length”,“Info”
“0.000000”,“10.24.108.94”,“54.157.158.237”,“TLSv1.2”,“418”,“Client Hello”
“0.016460”,“54.157.158.237”,“10.24.108.94”,“TLSv1.2”,“150”,“Server Hello”
“0.002777”,“54.157.158.237”,“10.24.108.94”,“TLSv1.2”,“685”,“Certificate”
“0.000000”,“54.157.158.237”,“10.24.108.94”,“TLSv1.2”,“438”,“Server Key Exchange, Certificate Request, Server Hello Done”
“0.976521”,“10.24.108.94”,“54.157.158.237”,“TLSv1.2”,“1006”,“Certificate, Client Key Exchange”
“0.014853”,“54.157.158.237”,“10.24.108.94”,“TLSv1.2”,“61”,“Alert (Level: Warning, Description: Close Notify)”

When it works it looks like this:

“Time”,“Source”,“Destination”,“Protocol”,“Length”,“Info”
“0.000000000”,“10.0.0.11”,“54.84.254.130”,“TLSv1.2”,“418”,“Client Hello”
“0.017495225”,“54.84.254.130”,“10.0.0.11”,“TLSv1.2”,“150”,“Server Hello”
“0.001743255”,“54.84.254.130”,“10.0.0.11”,“TLSv1.2”,“685”,“Certificate”
“0.000022282”,“54.84.254.130”,“10.0.0.11”,“TLSv1.2”,“392”,“Server Key Exchange”
“0.000004428”,“54.84.254.130”,“10.0.0.11”,“TLSv1.2”,“100”,“Certificate Request, Server Hello Done”
“2.606030441”,“10.0.0.11”,“54.84.254.130”,“TLSv1.2”,“1326”,“Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message”
“0.027421478”,“54.84.254.130”,“10.0.0.11”,“TLSv1.2”,“105”,“Change Cipher Spec, Encrypted Handshake Message”
“0.000971482”,“10.0.0.11”,“54.84.254.130”,“TLSv1.2”,“193”,“Application Data”
“0.075384354”,“54.84.254.130”,“10.0.0.11”,“TLSv1.2”,“87”,“Application Data”

The last two “Application Data” messages are the MQTT connect and connect ACK,

So before I filed a bug report, I upgraded to MbedTLS 2.28.4. The good news is that this client always (seems) to successfully negotiate the TLS handshake, but after receiving the first encrypted “Application Data” (connect), the broker immediately sends and “Encrypted Alert”, which I’m taking to be a Close Notify because a TCP FIN follows shortly thereafter.

The Wireshark capture of the 2.28.4 handshake looks almost exactly like the successful 2.16.2 handshake:

“Time”,“Source”,“Destination”,“Protocol”,“Length”,“Info”
“0.000000000”,“10.0.0.11”,“44.195.98.16”,“TLSv1.2”,“336”,“Client Hello”
“0.018450117”,“44.195.98.16”,“10.0.0.11”,“TLSv1.2”,“150”,“Server Hello”
“0.000524146”,“44.195.98.16”,“10.0.0.11”,“TLSv1.2”,“685”,“Certificate”
“0.000003168”,“44.195.98.16”,“10.0.0.11”,“TLSv1.2”,“392”,“Server Key Exchange”
“0.000002995”,“44.195.98.16”,“10.0.0.11”,“TLSv1.2”,“100”,“Certificate Request, Server Hello Done”
“2.517148776”,“10.0.0.11”,“44.195.98.16”,“TLSv1.2”,“1329”,“Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message”
“0.016572792”,“44.195.98.16”,“10.0.0.11”,“TLSv1.2”,“60”,“Change Cipher Spec”
“0.000007326”,“44.195.98.16”,“10.0.0.11”,“TLSv1.2”,“99”,“Encrypted Handshake Message”
“0.001500778”,“10.0.0.11”,“44.195.98.16”,“TLSv1.2”,“193”,“Application Data”
“0.031014646”,“44.195.98.16”,“10.0.0.11”,“TLSv1.2”,“85”,“Encrypted Alert”

… except for the “Encrypted Alert” from the broker that leads to the closing of the session.

I’m hoping someone has an explanation of the behavior I’m seeing and can suggest a path forward for me. Another data point is the fact that the client with MbedTLS 2.28.4 can successfully authentic and negotiate a TLS handshake with a mosquitto broker I’m running locally using self signed certificates.