Detailed Description

DTLS 1.3 server handshake state machine (RFC 9147 §5-6).

The transport-neutral core that drives one DTLS 1.3 server handshake: it consumes inbound datagrams and produces the outbound flight, wiring the reused TLS 1.3 message builders and key schedule (tls13_msg, tls13_kdf) through the DTLS record layer (dtls_record) and handshake framing (dtls_handshake). Like coap_server_process it has no sockets - the UDP glue (a later CoAPs front-end) feeds it datagrams and sends whatever it emits.

Profile: the single spec-valid suite the whole hand-rolled TLS 1.3 stack uses - TLS_AES_128_GCM_SHA256, X25519 key exchange, an Ed25519 server certificate. The handshake is the one-round-trip full handshake (no PSK, no 0-RTT, no client auth):

epoch 0 ClientHello -> <- ServerHello (epoch 0, DTLSPlaintext) epoch 2 <- EncryptedExtensions, Certificate, CertificateVerify, Finished (DTLSCiphertext) epoch 2 Finished -> epoch 3 application data (CoAP) protected with the app-traffic keys

Each handshake message fits one record in this profile. When the client does not offer an X25519 key_share up front, the server answers the first ClientHello with a HelloRetryRequest carrying a stateless, address-bound cookie and renegotiates the group to X25519 (RFC 9147 §5.1); the second ClientHello must echo the cookie before any asymmetric crypto is spent. Full ACK/timeout retransmission (§5.8, §7) beyond the Finished acknowledgement is a follow-on increment; the framing it needs already exists in dtls_handshake.

Author: Douglas Quigg (dstroy0)

Date: 2026

Definition in file dtls_conn.h.