dtls_handshake.h File Reference

DTLS 1.3 handshake framing and reliability (RFC 9147 §5, §7). More...

#include "protocore_config.h"

Go to the source code of this file.

Classes
struct	DtlsHsHeader
	Parsed view of one DTLS handshake message fragment (fields point into the caller buffer). More...
struct	DtlsHsReasm
	Reassembles the fragments of a single handshake message into a contiguous body. More...
struct	DtlsRecordNumber
	A record identified for acknowledgement: (epoch, sequence_number), 8 bytes each on the wire (RFC 9147 §7). More...
struct	DtlsHandshakeNs
	Dispatch table. Addressed by offset, so the layout is asserted below. More...

Macros
#define	PROTOCORE_DTLS_HS_HDR_LEN   12
	DTLS handshake header length: msg_type(1) + length(3) + message_seq(2) + fragment_offset(3)
#define	PROTOCORE_DTLS_HS_TYPE_MESSAGE_HASH   254
	message_hash synthetic-message type used when wrapping ClientHello1 for a HelloRetryRequest transcript (RFC 8446 §4.4.1). Framing constant; the transcript itself lives in protocore_dtls_conn.
#define	PROTOCORE_DTLS_HS_REASM_MAX_RANGES   8
	Max distinct byte ranges tracked while reassembling one message (bounds the work an adversary can force by sending maximally fragmented flights).
#define	PROTOCORE_DTLS_COOKIE_MAX   128
	Maximum cookie length this implementation emits / accepts. Overhead is 43 bytes (version + timestamp + payload_len + HMAC); the rest is available for the payload.

Functions
	PROTOCORE_NS_LAYOUT (DtlsHandshakeNs, header_parse, frag_build, reasm_init, reasm_add, ack_build, ack_parse, cookie_make, cookie_verify)
size_t	protocore_dtls_handshake_header_parse (uint8_t *work, const uint8_t *p, size_t len, DtlsHsHeader *out)
	The 12-byte DTLS handshake header; bytes consumed, or 0 if truncated.
size_t	protocore_dtls_handshake_frag_build (uint8_t *work, uint8_t msg_type, uint16_t msg_seq, uint32_t full_len, uint32_t frag_offset, const uint8_t *frag, uint32_t frag_len, uint8_t *out, size_t out_cap)
	One handshake fragment, header and body; bytes written, or 0 on .
void	protocore_dtls_handshake_reasm_init (uint8_t *work, DtlsHsReasm *r, uint16_t msg_seq, uint8_t *buf, size_t buf_cap)
	Bind a reassembler to a caller buffer for one message sequence .
size_t	protocore_dtls_handshake_reasm_add (uint8_t *work, DtlsHsReasm *r, const DtlsHsHeader *frag)
	Add one fragment to the reassembly in progress.
size_t	protocore_dtls_handshake_ack_build (uint8_t *work, const DtlsRecordNumber *nums, size_t count, uint8_t *out, size_t out_cap)
	An ACK body (RFC 9147 sec 7) over count record numbers.
proto_bool	protocore_dtls_handshake_ack_parse (uint8_t *work, const uint8_t *body, size_t len, DtlsRecordNumber *out, size_t out_cap, size_t *out_count)
	The same body back into at most out_cap record numbers.
size_t	protocore_dtls_handshake_cookie_make (uint8_t *work, uint8_t *mac_work, const uint8_t *protocore_hmac_key, uint64_t timestamp, const uint8_t *payload, size_t payload_len, const uint8_t *client_addr, size_t addr_len, uint8_t *out, size_t out_cap)
	A stateless HelloRetryRequest cookie bound to the client address.
proto_bool	protocore_dtls_handshake_cookie_verify (uint8_t *work, uint8_t *mac_work, const uint8_t *protocore_hmac_key, uint64_t now, uint64_t max_age, const uint8_t *client_addr, size_t addr_len, const uint8_t *cookie, size_t cookie_len, uint8_t *payload_out, size_t payload_cap, size_t *payload_len_out)
	The same cookie back, checking the address binding and the age .

Variables
PROTOCORE_NS DtlsHandshakeNs DtlsHandshake	PROTOCORE_UNUSED
	Module namespace.

Detailed Description

DTLS 1.3 handshake framing and reliability (RFC 9147 §5, §7).

The datagram-reliability layer that sits between the DTLS record layer (protocore_dtls_record) and the reused TLS 1.3 message builders (protocore_tls13_msg). TLS 1.3 assumes an in-order reliable byte stream; DTLS carries the same handshake messages over lossy, reorderable datagrams, so each message gains a 12-byte DTLS handshake header (RFC 9147 §5.2) that lets a fragment be placed independently of the record that carried it, and lost flights are recovered with acknowledgements (§7) rather than TCP retransmission.

This file is pure framing - no crypto state, no sockets. It provides:

• the 12-byte handshake header (protocore_dtls_hs_header_parse / protocore_dtls_hs_frag_build);
• overlap-tolerant message reassembly (DtlsHsReasm), modelled on the QUIC CRYPTO-stream reassembler - a fragment may arrive split, duplicated, or overlapping (§5.4);
• the ACK message (protocore_dtls_ack_build / protocore_dtls_ack_parse, content type 26, §7);
• the stateless HelloRetryRequest cookie (protocore_dtls_cookie_make / protocore_dtls_cookie_verify, the §5.1 return-routability / anti-amplification defense).

The handshake state machine that drives these (flights, epochs, PTO) is protocore_dtls_conn; the TLS 1.3 message bodies and key schedule are reused verbatim from the HTTP/3 stack (protocore_tls13_msg, protocore_tls13_kdf).

work is bytes the CALLER holds. This module reads none of them: it carries nothing between calls, so there is no state to keep and nothing to wipe. The parameter is there so a caller drives every namespace the same way.

Author

Douglas Quigg (dstroy0)

Date

2026

Definition in file dtls_handshake.h.

Macro Definition Documentation

PROTOCORE_DTLS_HS_HDR_LEN

#define PROTOCORE_DTLS_HS_HDR_LEN   12

DTLS handshake header length: msg_type(1) + length(3) + message_seq(2) + fragment_offset(3)

• fragment_length(3) = 12 bytes (RFC 9147 §5.2).

Definition at line 47 of file dtls_handshake.h.

PROTOCORE_DTLS_HS_TYPE_MESSAGE_HASH

#define PROTOCORE_DTLS_HS_TYPE_MESSAGE_HASH   254

message_hash synthetic-message type used when wrapping ClientHello1 for a HelloRetryRequest transcript (RFC 8446 §4.4.1). Framing constant; the transcript itself lives in protocore_dtls_conn.

Definition at line 51 of file dtls_handshake.h.

PROTOCORE_DTLS_HS_REASM_MAX_RANGES

#define PROTOCORE_DTLS_HS_REASM_MAX_RANGES   8

Max distinct byte ranges tracked while reassembling one message (bounds the work an adversary can force by sending maximally fragmented flights).

Definition at line 55 of file dtls_handshake.h.

PROTOCORE_DTLS_COOKIE_MAX

#define PROTOCORE_DTLS_COOKIE_MAX   128

Maximum cookie length this implementation emits / accepts. Overhead is 43 bytes (version + timestamp + payload_len + HMAC); the rest is available for the payload.

Definition at line 59 of file dtls_handshake.h.

Function Documentation

PROTOCORE_NS_LAYOUT()

PROTOCORE_NS_LAYOUT	(	DtlsHandshakeNs	,
		header_parse	,
		frag_build	,
		reasm_init	,
		reasm_add	,
		ack_build	,
		ack_parse	,
		cookie_make	,
		cookie_verify
	)

protocore_dtls_handshake_header_parse()

size_t protocore_dtls_handshake_header_parse	(	uint8_t *	work,
		const uint8_t *	p,
		size_t	len,
		DtlsHsHeader *	out
	)

The 12-byte DTLS handshake header; bytes consumed, or 0 if truncated.

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
p	P
len	Len
out	Out

Returns

The size_t.

protocore_dtls_handshake_frag_build()

size_t protocore_dtls_handshake_frag_build	(	uint8_t *	work,
		uint8_t	msg_type,
		uint16_t	msg_seq,
		uint32_t	full_len,
		uint32_t	frag_offset,
		const uint8_t *	frag,
		uint32_t	frag_len,
		uint8_t *	out,
		size_t	out_cap
	)

One handshake fragment, header and body; bytes written, or 0 on .

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
msg_type	Msg type
msg_seq	Msg seq
full_len	Full len
frag_offset	Frag offset
frag	Frag
frag_len	Frag len
out	Out
out_cap	Out cap

Returns

The size_t.

protocore_dtls_handshake_reasm_init()

void protocore_dtls_handshake_reasm_init	(	uint8_t *	work,
		DtlsHsReasm *	r,
		uint16_t	msg_seq,
		uint8_t *	buf,
		size_t	buf_cap
	)

Bind a reassembler to a caller buffer for one message sequence .

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
r	R
msg_seq	Msg seq
buf	Buf
buf_cap	Buf cap

protocore_dtls_handshake_reasm_add()

size_t protocore_dtls_handshake_reasm_add	(	uint8_t *	work,
		DtlsHsReasm *	r,
		const DtlsHsHeader *	frag
	)

Add one fragment to the reassembly in progress.

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
r	R
frag	Frag

Returns

The size_t.

protocore_dtls_handshake_ack_build()

size_t protocore_dtls_handshake_ack_build	(	uint8_t *	work,
		const DtlsRecordNumber *	nums,
		size_t	count,
		uint8_t *	out,
		size_t	out_cap
	)

An ACK body (RFC 9147 sec 7) over count record numbers.

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
nums	Nums
count	Count
out	Out
out_cap	Out cap

Returns

The size_t.

protocore_dtls_handshake_ack_parse()

proto_bool protocore_dtls_handshake_ack_parse	(	uint8_t *	work,
		const uint8_t *	body,
		size_t	len,
		DtlsRecordNumber *	out,
		size_t	out_cap,
		size_t *	out_count
	)

The same body back into at most out_cap record numbers.

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
body	Body
len	Len
out	Out
out_cap	Out cap
out_count	Out count

Returns

PROTO_TRUE on success.

protocore_dtls_handshake_cookie_make()

size_t protocore_dtls_handshake_cookie_make	(	uint8_t *	work,
		uint8_t *	mac_work,
		const uint8_t *	protocore_hmac_key,
		uint64_t	timestamp,
		const uint8_t *	payload,
		size_t	payload_len,
		const uint8_t *	client_addr,
		size_t	addr_len,
		uint8_t *	out,
		size_t	out_cap
	)

A stateless HelloRetryRequest cookie bound to the client address.

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
mac_work	Mac work
protocore_hmac_key	32 bytes
timestamp	Timestamp
payload	Payload
payload_len	Payload len
client_addr	Client addr
addr_len	Addr len
out	Out
out_cap	Out cap

Returns

The size_t.

protocore_dtls_handshake_cookie_verify()

proto_bool protocore_dtls_handshake_cookie_verify	(	uint8_t *	work,
		uint8_t *	mac_work,
		const uint8_t *	protocore_hmac_key,
		uint64_t	now,
		uint64_t	max_age,
		const uint8_t *	client_addr,
		size_t	addr_len,
		const uint8_t *	cookie,
		size_t	cookie_len,
		uint8_t *	payload_out,
		size_t	payload_cap,
		size_t *	payload_len_out
	)

The same cookie back, checking the address binding and the age .

Parameters

work	PROTOCORE_DTLS_HANDSHAKE_BORROW bytes the caller took. Not held past the call.
mac_work	Mac work
protocore_hmac_key	32 bytes
now	Now
max_age	Max age
client_addr	Client addr
addr_len	Addr len
cookie	Cookie
cookie_len	Cookie len
payload_out	Payload out
payload_cap	Payload cap
payload_len_out	Payload len out

Returns

PROTO_TRUE on success.

Variable Documentation

PROTOCORE_UNUSED

PROTOCORE_NS DtlsHandshakeNs DtlsHandshake PROTOCORE_UNUSED

Initial value:

= {.header_parse = protocore_dtls_handshake_header_parse,
                                                               .frag_build = protocore_dtls_handshake_frag_build,
                                                               .reasm_init = protocore_dtls_handshake_reasm_init,
                                                               .reasm_add = protocore_dtls_handshake_reasm_add,
                                                               .ack_build = protocore_dtls_handshake_ack_build,
                                                               .ack_parse = protocore_dtls_handshake_ack_parse,
                                                               .cookie_make = protocore_dtls_handshake_cookie_make,
                                                               .cookie_verify = protocore_dtls_handshake_cookie_verify}

Module namespace.

Definition at line 223 of file dtls_handshake.h.