|
ProtoCore v1.0.16
Deterministic, zero-heap network stack for embedded targets
|
How to size the worker model for a given workload without giving up the library's determinism guarantees (no heap after begin(), fixed buffers, bounded latency). Every knob here is compile-time; the default build is the tested-deterministic path, so you only change these when you have a specific reason.
Prefer a UI? The interactive build configurator lets you tick features, tune every knob, and copy out the
build_flags/#defines. It is generated fromsrc/protocore_config.h, so it always matches the library.
The server runs in one or more dedicated FreeRTOS worker tasks, not the user's loop(). Each worker owns a disjoint partition of connection slots (slot i -> worker i % PROTOCORE_WORKER_COUNT) plus its own event queue and scratch arena, so no two workers ever touch the same state: there are no hot-path locks, and that keeps latency bounded (= deterministic) while cores run disjoint connections in parallel. A worker blocks on its FreeRTOS task notification and is woken the moment an event or a deferred callback is queued, so event latency is independent of the idle-sweep cadence. PROTOCORE_WORKER_COUNT == 1 (the default) is byte-for-byte the original single-pipeline behavior. The lwIP callbacks that run on the tcpip thread (shared with WiFi on Core 0) are kept minimal: a received segment is bulk-copied into the slot's ring with a single SPSC publish, then one event is posted.
Every byte the library owns is BSS: allocated at link time, zero-initialized by the C runtime, never heap-allocated after begin().
Everything is bound in two places: at ingestion, and by mmgr. A length is bound where bytes enter the library - the receive path refuses a segment that will not fit the ring, never truncating it, and every parser downstream carries an explicit run length instead of scanning for a terminator (strlen is banned in src/). Working memory is bound by the two pools below. Nothing between those two points re-derives a bound, and the footprint is a number you can compute before flashing. It is not a property you measure afterwards.
Working memory is therefore not per-feature buffers but two pools, both the same mechanism (protocore_arena, mmgr/arena.h) instantiated twice, with one arena per slot - one per worker, plus the ghost, the library's own.
| Pool | Holds | Reclaim |
|---|---|---|
plaintext (mmgr/plaintext.h) | transient bytes that are not secret | protocore_plaintext_reset() per dispatch; protocore_plaintext_release(mark) |
secure (mmgr/secure.h) | key material: shared secrets, private scalars, schedules | same, and the release wipes before the position moves |
The two differ in exactly one thing: reclaiming the secure pool zeroes the region before it becomes available again. A secret cannot outlive its borrow. That makes the rule structural instead of a discipline every caller has to remember on every return path. The regions are also disjoint, so protocore_secure_owns() and protocore_plaintext_owns() are mutually exclusive by construction - a secure borrow can never be accepted where a plaintext one is expected, with no tagging and no per-allocation metadata.
Lifetime is not what picks the pool. Both carry long-lived and ephemeral borrows; the only question is whether the bytes are secret.
The caller asks for RAM and gets a pointer, and the RAM is guaranteed to be there. There is no fail-closed branch to write at a call site, because the size was settled before the compiler ran. mark / release are there for lifetime and, on the secure side, for the wipe - not for handling a failure that cannot happen.
The second argument is the alignment, and it is a condition the core sets, not a favor it does. The core publishes what it provides - storage of a declared span, at a declared alignment, for a declared lifetime - and a vendor backend meets those conditions or it does not ship. The obligation runs that way round, and it is settled in test/core_setup/, the only place a vendor type is named at all: the backend static_asserts that its context fits the span and satisfies the alignment. A vendor header that changes underneath us fails the build there, named, instead of becoming a run-time surprise in the core.
Because they meet those conditions, the core behaves as advertised: the pointer is guaranteed, the footprint is a number you computed before flashing, and no call site has a check to write. A consumer writes protocore_secure_alloc(PROTOCORE_WORK_AES128GCM, 8) and is done - the macro settles the size, the backend has already satisfied the requirement, and neither is restated at the point of use.
The arena's own base is the same argument one level down. The pool aligns allocation offsets, so the base must already satisfy the strictest alignment any caller can request; left an ordinary struct member it would inherit only 8, so it is declared _Alignas(32) where the storage lives - stated once. No borrow has to hope the base is good enough.
C does not do this for you: on the secure side every return path must reach protocore_secure_release(), including the early ones taken when a peer sends something malformed. That is where the wipe happens.
The pool sizes are not chosen numbers. Each translation unit precomputes its span - the worst-case bytes it borrows in a single call - and declares it as a PROTOCORE_WORK_<MODULE> constant in protocore_config.h, the only place that can see them all, since every module header includes it.
mmgr therefore has preknowledge of every TU's span before the build runs. It is not handed a request at run time and asked whether it fits; the set of spans it will ever be asked for is an input to how it was sized. That is why the pointer is guaranteed and there is nothing to check.
And every address is preknown too. A module declares a span. It never declares an offset, so nothing couples one module to another - but mmgr resolves that set of spans into a layout at compile time, which makes each TU's base a constant, decided before any run-time bump. Two TUs that the time domain proves are never live together resolve to the same base: that overlap is precisely why the peak-concurrent figure is smaller than the sum, and it costs nothing, because the exclusivity was already known.
A borrow resolves to a known address in known storage. There is no search, no free list, no fragmentation, and no layout decision left to make while the device is running.
Each span is proved where the struct lives, by a static_assert(sizeof(X) <= PROTOCORE_WORK_X) in the module that owns it. A working set that grows past its declaration fails the build naming itself. It never exhausts a pool at run time. The declaration and the truth cannot drift apart.
These are sizes, not offsets. Nothing couples one module to another: each is a term, order is irrelevant, and adding a module shifts no one - the difference from the crypto_work region map they replaced.
The two pools then resolve those terms differently, because their time domains differ:
Both are feature-gated. A build pays only for the code it compiled.
| Macro | Default | What it does |
|---|---|---|
PROTOCORE_WORKER_COUNT | 1 | Number of worker tasks. > 1 partitions slots across cores. Must be <= MAX_CONNS. |
PROTOCORE_WORKER_CORE | 1 | Core that worker 0 pins to; worker k pins to (PROTOCORE_WORKER_CORE + k) % cores. |
PROTOCORE_WORKER_TASK_PRIORITY | 5 | FreeRTOS priority of the worker task(s). |
PROTOCORE_WORKER_TASK_STACK | 8192 | Per-worker task stack (bytes). A build guard requires >= PROTOCORE_WORKER_STACK_RSA_MIN when OIDC or SSH is enabled (RSA-2048 verify needs ~7 KB). |
PROTOCORE_WORKER_STACK_RSA_MIN | 8192 | Enforced floor for PROTOCORE_WORKER_TASK_STACK once an RSA-2048 verifier (OIDC/SSH) is compiled in. Lower it only if you marshal RSA verifies off the worker. |
PROTOCORE_WORKER_POLL_TICKS | 1 | Idle-sweep block timeout (ticks). Events wake the worker immediately regardless; this only sets how often an idle worker wakes to run the timeout sweep. |
EVT_QUEUE_DEPTH | MAX_CONNS * 4 (32) | Per-queue event slots; tracks MAX_CONNS. A raised pool never trips the >= MAX_CONNS * 4 guard. Raise it to absorb larger connection bursts. |
MAX_CONNS | 8 | Connection pool size. The hard ceiling on concurrent connections. |
PROTO_WORD_BITS | 32 | The target's natural register width. Every narrow value is carried in it and truncated at the boundary, because arithmetic narrower than the register costs the mask that keeps the unused half correct. Must be 16, 32 or 64. |
PROTO_INDEX_BITS | 32 | Width of proto_idx, which is every offset, length and capacity the library declares (never size_t, whose width is inherited from the pointer and so differs between a device build and the host test). Must be 16 or 32, and <= PROTO_WORD_BITS. |
PROTO_SWAR_BITS | PROTO_WORD_BITS | Width of the lane carrier the byte-parallel scans and compares work in (mmgr/swar.h). Must be 8, 16, 32 or 64 and <= PROTO_WORD_BITS - a wider carrier is synthesized from halves and is slower than the width it decomposes into. 8 degenerates to one lane per word. |
Every per-feature tuning knob (buffer sizes, table depths, message limits, thresholds) lives in one place: src/protocore_config.h, in the section **"Feature tuning knobs (grouped and gated by feature)"** at the end of the file. You never have to open a feature header to turn one. Each is an override-able default, so you set a new value in your build_flags (for example -D PROTOCORE_OPCUA_READ_MAX=16 or -D PROTOCORE_GQL_MAX_DEPTH=8) and the owning module picks it up. A group is wrapped in its feature's PROTOCORE_ENABLE_* flag. A knob only exists when that feature is compiled in.
What is deliberately not a knob and stays next to its code: protocol- and algorithm-fixed constants (wire opcodes, magic bytes, crypto digest/block sizes, spec-mandated PDU/field widths, and the deflate/inflate scratch sizes a static_assert pins to the table layout). Changing those breaks on-the-wire conformance, so they are not exposed as knobs.
The sizing defaults above are not one flat set. They used to be, tuned to fit the smallest classic-ESP32 DRAM ceiling. A board with far more RAM or flash silently inherited the same cramped numbers. Instead, vendor/board_profiles/ layers defaults along three independent axes, selected in board_profile.h (included first thing in protocore_config.h):
classic_defaults.h (ESP32) beside board_profile.h, and under vendor/board_profiles/esp/ the s2 / s3 / c2 / c3 / c5 / c6 / c61 / h2 / p4 _defaults.h, plus preview targets s31 / h4 / h21 (in ESP-IDF master only). Auto-selected from CONFIG_IDF_TARGET_*; classic ESP32 and host builds use the classic floor. Holds each die's chip-appropriate sizing and its per-die HW-crypto flags - PROTOCORE_HW_AES / _SHA / _RSA / _ECC / _ECDSA / _HMAC / _DS - which are genuinely different across the lineup (e.g. C2/C61 have no general-purpose AES peripheral and no RSA/MPI; C6 has no ECDSA; H4 has no RSA/DS), so gate a HW path on the specific flag, never on "it's an ESP32". Values track each target's ESP-IDF soc_caps.h.esp/2mbpsram.h / 4mbpsram.h / 8mbpsram.h / 16mbpsram.h / 32mbpsram.h. A given chip ships with or without PSRAM, so this is its own axis. Scales the RAM-backed pools up.esp/2mbflash.h / 4mbflash.h / 8mbflash.h / 16mbflash.h / 32mbflash.h. Likewise independent; for flash-backed sizing.Every profile default is #ifndef-guarded, so precedence is first definition wins:
Nothing here overrides a value you set yourself, and the classic ESP32 gets exactly the historical numbers, so no existing board regresses - larger variants just stop being capped to the smallest one. Piloted so far: the edge-cache and mesh pools (PROTOCORE_EDGE_CACHE_SLOTS, PROTOCORE_EDGE_BODY_MAX, PROTOCORE_EDGE_FETCH_SLOTS, PROTOCORE_MESH_MAX_PEERS, PROTOCORE_MESH_MAX_CONNS); more sizing knobs migrate into the profiles over time.
The chip is detected automatically. PSRAM and flash size can't be read reliably from the Arduino core, so set them for your board (they default to "none / smallest"):
ESP-IDF builds fill both in automatically from CONFIG_SPIRAM_SIZE / CONFIG_ESPTOOLPY_FLASHSIZE_*. You can still pin any individual knob with a -D override, which always wins over the profile.
Event latency is decoupled from the idle-sweep cadence. With WiFi power-save off to isolate scheduling, GET /health over 15 requests:
PROTOCORE_WORKER_POLL_TICKS | avg | min | max |
|---|---|---|---|
| 1 | 27.2 ms | 12.4 ms | 35.1 ms |
| 100 | 28.0 ms | 12.5 ms | 42.2 ms |
Identical at a 100x longer idle sweep. The pre-notification poll would have reached one full sweep per request (~50 ms average at POLL_TICKS=100).
Idle worker wakeups scale as tick_rate / PROTOCORE_WORKER_POLL_TICKS. At the Arduino 1 kHz tick that is 1000 / POLL_TICKS wakeups per second with no traffic (1000/s at the default 1, 10/s at 100), each wakeup being one context switch plus one service pass over idle slots. Raising the knob cuts that idle service-loop CPU proportionally at no latency cost. Absolute idle CPU on a real build is dominated by WiFi/IDF housekeeping, so the headline benefit of a high value is fewer wakeups (CPU/power headroom), not a change to request latency.
Parallel throughput (existing benchmark). A CPU-bound handler (~0.2 s) under 4-way concurrency: PROTOCORE_WORKER_COUNT=1 ~5.9 req/s vs =2 ~9.1 req/s (~1.5x). It is not a full 2x because worker 1 shares Core 0 with WiFi/lwIP; single-request latency is unchanged.
loop() freed, events serviced immediately.PROTOCORE_WORKER_POLL_TICKS (e.g. 100 for a ~10 Hz idle sweep). Far fewer idle wakeups, and because events still wake the worker immediately there is no latency cost. Keep it well below your connection timeout so stale connections are still reaped promptly.PROTOCORE_WORKER_COUNT=2 to run handlers on both cores. Expect ~1.5x, not 2x (Core 0 also runs WiFi/lwIP). Ensure MAX_CONNS >= PROTOCORE_WORKER_COUNT and that handlers touch only their own slot's state (the model already guarantees slot isolation).EVT_QUEUE_DEPTH to keep a burst of accepts/data events cannot overflow a queue (an overflow is dropped, not blocked, to keep the tcpip thread non-blocking). Raise MAX_CONNS for more concurrent connections (BSS cost is fixed and linear).PROTOCORE_WORKER_CORE=0 only if your app keeps Core 1 busy; by default Core 1 is the right home (Core 0 carries WiFi/lwIP).PROTOCORE_WORKER_COUNT > 1 adds PROTOCORE_PLAINTEXT_ARENA_SIZE of BSS per extra worker and one event queue per worker; all static.PROTOCORE_WORKER_POLL_TICKS (see src/server/clock/clock.h), so timeouts keep their tested 1 ms granularity.