> ## Documentation Index
> Fetch the complete documentation index at: https://totem-cb8b3887.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# 2.4 GHz protocols overview

> The three radios that share the 2.4 GHz band and how completely each is captured.

The Totem Compass uses the ESP32's single 2.4 GHz radio in three roles. They are
time-multiplexed by the WiFi/BT coexistence layer, never simultaneously active in a way
that conflicts on-air.

<CardGroup cols={3}>
  <Card title="BLE" icon="bluetooth" href="/protocols/ble">
    Phone app ↔ Totem. GATT peripheral, chunked transfer, `(cat_id, cmd_id)` messages.
  </Card>

  <Card title="ESP-NOW mesh" icon="diagram-project" href="/protocols/espnow-mesh">
    Totem ↔ Totem "Unity Mesh". Peer sync and relay — cleartext, no encryption.
  </Card>

  <Card title="WiFi" icon="wifi" href="/protocols/wifi-ota">
    Station-mode OTA. Joins a WiFi network, downloads firmware over plain HTTP (`api.totemportal.com` / S3). The device-hosted hotspot OTA path is sunset.
  </Card>
</CardGroup>

## One message model across radios

The most important protocol finding: BLE and the mesh share a **single application
message model**. Every message is addressed by a two-byte tuple:

```text theme={null}
(cat_id, cmd_id)   +   payload
 │        │
 │        └── command within the category
 └── category (Static Data, Live Data, Peer, …)
```

The BLE link logs it directly — `[BLE] DataXfer cat_id: {} | cmd_id: {}` and
`Conn Status Update | cat_id: {} | cmd_id: {}` — and the same categories drive peer
sync over ESP-NOW. Large BLE file/OTA transfers are fragmented by a shared **chunking**
layer; mesh frames are single fixed-size packets (57 B / 69 B, well under the 250-byte
ESP-NOW MTU). The full model is on the [message format](/protocols/message-format) page.

<Note>
  The `(cat_id, cmd_id)` **addressing** is shared, but the two transports bind each key to a
  **different** payload layout — BLE uses the `gen_*` builders in `ble_manager.py` /
  `ble_core.py`, ESP-NOW uses `TOTEM_MSG_MAP` / `EXTENDED` in `espnow_conn_v2.py`. Same key,
  different bytes: BLE `gen_live_data` `(0x03, 0x01)` packs `<bfi3fb4Bi3b2hbiffb3ibHBBb`
  (69 B), while `TOTEM_MSG_MAP (3,1)` is `<BBHH4b4BHHBb` (20 B). A client must pick the
  format for the transport it is on.
</Note>

## Capture completeness

This table is the direct answer to "is the 2.4 GHz protocol fully captured?" It grades
each element by how firmly the firmware binary supports it.

| Element                                                               | Status                       | Basis                                                                                                                                                                                                                                                     |
| --------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Radios / transports (BLE, ESP-NOW, WiFi)                              | ✅ Confirmed                  | module set, IDF error strings, log lines                                                                                                                                                                                                                  |
| Message addressing `(cat_id, cmd_id)`                                 | ✅ Confirmed                  | `[BLE] DataXfer cat_id/cmd_id`, `ConnStatus` logs                                                                                                                                                                                                         |
| Category values 0x01 / 0x03 / 0x06                                    | ✅ Confirmed                  | `DataTransfer (0x01,0x02)` Static, `(0x03,0x01)` Live, `(0x06,0x07)` Peer Sync                                                                                                                                                                            |
| Category taxonomy (Static/Live/Peer/Conn/OTA/WiFi/Options/DemiGod)    | ✅ Confirmed                  | `*_cmd_id` symbols + ACK log lines                                                                                                                                                                                                                        |
| GATT service & characteristic UUIDs                                   | ✅ Confirmed                  | literal UUID strings in rodata; v5.0.3 adds `…-0003` (`chars__on_demand`, file-upload chunks)                                                                                                                                                             |
| BLE connection handshake (ConnStatus Ready, half-duplex, handoff)     | ✅ Confirmed                  | `send_data_v2` log lines                                                                                                                                                                                                                                  |
| Chunking / file-transfer semantics (file\_id, chunk\_no, resume, ACK) | ✅ Confirmed                  | `[Upload]` + `Chunk No:` log lines                                                                                                                                                                                                                        |
| ESP-NOW relay                                                         | ✅ Confirmed                  | `_relay_frame`, hop-count fields                                                                                                                                                                                                                          |
| ESP-NOW encryption: **there is none**                                 | ✅ Confirmed                  | broadcast **and** unicast are cleartext — `set_pmk`/`lmk`/`encrypt` qstrs appear in no frozen module; `add_peer(mac)` is called positionally → `encrypt=False`; no PMK is ever programmed                                                                 |
| ESP-NOW frame integrity: **no CRC/checksum**                          | ✅ Confirmed                  | validation is SyncWord (`0xA7 0x74`) + `len >= 4` + per-`(cat,cmd)` size check only; the `Invalid Checksum value for: {}` string belongs to the u-blox UBX GNSS parser, not this path                                                                     |
| BLE / application-layer auth: **none required**                       | ✅ Confirmed                  | characteristics register flags `0x3E` only, no `ble.config()` security, no passkey/encryption IRQ events; no token/HMAC/challenge on any message path                                                                                                     |
| Demi-god broadcast command set                                        | ✅ Confirmed                  | `demigod_gen_*` symbols                                                                                                                                                                                                                                   |
| Wire struct layouts (catalog of `struct` formats)                     | ✅ Confirmed                  | format strings in rodata                                                                                                                                                                                                                                  |
| **Exact `cmd_id` enumeration per category** (`TOTEM_MSG_MAP`)         | ✅ Recovered                  | flat `(cat_id, cmd_id)` dict decoded from `espnow_conn_v2.py` bytecode (see below); adds categories `0x00` / `0x02` / `0x07`                                                                                                                              |
| **Which `struct` format binds to which message**                      | ✅ Recovered                  | bindings decoded from `TOTEM_MSG_MAP` / `EXTENDED` (see below)                                                                                                                                                                                            |
| **Chunk / transfer headers**                                          | ✅ Recovered                  | transfer header `<HBBBiHiB` (16 B) from the live `struct.pack_into` in `chunking.gen_transfer_buff`; v5.0.3 defines `CHUNK_HDR_FMT = '<HHiH'` / `CHUNK_HDR_SZ = 12` for the separate per-chunk header (v5.0.2 imported those names without defining them) |
| BLE transmit modes                                                    | ✅ Confirmed, hardware-tested | `frame_schema_id == 0` runs the legacy `send_data` loop (notifications plus app acks); `> 0` runs half-duplex `send_data_v2` (indications with a 500 ms confirmation timeout), which stalls on macOS/iOS                                                  |

### Recovered from bytecode

Frozen-MicroPython disassembly (v1.25.0 / .mpy v6.3) has since decoded the three
formerly-pending items. `TOTEM_MSG_MAP` is a **flat** dict built in `espnow_conn_v2.py`,
keyed by the 2-byte `(cat_id, cmd_id)`; **every value is a payload `struct` format** — the
mesh frame layout for that message.

<Warning>
  An earlier pass decoded the qstr-immediate operands with `>>2` and produced bogus
  "handler names" (`disconn_animation`, `device_power`, `dev_info`,
  `dev_total_lightsleep_ms`, `disabled`, `dev_sampling`, …). Those were **decode artifacts
  and are wrong.** ESP32 MicroPython's REPR\_A tags a qstr-immediate as `(o & 7) == 2` with
  value `o >> 3`; under the correct `>>3` decode **every** entry resolves to a `struct`
  format string, shown below. There are no handler-name values in the map.
</Warning>

| `(cat_id, cmd_id)`                               | `struct` format      | size (B) |
| ------------------------------------------------ | -------------------- | -------- |
| `(0x00, 0x00)` / `(0x00, 0x01)` / `(0x00, 0x02)` | `<BBffbbhbbb6B`      | 23       |
| `(0x01, 0x00)`                                   | `<BBB7bHbb`          | 14       |
| `(0x01, 0x02)`                                   | `<BBiffHii`          | 24       |
| `(0x01, 0x05)`                                   | `<BB9B8bhh4B`        | 27       |
| `(0x01, 0x06)`                                   | `<BBbbB`             | 5        |
| `(0x01, 0x07)`                                   | `<3Bbb`              | 5        |
| `(0x02, 0x00)`                                   | `<BB6BffbbHbbbhhBBi` | 33       |
| `(0x03, 0x01)` / `(0x03, 0x02)`                  | `<BBHH4b4BHHBb`      | 20       |
| `(0x07, 0x00)`                                   | `<BBffbbbHbbH`       | 19       |
| `(0x07, 0x01)`                                   | `<BBHbffb`           | 14       |

Every `<BB…` format leads with the echoed `(cat_id, cmd_id)`. The `(0x00, *)` 23-byte
family is the mesh peer beacon (2 floats lat/lon + flags + a 6-byte MAC). Field-by-field
mesh semantics are **partial** — the layouts are confirmed but the fields are not all
labelled.

A parallel `EXTENDED` map holds schema-versioned variants, keyed
`(cat, cmd) → {schema_id → fmt}`:

| `(cat_id, cmd_id)` | schema | `struct` format                     | size (B) |
| ------------------ | ------ | ----------------------------------- | -------- |
| `(0x00, 0x00)`     | 25     | `<BBffbbhbbb6B`                     | 23       |
| `(0x00, 0x00)`     | 59     | `<BBffbbhbbb6Bhii4BhHehhffbB`       | 57       |
| `(0x00, 0x00)`     | 72     | `<BBffbbhbbb6Bhii4BhHehhffbBBiiBBb` | 69       |
| `(0x01, 0x06)`     | 5      | `<BBbbB`                            | 5        |
| `(0x01, 0x06)`     | 29     | `<BBbb9BBB3i`                       | 27       |
| `(0x02, 0x00)`     | 45     | `<BB6BffbbHbbbhhBBiffh`             | 43       |

The 57-B and 69-B `(0x00, 0x00)` schemas are the fixed-size mesh frames. Category `0x06`
(Peer Sync) is **absent** from this registry — it is handled by separate peer-sync logic,
not the payload-format map.

**Chunk headers.** `chunking.gen_transfer_buff` packs the 16-byte **transfer** header
`<HBBBiHiB` at offset 2 of a `[0x02, 0x02, …]` frame: `file_id` (H), `status_id` (B),
`action_id` (B), `file_type_id` (B), `byte_pos` (i), `chunk_no` (H), `file_size` (i), then a
byte that is literal `0` in v5.0.2 and `pack_flags(is_last_chunk, is_origin_compass)` in
v5.0.3. The name `CHUNK_HDR_FMT` belongs to a different header. v5.0.3 binds it to the
12-byte per-chunk header `'<HHiH'` streamed on `…-0003`. v5.0.2 imported the name without
defining it, so its uploader was dead code. See [chunking](/protocols/message-format#chunking).

<Note>
  **Verdict.** The protocol's transport, addressing scheme, category taxonomy, GATT
  identifiers, connection state machine, chunking/file-transfer, and the demi-god command
  set are **fully captured**. The three items that were previously pending in bytecode — the
  per-category `cmd_id` map (`TOTEM_MSG_MAP`), the `struct`-to-message bindings, and the
  chunk-header layout — have now been recovered by disassembling the frozen
  MicroPython modules: `TOTEM_MSG_MAP` and its bindings live in `espnow_conn_v2.py`, and
  the chunk headers are packed in `f_ble/chunking.py` and `f_ble/file_upload.py` (see the tables above).

  Crucially, **there is no confidentiality or integrity layer to defeat**: ESP-NOW is
  cleartext on both broadcast and unicast (no PMK/LMK ever programmed), the ESP-NOW frame
  carries **no CRC or checksum** (SyncWord + length + size-check only), BLE needs no
  pairing, and no message path carries a token, HMAC, or signature. A client interoperates
  by matching the wire format alone. See [methodology](/reference/methodology).
</Note>

## Coexistence

ESP-IDF's WiFi/BT coexistence arbitrates the radio. The mesh runs on a fixed home
channel; a peer whose channel differs is rejected (`Peer channel is not equal to the
home channel, send fail!`). During a hotspot OTA the device is a WiFi station on the
phone's channel, and BLE/mesh traffic is suspended for the duration of the download.

## Protocol lineage & prior art

Totem's protocol is **not** a single named standard, and it is **not** a novel scheme
either. It is a **custom application layer assembled on top of standard, open transports**,
using conventional design patterns rather than anything proprietary or exotic.

### Standard / open building blocks (reused)

| Layer                 | Component                                        | Origin                                                                           |
| --------------------- | ------------------------------------------------ | -------------------------------------------------------------------------------- |
| BLE host stack        | NimBLE (via MicroPython `bluetooth`)             | open source (Apache-2.0)                                                         |
| BLE app transport     | GATT write/notify pair with custom 128-bit UUIDs | Bluetooth SIG standard; the "serial over GATT" pattern (cf. Nordic UART Service) |
| Totem-to-Totem radio  | ESP-NOW via MicroPython `espnow` / `aioespnow`   | Espressif's documented connectionless protocol; `aioespnow` is open source       |
| Cloud / OTA transport | HTTP/1.0 + WebSocket                             | web standards                                                                    |
| Firmware update       | ESP-IDF dual-slot OTA + rollback                 | open source (Espressif)                                                          |
| GNSS                  | u-blox UBX binary protocol                       | documented vendor protocol                                                       |
| Geomagnetism          | World Magnetic Model 2025 (`WMM_2025`)           | public NOAA NCEI / BGS model                                                     |
| Sensor / LED drivers  | `neopixel`, `onewire` / `ds18x20`                | open-source MicroPython drivers                                                  |

### Custom (Totem-authored) layer

* **Wire framing** — a SyncWord (`0xA774`) + `(cat_id, cmd_id)` + `struct.pack`-ed binary
  payloads. This is the conventional "magic + type + fixed struct" idiom; there is **no
  serialization library on the air** (no Protobuf, CBOR, msgpack, or JSON — JSON appears
  only in the HTTP/OTA config, and WebSocket only for the cloud OTA trigger).
* **Mesh ("Unity Mesh")** — a custom **managed-flooding** design over ESP-NOW broadcast:
  flood + relay suppression (overhearing / slots) + message-UID dedup + hop limit. The
  *pattern* is well known (the same family as BLE Mesh's managed flooding); the
  implementation is Totem's own Python.
* **Chunked file transfer** — a generic `file_id` / `chunk_no` / resume / ACK / SHA-256
  scheme, not a named protocol.

### What it is explicitly NOT

The disassembly rules out the usual suspects: **not** Bluetooth **BLE Mesh**; **not**
Espressif **esp-wifi-mesh** (the SDK's `WIFI_MESH_EVENT` symbols are present but unused);
**not** Thread, Zigbee, Matter, or 6LoWPAN; **not** painlessMesh; **not** MQTT or CoAP.
There is also no application-layer cryptography — traffic is cleartext, and the only
obfuscation is a trivial `rot13` (not encryption).

<Note>
  **In short:** a bespoke command-and-mesh protocol built from standard, open components,
  with a Totem-specific binary message layer on top. A client interoperates by speaking the
  standard transports plus Totem's custom framing — there is no proprietary or exotic
  protocol to reimplement. See [Building a custom client](/reference/building-a-client).
</Note>