> ## Documentation Index
> Fetch the complete documentation index at: https://totem-cb8b3887.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Building a custom client

> A practical reference for implementing your own client that talks to your own Totems over BLE, the ESP-NOW mesh, and a custom OTA server — no keys required.

This page collects everything a from-scratch client needs to talk to Totems you own:
the BLE GATT link the phone app uses, the ESP-NOW mesh frame, a drop-in OTA server, and
the chunked-transfer header. Every value here is verified against the recovered
`firmware_v5.0.3.bin` image (and unchanged from v5.0.2 unless marked). The BLE parts are
also tested on hardware. Confidence is graded **Confirmed** / **Inferred** / **Partial**
throughout.

## No cryptographic secrets are required

The single most important fact for a client author: **you need no keys, no pairing, no
signatures, and no shared secret** to interoperate. Nothing on the application path is
authenticated or encrypted (**Confirmed**).

| Path                                | Protection a client must handle                                                   |
| ----------------------------------- | --------------------------------------------------------------------------------- |
| ESP-NOW (broadcast **and** unicast) | none — entirely cleartext, no PMK/LMK ever programmed                             |
| BLE GATT                            | none — no pairing/bonding/encryption; an unpaired central can read and write      |
| Application messages (BLE + mesh)   | none — no token, HMAC, challenge/response, or signature anywhere                  |
| OTA                                 | integrity only — SHA-256 hash, **no** firmware signature, fetched over plain HTTP |

<Note>
  The ESP-NOW ROM module *contains* the concept of encryption (`set_pmk`, `lmk`,
  `encrypt`), but no frozen Python module references those symbols, and `add_peer(mac)` is
  always called with a single positional argument, so `lmk=None` and `encrypt=False`. The
  only crypto in the image is the bundled (and, on the app path, unused) mbedTLS stack plus
  the OTA SHA-256. `ble_keys.bin` holds optional BLE reconnect secrets — not required to
  connect.
</Note>

## BLE client

The Totem is a BLE **peripheral**; your client is the **central**. No pairing is required.
A working reference implementation in Go, `totemctl`, lives in this repository
(`protocol/`, `client/`, `cmd/totemctl/`). It has been verified against a real device on
firmware 4.1.3 and 5.0.3.

### Connect

Bluetooth is off until the user **double-presses the power button**, which toggles
advertising; the crystal breathes blue while it advertises. The device advertises the name
`totem` and the service UUID, so either can be matched.

| Item              | Value                                  |
| ----------------- | -------------------------------------- |
| Service UUID      | `7913b588-0000-4635-b066-baa2cfc197cf` |
| Advertised name   | `totem` (appearance 1361)              |
| Pairing / bonding | not required                           |

`ble_core.py` registers these characteristics under the service, all with GATT flags
`0x3E` (READ | WRITE | WRITE\_NO\_RESP | NOTIFY | INDICATE):

| Characteristic UUID                    | Name                   | Role                                                                                                |
| -------------------------------------- | ---------------------- | --------------------------------------------------------------------------------------------------- |
| `7913b588-0001-4635-b066-baa2cfc197cf` | `chars__conn_status`   | control / handshake: the client writes here, and the device announces handoffs and disconnects here |
| `7913b588-0002-4635-b066-baa2cfc197cf` | `chars__data_transfer` | application data                                                                                    |
| `7913b588-0003-4635-b066-baa2cfc197cf` | `chars__on_demand`     | **v5.0.3+ only**: file-upload chunks, used only if the client announced upload support              |

Direction is fixed: **client → device is a GATT write**; **device → client is
notify/indicate** (on connection handle 0). Subscribe to **both** `…-0001` and `…-0002`. The
data characteristic's GATT value buffer is **185 bytes**, so a single write must fit in it.
MTU is client-negotiated; macOS negotiates 256 with the device.

### Handshake (ConnStatus-Ready)

<Steps>
  <Step title="Subscribe">
    Enable notifications on `…-0002` and `…-0001`.
  </Step>

  <Step title="Report app state (optional)">
    Write `[0x03, 0x00, bits]` to `…-0001`: bit0 isActive, bit1 isFocused, bit2 isLocked,
    bit3 isUiClosed, bit4 isService, bit5 lets the device drop BLE on its own schedule.
  </Step>

  <Step title="Write the Ready frame">
    Write `[0x00, 0x01, 0x01, 0x00]` to `…-0001` **within 15 s** of connecting (v5.0.3 drops
    silent links). A last byte of `0x00` selects the **legacy full-duplex** transmit loop,
    which works on every platform. A nonzero `frame_schema_id` selects the half-duplex loop,
    which [stalls on macOS/iOS](/protocols/ble#transmit-modes). v5.0.3 accepts an optional 5th
    byte whose bit 0 announces file-upload support; leave it out unless you implement uploads.
  </Step>

  <Step title="Request and acknowledge records">
    Write `(0x01, 0x01)` to `…-0002` to request Static Data. In the legacy loop the device
    repeats Static Data, the WiFi list and Peer Sync until you acknowledge them with
    `(0x01, 0x00)`, `(0x02, 0x00)` and a cat-6 command such as `(0x06, 0x08)`. Live Data then
    arrives every \~3 s, and Peer Pings whenever a peer changes. Read them with the `gen_*`
    layouts below.
  </Step>

  <Step title="Send commands, then disconnect">
    Write commands to `…-0002` at any time; the full list is in the
    [command map](/protocols/message-format#categories). To disconnect gracefully, write
    `[0x00, 0x03]` to `…-0001`.
  </Step>
</Steps>

<Info>
  `recv_status_msgs` applies these gates (**Confirmed**): `conn_mode == 1` sets TX owner and
  clears pending static/peer requests; `frame_schema_id > 0` sets TX ready and makes the
  legacy loop exit in favour of `send_data_v2`. The ids **25, 45, 59, 72** are real `EXTENDED`
  schema variants, but no BLE record layout depends on the schema id. In the half-duplex
  loop the device waits for the ATT confirmation of every indication (500 ms timeout), hands TX
  to the app with `[0x04, 0x02, 0x02, …]` on `…-0001`, and expects `[0x04, 0x03, 0x02]` to
  take it back. See [Half-duplex & comms handoff](/protocols/ble#half-duplex--comms-handoff).
</Info>

### Messages to read (BLE `gen_*` layouts)

<Warning>
  The BLE GATT layouts are **not** the ESP-NOW layouts. The same `(cat_id, cmd_id)` key is
  reused by two transports with different struct layouts — for example BLE `gen_live_data`
  `(0x03,0x01)` is a 69-byte record, while `TOTEM_MSG_MAP (3,1)` is 20 bytes. A BLE client
  must use the `gen_*` layouts here; a mesh client must use the [mesh map](#esp-now-mesh-client).
</Warning>

All BLE records prefix `buff[0:2] = (cat_id, cmd_id)`. **Confirmed** layouts:

**Live Data `(0x03, 0x01)`** — `gen_live_data`, `struct <bfi3fb4Bi3b2hbiffb3ibHBBb` at
offset 2 (69 B). Fields in order:

| #     | code | Field                                 |
| ----- | ---- | ------------------------------------- |
| 1     | `b`  | sat\_count                            |
| 2     | `f`  | p\_acc, m (−1 = None)                 |
| 3     | `i`  | altitude, m (−500 = None)             |
| 4     | `f`  | latitude, °                           |
| 5     | `f`  | longitude, °                          |
| 6     | `f`  | batt\_volts, V                        |
| 7     | `b`  | esp-now channel                       |
| 8     | `B`  | power\_bits (power\_mode in bits 0-2) |
| 9     | `B`  | max\_hop\_cnt                         |
| 10    | `B`  | mesh\_rx                              |
| 11    | `B`  | mesh\_relayed                         |
| 12    | `i`  | unix\_ts, s                           |
| 13    | `b`  | color\_id                             |
| 14    | `b`  | orientation                           |
| 15    | `b`  | solution\_id                          |
| 16    | `h`  | heading, °                            |
| 17    | `h`  | azimuth, °                            |
| 18    | `b`  | speed (cap 127, −1 = None)            |
| 19    | `i`  | odometer                              |
| 20-21 | `ff` | reserved ×2 (always 0)                |
| 22    | `b`  | reserved (always −1)                  |
| 23    | `i`  | uptime, s                             |
| 24    | `i`  | age                                   |
| 25    | `i`  | reserved (always 0)                   |
| 26    | `b`  | power\_level: battery health, 2 = low |
| 27    | `H`  | reserved (always 0)                   |
| 28    | `B`  | flags                                 |
| 29    | `B`  | reserved (always 0)                   |
| 30    | `b`  | batt\_pct                             |

The `flags` byte (field 28) is `pack_flags(is_sos, is_eco, led_brt ≥ GLOBAL_BRT,
gnss_location_set, power_level == 2, is_charging, 0, is_mag_cal_needed)`: bit 2 is normal
(undimmed) brightness and bit 4 is low battery.

The reserved fields hold these constants in every published firmware (3.2.12 to 5.0.3), and
the official app reads and discards them, so a client can ignore them.

**Static Data `(0x01, 0x02)`** — `gen_static_data`. `buff[2] = total_len & 255`,
`buff[3:9] = MAC` (6 B), then `struct <biHBBBbBBBbhhiiibbb` at offset 9 (34 B), then three
UTF-8 strings concatenated from offset 43. There are no per-string prefixes: their lengths
are the struct's last three fields.

| Region         | Field                                                                                       |
| -------------- | ------------------------------------------------------------------------------------------- |
| `buff[2]`      | total\_len & 255                                                                            |
| `buff[3:9]`    | device MAC (6 B)                                                                            |
| struct @9: `b` | reserved (always 0)                                                                         |
| `i`            | age                                                                                         |
| `H`            | release\_id                                                                                 |
| `B B B`        | ver\_major, ver\_minor, ver\_patch                                                          |
| `b`            | color\_id                                                                                   |
| `B`            | settings\_flags (bit0 persistent north, bit1 compass lock; the app reads bit3 as bond chat) |
| `B`            | capabilities (bit0 = half-duplex loop; 1 in 5.x, 0 in 4.1.3)                                |
| `B`            | service\_id                                                                                 |
| `bhhiii`       | 6× reserved (always 0; the app discards them)                                               |
| `b b b`        | len(device\_name), len(branch), len(wifi\_ssid)                                             |
| strings @43    | device\_name, git branch (`N/A`), wifi\_ssid (`""`)                                         |

**Peer Ping `(0x06, 0x02)`** — `gen_peer_ping`. `buff[2]` = total length, `buff[3:9]` =
peer MAC (6 B), `buff[9]` = mesh hops, then `struct <ffbbh4BHbb4BiihBbf` at offset 10 (40 B):
lat, lon, p\_acc, speed, bearing, flag byte A, r, g, b, 0 (the app's `dtim`), name length, rssi, msg\_rx, msg\_tx,
mesh\_rx, mesh\_send\_count, last\_update, last\_coords\_unix, distance\_diff, flag byte B,
orientation, volts. Then the peer name (from offset 50) and `<bH` = (batt\_pct, release\_id).
A = `pack_flags(sos, is_poi, is_mesh, is_stale, is_collected, 0, is_unknown, 0)`,
B = `pack_flags(is_hidden, is_locked, 0×6)`. The app reads A's bit 5 as `isIdle`; the
firmware always sends 0 there. `is_unknown` is set once a peer has had no
coordinates for 2 h (v5.0.3; 4 h in v5.0.2).

**Peer Sync `(0x06, 0x07)`** — `gen_peer_sync`: a peer-MAC list,
`[0x06, 0x07, total_len & 255, peer_count, mac0(6), mac1(6), …]`.

## ESP-NOW mesh client

To join the mesh peer-to-peer instead of going through a phone, send/receive raw ESP-NOW
frames on the fleet's channel.

### Radio setup

| Item       | Value                                                                                 |
| ---------- | ------------------------------------------------------------------------------------- |
| Channel    | **6** (`self.channel = 6` default; also settable at runtime via `set_channel`)        |
| PHY        | `WIFI_PROTOCOL_LR` (Espressif Long Range, protocol bit 8) in ESP-NOW-only / BLE modes |
| LR rate    | **250 K**: `EspConn` calls `e.config(rate=41)` (`WIFI_PHY_RATE_LORA_250K`)            |
| Encryption | none — `add_peer(mac)` with no `lmk`                                                  |

The fleet is pinned to a single channel; ESP-IDF drops off-channel frames, so a mesh client
**must** match it. Channel 6 is the built-in default (**Confirmed**), but the live value can
be changed at runtime (**Partial**) — see [what to confirm on-device](#still-missing--confirm-on-device).

### Frame layout

```text theme={null}
┌──────────┬──────────┬────────┬────────┬───────────────────────────┐
│ SyncWord │ SyncWord │ cat_id │ cmd_id │ payload (struct-packed)   │
│  0xA7    │  0x74    │  u8    │  u8    │ variable                  │
└──────────┴──────────┴────────┴────────┴───────────────────────────┘
  [0]        [1]        [2]      [3]      [4:]
```

* **SyncWord** = the 2 bytes `0xA7 0x74` (literal in the image).
* Validation is **SyncWord match + `len >= 4` + a per-`(cat, cmd)` payload-size check**.
* There is **no CRC or checksum** on the ESP-NOW frame. (The rodata string
  `Invalid Checksum value for: {}` belongs to the u-blox UBX GNSS parser, not this path.)
* Mesh dedup UID is a `uint16 randint(1, 65534)` at mesh-frame offset 20;
  `MSG_EXP_MSECS = 150000` ms.

### Payload formats (`TOTEM_MSG_MAP`, corrected)

<Warning>
  Earlier analyses decoded the map's qstr-immediates with `>>2`, producing bogus "handler
  names" (`disconn_animation`, `device_power`, `dev_info`, `disabled`, …). Those are decode
  artifacts and are **wrong**. ESP32 MicroPython (REPR\_A) tags qstr-immediates as
  `(o & 7) == 2` with value `o >> 3`; under the correct `>>3` decode **every** value is a
  `struct` format string. The corrected table follows (**Confirmed**).
</Warning>

Every payload leads with the echoed `(cat_id, cmd_id)` as `BB`.

| `(cat, cmd)`                | Format               | Size |
| --------------------------- | -------------------- | ---- |
| `(0,0)` / `(0,1)` / `(0,2)` | `<BBffbbhbbb6B`      | 23 B |
| `(1,0)`                     | `<BBB7bHbb`          | 14 B |
| `(1,2)`                     | `<BBiffHii`          | 24 B |
| `(1,5)`                     | `<BB9B8bhh4B`        | 27 B |
| `(1,6)`                     | `<BBbbB`             | 5 B  |
| `(1,7)`                     | `<3Bbb`              | 5 B  |
| `(2,0)`                     | `<BB6BffbbHbbbhhBBi` | 33 B |
| `(3,1)` / `(3,2)`           | `<BBHH4b4BHHBb`      | 20 B |
| `(7,0)`                     | `<BBffbbbHbbH`       | 19 B |
| `(7,1)`                     | `<BBHbffb`           | 14 B |

`EXTENDED` schema-versioned variants — `(cat, cmd) → {schema_id → fmt}`:

| `(cat, cmd)` | Schema id | Format                              | Size |
| ------------ | --------- | ----------------------------------- | ---- |
| `(0,0)`      | 25        | `<BBffbbhbbb6B`                     | 23 B |
| `(0,0)`      | 59        | `<BBffbbhbbb6Bhii4BhHehhffbB`       | 57 B |
| `(0,0)`      | 72        | `<BBffbbhbbb6Bhii4BhHehhffbBBiiBBb` | 69 B |
| `(1,6)`      | 5         | `<BBbbB`                            | 5 B  |
| `(1,6)`      | 29        | `<BBbb9BBB3i`                       | 27 B |
| `(2,0)`      | 45        | `<BB6BffbbHbbbhhBBiffh`             | 43 B |

The `(0,*)` 23-byte family is the mesh peer beacon: two floats (lat/lon), flags, and a
6-byte MAC. Field-by-field mesh semantics beyond the struct layout are **Partial** — the
byte layouts are recovered, but individual fields are not fully labelled.

## Custom OTA server

You can point a Totem at your own OTA server; it needs **no signing key** (SHA-256 only,
plain HTTP). The contract (**Confirmed**):

<Steps>
  <Step title="Device announces">
    `POST http://api.totemportal.com/devices/{MAC}/ota` (plain HTTP, no auth) with JSON body
    `{version, endpoint_id, device_type_id, lat, lon, gnss_time, release_id}`.
  </Step>

  <Step title="Server replies with a release object">
    Expose `ota_url, version, product, branch, release_code, release_id`.
  </Step>

  <Step title="Device fetches the manifest">
    `GET {ota_url}/contents.json` — a JSON **array** of filenames. The device picks the
    entry ending `.bin` (firmware) or `.tgz` (preview).
  </Step>

  <Step title="Download, verify, flash">
    The device downloads the chosen file, verifies **SHA-256 only**, flashes, reboots, and
    reports back with `POST …/ota?updated`.
  </Step>
</Steps>

<Note>
  A device WebSocket to `api.totemportal.com` (`ws://`) carries push OTA triggers shaped
  like `{"cmd":…}` (**Partial**). A demi-god ESP-NOW OTA trigger also exists
  (`demigod_gen_ota_update`), but its exact `(cat, cmd)` and struct are **not recovered** —
  do not fabricate them.
</Note>

## Chunked transfer

Only needed if your client accepts the device's **log uploads** (v5.0.3) or does BLE OTA.
A client that sends the plain 4-byte Ready frame never receives uploads. Layouts
(**Confirmed**; details in [chunking](/protocols/message-format#chunking)):

* **Transfer header** (announce / finish), indicated on `…-0001`: `[0x02, 0x02]` +
  `struct '<HBBBiHiB'` = file\_id, status\_id, action\_id, file\_type\_id, byte\_pos, chunk\_no,
  file\_size, flags (v5.0.3: bit0 last chunk, bit1 from compass). Then the 32-byte SHA-256 at
  `buff[18:50]`, the name length at `buff[50]`, the name, and `err_no`.
  `file_id = sha256[0] | sha256[1] << 8`.
* **Chunk** (v5.0.3), notified on `…-0003`: `[0x00, 0x02]` + `struct '<HHiH'` = file\_id,
  length, byte\_pos, chunk\_no, then the data (`CHUNK_HDR_SZ = 12`).
* **App reply**, written to `…-0001` (v5.0.3): `(0x02, 0x03)` + `struct '<HbBBiHiB'`, 18 bytes
  in total. It is read as file\_id, status, action, chunk: `status ∈ {2, 3, 4}` is terminal,
  `action 4` = resume at `chunk`, `action 1` = ready.

| Field          | Enum values                                                                                |
| -------------- | ------------------------------------------------------------------------------------------ |
| `status_id`    | 1 = normal / in-progress, 4 = error / abort                                                |
| `action_id`    | 0 = header / announce, 1 = last-chunk / complete                                           |
| `file_type_id` | 2 = upload-to-app (destinations `SAVE_TO_VFS = 1`, `SAVE_TO_OTA = 2`, `UPLOAD_TO_APP = 3`) |

Overall result codes: `1 done, 2 no-app, 3 retry, 4 failed, 5 cancel, 6 abort`.

## Still missing / confirm on-device

These are known-structure-but-unconfirmed items. Verify them against your own hardware
before relying on them:

| Item                                     | Status                                                                                                           | How to confirm                                                                               |
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| Home channel                             | default 6 (**Confirmed**), runtime-settable (**Partial**)                                                        | sniff the fleet — the live channel can be changed via `set_channel`                          |
| `frame_schema_id` / transmit mode        | **Confirmed and hardware-tested**: `0` = legacy loop (works on macOS), `> 0` = half duplex (stalls on macOS/iOS) | `totemctl --trace info`, or add `--half-duplex` to compare                                   |
| Half duplex on Linux / Android / Windows | **Untested** (those stacks confirm indications, so it should work)                                               | `totemctl --half-duplex --trace watch` on such a host                                        |
| Mesh payload field semantics             | struct layouts **Confirmed**, per-field meaning **Partial**                                                      | correlate captured frames against known device state                                         |
| Demi-god OTA frame                       | **Partial / missing**                                                                                            | `(cat, cmd)` + struct not recovered — do not fabricate                                       |
| Runtime-only values                      | **Partial**                                                                                                      | venue code, group id, and device name live in runtime state / bytecode, not the static image |

## Safety

<Warning>
  Interoperate only with **Totems you own.** Because the mesh is unauthenticated and
  unencrypted, the demi-god ESP-NOW broadcast path can affect **nearby** devices — its only
  gate is RSSI proximity plus message-UID dedupe, with no signature or HMAC. Do not send
  demi-god or mesh commands against devices you do not own; broadcasting near other people's
  Totems can change their behaviour without their consent.
</Warning>