Building a custom client

This page collects everything a from-scratch client needs to talk to Totems you own: the BLE GATT link the phone app uses, the ESP-NOW mesh frame, a drop-in OTA server, and the chunked-transfer header. Every value here is verified against the recovered firmware_v5.0.3.bin image (and unchanged from v5.0.2 unless marked). The BLE parts are also tested on hardware. Confidence is graded Confirmed / Inferred / Partial throughout.

No cryptographic secrets are required

The single most important fact for a client author: you need no keys, no pairing, no signatures, and no shared secret to interoperate. Nothing on the application path is authenticated or encrypted (Confirmed).

Path	Protection a client must handle
ESP-NOW (broadcast and unicast)	none — entirely cleartext, no PMK/LMK ever programmed
BLE GATT	none — no pairing/bonding/encryption; an unpaired central can read and write
Application messages (BLE + mesh)	none — no token, HMAC, challenge/response, or signature anywhere
OTA	integrity only — SHA-256 hash, no firmware signature, fetched over plain HTTP

The ESP-NOW ROM module contains the concept of encryption (set_pmk, lmk, encrypt), but no frozen Python module references those symbols, and add_peer(mac) is always called with a single positional argument, so lmk=None and encrypt=False. The only crypto in the image is the bundled (and, on the app path, unused) mbedTLS stack plus the OTA SHA-256. ble_keys.bin holds optional BLE reconnect secrets — not required to connect.

BLE client

The Totem is a BLE peripheral; your client is the central. No pairing is required. A working reference implementation in Go, totemctl, lives in this repository (protocol/, client/, cmd/totemctl/). It has been verified against a real device on firmware 4.1.3 and 5.0.3.

Connect

Bluetooth is off until the user double-presses the power button, which toggles advertising; the crystal breathes blue while it advertises. The device advertises the name totem and the service UUID, so either can be matched.

Item	Value
Service UUID	`7913b588-0000-4635-b066-baa2cfc197cf`
Advertised name	`totem` (appearance 1361)
Pairing / bonding	not required

ble_core.py registers these characteristics under the service, all with GATT flags 0x3E (READ | WRITE | WRITE_NO_RESP | NOTIFY | INDICATE):

Characteristic UUID	Name	Role
`7913b588-0001-4635-b066-baa2cfc197cf`	`chars__conn_status`	control / handshake: the client writes here, and the device announces handoffs and disconnects here
`7913b588-0002-4635-b066-baa2cfc197cf`	`chars__data_transfer`	application data
`7913b588-0003-4635-b066-baa2cfc197cf`	`chars__on_demand`	v5.0.3+ only: file-upload chunks, used only if the client announced upload support

Direction is fixed: client → device is a GATT write; device → client is notify/indicate (on connection handle 0). Subscribe to both …-0001 and …-0002. The data characteristic’s GATT value buffer is 185 bytes, so a single write must fit in it. MTU is client-negotiated; macOS negotiates 256 with the device.

Handshake (ConnStatus-Ready)

Enable notifications on …-0002 and …-0001.

Report app state (optional)

Write [0x03, 0x00, bits] to …-0001: bit0 isActive, bit1 isFocused, bit2 isLocked, bit3 isUiClosed, bit4 isService, bit5 lets the device drop BLE on its own schedule.

Write the Ready frame

Write [0x00, 0x01, 0x01, 0x00] to …-0001 within 15 s of connecting (v5.0.3 drops silent links). A last byte of 0x00 selects the legacy full-duplex transmit loop, which works on every platform. A nonzero frame_schema_id selects the half-duplex loop, which stalls on macOS/iOS. v5.0.3 accepts an optional 5th byte whose bit 0 announces file-upload support; leave it out unless you implement uploads.

Request and acknowledge records

Write (0x01, 0x01) to …-0002 to request Static Data. In the legacy loop the device repeats Static Data, the WiFi list and Peer Sync until you acknowledge them with (0x01, 0x00), (0x02, 0x00) and a cat-6 command such as (0x06, 0x08). Live Data then arrives every ~3 s, and Peer Pings whenever a peer changes. Read them with the gen_* layouts below.

Send commands, then disconnect

Write commands to …-0002 at any time; the full list is in the command map. To disconnect gracefully, write [0x00, 0x03] to …-0001.

recv_status_msgs applies these gates (Confirmed): conn_mode == 1 sets TX owner and clears pending static/peer requests; frame_schema_id > 0 sets TX ready and makes the legacy loop exit in favour of send_data_v2. The ids 25, 45, 59, 72 are real EXTENDED schema variants, but no BLE record layout depends on the schema id. In the half-duplex loop the device waits for the ATT confirmation of every indication (500 ms timeout), hands TX to the app with [0x04, 0x02, 0x02, …] on …-0001, and expects [0x04, 0x03, 0x02] to take it back. See Half-duplex & comms handoff.

Messages to read (BLE `gen_*` layouts)

The BLE GATT layouts are not the ESP-NOW layouts. The same (cat_id, cmd_id) key is reused by two transports with different struct layouts — for example BLE gen_live_data (0x03,0x01) is a 69-byte record, while TOTEM_MSG_MAP (3,1) is 20 bytes. A BLE client must use the gen_* layouts here; a mesh client must use the mesh map.

All BLE records prefix buff[0:2] = (cat_id, cmd_id). Confirmed layouts: Live Data (0x03, 0x01) — gen_live_data, struct <bfi3fb4Bi3b2hbiffb3ibHBBb at offset 2 (69 B). Fields in order:

#	code	Field
1	`b`	sat_count
2	`f`	p_acc, m (−1 = None)
3	`i`	altitude, m (−500 = None)
4	`f`	latitude, °
5	`f`	longitude, °
6	`f`	batt_volts, V
7	`b`	esp-now channel
8	`B`	power_bits (power_mode in bits 0-2)
9	`B`	max_hop_cnt
10	`B`	mesh_rx
11	`B`	mesh_relayed
12	`i`	unix_ts, s
13	`b`	color_id
14	`b`	orientation
15	`b`	solution_id
16	`h`	heading, °
17	`h`	azimuth, °
18	`b`	speed (cap 127, −1 = None)
19	`i`	odometer
20-21	`ff`	reserved ×2 (always 0)
22	`b`	reserved (always −1)
23	`i`	uptime, s
24	`i`	age
25	`i`	reserved (always 0)
26	`b`	power_level: battery health, 2 = low
27	`H`	reserved (always 0)
28	`B`	flags
29	`B`	reserved (always 0)
30	`b`	batt_pct

The flags byte (field 28) is

pack_flags(is_sos, is_eco, led_brt ≥ GLOBAL_BRT, gnss_location_set, power_level == 2, is_charging, 0, is_mag_cal_needed)

: bit 2 is normal (undimmed) brightness and bit 4 is low battery. The reserved fields hold these constants in every published firmware (3.2.12 to 5.0.3), and the official app reads and discards them, so a client can ignore them. Static Data (0x01, 0x02) — gen_static_data. buff[2] = total_len & 255, buff[3:9] = MAC (6 B), then struct <biHBBBbBBBbhhiiibbb at offset 9 (34 B), then three UTF-8 strings concatenated from offset 43. There are no per-string prefixes: their lengths are the struct’s last three fields.

Region	Field
`buff[2]`	total_len & 255
`buff[3:9]`	device MAC (6 B)
struct @9: `b`	reserved (always 0)
`i`	age
`H`	release_id
`B B B`	ver_major, ver_minor, ver_patch
`b`	color_id
`B`	settings_flags (bit0 persistent north, bit1 compass lock; the app reads bit3 as bond chat)
`B`	capabilities (bit0 = half-duplex loop; 1 in 5.x, 0 in 4.1.3)
`B`	service_id
`bhhiii`	6× reserved (always 0; the app discards them)
`b b b`	len(device_name), len(branch), len(wifi_ssid)
strings @43	device_name, git branch (`N/A`), wifi_ssid (`""`)

Peer Ping (0x06, 0x02) — gen_peer_ping. buff[2] = total length, buff[3:9] = peer MAC (6 B), buff[9] = mesh hops, then struct <ffbbh4BHbb4BiihBbf at offset 10 (40 B): lat, lon, p_acc, speed, bearing, flag byte A, r, g, b, 0 (the app’s dtim), name length, rssi, msg_rx, msg_tx, mesh_rx, mesh_send_count, last_update, last_coords_unix, distance_diff, flag byte B, orientation, volts. Then the peer name (from offset 50) and <bH = (batt_pct, release_id). A = pack_flags(sos, is_poi, is_mesh, is_stale, is_collected, 0, is_unknown, 0), B = pack_flags(is_hidden, is_locked, 0×6). The app reads A’s bit 5 as isIdle; the firmware always sends 0 there. is_unknown is set once a peer has had no coordinates for 2 h (v5.0.3; 4 h in v5.0.2). Peer Sync (0x06, 0x07) — gen_peer_sync: a peer-MAC list, [0x06, 0x07, total_len & 255, peer_count, mac0(6), mac1(6), …].

ESP-NOW mesh client

To join the mesh peer-to-peer instead of going through a phone, send/receive raw ESP-NOW frames on the fleet’s channel.

Radio setup

Item	Value
Channel	6 (`self.channel = 6` default; also settable at runtime via `set_channel`)
PHY	`WIFI_PROTOCOL_LR` (Espressif Long Range, protocol bit 8) in ESP-NOW-only / BLE modes
LR rate	250 K: `EspConn` calls `e.config(rate=41)` (`WIFI_PHY_RATE_LORA_250K`)
Encryption	none — `add_peer(mac)` with no `lmk`

The fleet is pinned to a single channel; ESP-IDF drops off-channel frames, so a mesh client must match it. Channel 6 is the built-in default (Confirmed), but the live value can be changed at runtime (Partial) — see what to confirm on-device.

Frame layout

┌──────────┬──────────┬────────┬────────┬───────────────────────────┐
│ SyncWord │ SyncWord │ cat_id │ cmd_id │ payload (struct-packed)   │
│  0xA7    │  0x74    │  u8    │  u8    │ variable                  │
└──────────┴──────────┴────────┴────────┴───────────────────────────┘
  [0]        [1]        [2]      [3]      [4:]

SyncWord = the 2 bytes 0xA7 0x74 (literal in the image).
Validation is SyncWord match + len >= 4 + a per-(cat, cmd) payload-size check.
There is no CRC or checksum on the ESP-NOW frame. (The rodata string Invalid Checksum value for: {} belongs to the u-blox UBX GNSS parser, not this path.)
Mesh dedup UID is a uint16 randint(1, 65534) at mesh-frame offset 20; MSG_EXP_MSECS = 150000 ms.

Payload formats (`TOTEM_MSG_MAP`, corrected)

Earlier analyses decoded the map’s qstr-immediates with >>2, producing bogus “handler names” (disconn_animation, device_power, dev_info, disabled, …). Those are decode artifacts and are wrong. ESP32 MicroPython (REPR_A) tags qstr-immediates as (o & 7) == 2 with value o >> 3; under the correct >>3 decode every value is a struct format string. The corrected table follows (Confirmed).

Every payload leads with the echoed (cat_id, cmd_id) as BB.

`(cat, cmd)`	Format	Size
`(0,0)` / `(0,1)` / `(0,2)`	`<BBffbbhbbb6B`	23 B
`(1,0)`	`<BBB7bHbb`	14 B
`(1,2)`	`<BBiffHii`	24 B
`(1,5)`	`<BB9B8bhh4B`	27 B
`(1,6)`	`<BBbbB`	5 B
`(1,7)`	`<3Bbb`	5 B
`(2,0)`	`<BB6BffbbHbbbhhBBi`	33 B
`(3,1)` / `(3,2)`	`<BBHH4b4BHHBb`	20 B
`(7,0)`	`<BBffbbbHbbH`	19 B
`(7,1)`	`<BBHbffb`	14 B

EXTENDED schema-versioned variants — (cat, cmd) → {schema_id → fmt}:

`(cat, cmd)`	Schema id	Format	Size
`(0,0)`	25	`<BBffbbhbbb6B`	23 B
`(0,0)`	59	`<BBffbbhbbb6Bhii4BhHehhffbB`	57 B
`(0,0)`	72	`<BBffbbhbbb6Bhii4BhHehhffbBBiiBBb`	69 B
`(1,6)`	5	`<BBbbB`	5 B
`(1,6)`	29	`<BBbb9BBB3i`	27 B
`(2,0)`	45	`<BB6BffbbHbbbhhBBiffh`	43 B

The (0,*) 23-byte family is the mesh peer beacon: two floats (lat/lon), flags, and a 6-byte MAC. Field-by-field mesh semantics beyond the struct layout are Partial — the byte layouts are recovered, but individual fields are not fully labelled.

Custom OTA server

You can point a Totem at your own OTA server; it needs no signing key (SHA-256 only, plain HTTP). The contract (Confirmed):

Device announces

POST http://api.totemportal.com/devices/{MAC}/ota (plain HTTP, no auth) with JSON body {version, endpoint_id, device_type_id, lat, lon, gnss_time, release_id}.

Server replies with a release object

Expose ota_url, version, product, branch, release_code, release_id.

Device fetches the manifest

GET {ota_url}/contents.json — a JSON array of filenames. The device picks the entry ending .bin (firmware) or .tgz (preview).

Download, verify, flash

The device downloads the chosen file, verifies SHA-256 only, flashes, reboots, and reports back with POST …/ota?updated.

A device WebSocket to api.totemportal.com (ws://) carries push OTA triggers shaped like {"cmd":…} (Partial). A demi-god ESP-NOW OTA trigger also exists (demigod_gen_ota_update), but its exact (cat, cmd) and struct are not recovered — do not fabricate them.

Chunked transfer

Only needed if your client accepts the device’s log uploads (v5.0.3) or does BLE OTA. A client that sends the plain 4-byte Ready frame never receives uploads. Layouts (Confirmed; details in chunking):

Transfer header (announce / finish), indicated on …-0001: [0x02, 0x02] + struct '<HBBBiHiB' = file_id, status_id, action_id, file_type_id, byte_pos, chunk_no, file_size, flags (v5.0.3: bit0 last chunk, bit1 from compass). Then the 32-byte SHA-256 at buff[18:50], the name length at buff[50], the name, and err_no. file_id = sha256[0] | sha256[1] << 8.
Chunk (v5.0.3), notified on …-0003: [0x00, 0x02] + struct '<HHiH' = file_id, length, byte_pos, chunk_no, then the data (CHUNK_HDR_SZ = 12).
App reply, written to …-0001 (v5.0.3): (0x02, 0x03) + struct '<HbBBiHiB', 18 bytes in total. It is read as file_id, status, action, chunk: status ∈ {2, 3, 4} is terminal, action 4 = resume at chunk, action 1 = ready.

Field	Enum values
`status_id`	1 = normal / in-progress, 4 = error / abort
`action_id`	0 = header / announce, 1 = last-chunk / complete
`file_type_id`	2 = upload-to-app (destinations `SAVE_TO_VFS = 1`, `SAVE_TO_OTA = 2`, `UPLOAD_TO_APP = 3`)

Overall result codes: 1 done, 2 no-app, 3 retry, 4 failed, 5 cancel, 6 abort.

Still missing / confirm on-device

These are known-structure-but-unconfirmed items. Verify them against your own hardware before relying on them:

Item	Status	How to confirm
Home channel	default 6 (Confirmed), runtime-settable (Partial)	sniff the fleet — the live channel can be changed via `set_channel`
`frame_schema_id` / transmit mode	Confirmed and hardware-tested: `0` = legacy loop (works on macOS), `> 0` = half duplex (stalls on macOS/iOS)	`totemctl --trace info`, or add `--half-duplex` to compare
Half duplex on Linux / Android / Windows	Untested (those stacks confirm indications, so it should work)	`totemctl --half-duplex --trace watch` on such a host
Mesh payload field semantics	struct layouts Confirmed, per-field meaning Partial	correlate captured frames against known device state
Demi-god OTA frame	Partial / missing	`(cat, cmd)` + struct not recovered — do not fabricate
Runtime-only values	Partial	venue code, group id, and device name live in runtime state / bytecode, not the static image

Safety

Interoperate only with Totems you own. Because the mesh is unauthenticated and unencrypted, the demi-god ESP-NOW broadcast path can affect nearby devices — its only gate is RSSI proximity plus message-UID dedupe, with no signature or HMAC. Do not send demi-god or mesh commands against devices you do not own; broadcasting near other people’s Totems can change their behaviour without their consent.

Overview

2.4 GHz Communication Protocols

Subsystems

Reference

No cryptographic secrets are required

BLE client

Connect

Handshake (ConnStatus-Ready)

Messages to read (BLE `gen_*` layouts)

ESP-NOW mesh client

Radio setup

Frame layout

Payload formats (`TOTEM_MSG_MAP`, corrected)

Custom OTA server

Chunked transfer

Still missing / confirm on-device

Safety

​No cryptographic secrets are required

​BLE client

​Connect

​Handshake (ConnStatus-Ready)

​Messages to read (BLE gen_* layouts)

​ESP-NOW mesh client

​Radio setup

​Frame layout

​Payload formats (TOTEM_MSG_MAP, corrected)

​Custom OTA server

​Chunked transfer

​Still missing / confirm on-device

​Safety

No cryptographic secrets are required

BLE client

Connect

Handshake (ConnStatus-Ready)

Messages to read (BLE `gen_*` layouts)

ESP-NOW mesh client

Radio setup

Frame layout

Payload formats (`TOTEM_MSG_MAP`, corrected)

Custom OTA server

Chunked transfer

Still missing / confirm on-device

Safety