Skip to main content

Addressing: (cat_id, cmd_id)

Every application message carries a one-byte category and a one-byte command. This tuple is used on both the BLE link and the ESP-NOW mesh — but the same (cat_id, cmd_id) maps to a different wire layout on each transport (see Two format registries).
Evidence: [BLE] DataXfer cat_id: {} | cmd_id: {}, [BLE] ConnStatus cat_id: {} | cmd_id: {} | len: {}, A cmd_id must be provided for demi-god messages.

Two format registries — do not conflate

(cat_id, cmd_id) is reused by two transports with different struct layouts. A BLE client MUST use the BLE gen_* formats below — the ESP-NOW TOTEM_MSG_MAP formats will decode a BLE frame into garbage.
Proof they differ for the same key: BLE gen_live_data (0x03, 0x01) packs <bfi3fb4Bi3b2hbiffb3ibHBBb (69 B), while TOTEM_MSG_MAP (3,1) = <BBHH4b4BHHBb (20 B). Same key, different bytes. (Confirmed)

Categories

Confirmed BLE (cat_id, cmd_id) pairs and their labels, taken directly from the send_data_v2 transmit logs: The receive path dispatches app→device commands on cat_id. Writes to …-0002 go to recv_data_msgs; writes to …-0001 go to recv_status_msgs. The complete map, decoded from both handlers (Confirmed; identical in v5.0.2 and v5.0.3 except where marked): Data characteristic (…-0002) Conn-status characteristic (…-0001) The legacy transmit loop logs its sends per category (Sending static data, Sending BLE | DataTransfer (0x06, 0x07), …), and the half-duplex loop logs a confirmation per record: BLE ACK Static Data, BLE ACK Live Data, BLE ACK Peer Ping, BLE ACK Peer Sync, BLE ACK Comms Handoff.

BLE message formats (gen_*) — what a client reads

Each gen_* builder writes buff[0:2] = (cat_id, cmd_id) then packs the payload from live device state; the LOAD_ATTR names are the field semantics. These are the formats a BLE client actually reads (Confirmed from the struct.pack_into opcode stream).

Live Data — (0x03, 0x01) gen_live_data

struct <bfi3fb4Bi3b2hbiffb3ibHBBb packed at offset 2 (calcsize = 69). 30 values, in this exact order: Flags byte #28 = pack_flags(is_sos, is_eco_mode, led_brt≥GLOBAL_BRT, gnss_location_set, power_level==2, is_charging, 0, is_mag_cal_needed) in that bit order. Bit 2 is set at normal brightness (GLOBAL_BRT = 0.6; eco dims to 0.1). Bit 4 means low battery (change_power_level logs Changing battery health to: {}). Byte #8 power_bits encodes config.power_mode in bits[0:3]. The reserved slots are constants in every published firmware. 3.2.12, 4.1.3, 5.0.2 and 5.0.3 all pass the literals above to struct.pack_into. The official app (2.3.0, useLiveDataParser) reads each one at the offset shown and discards it. It also runs offset 69 through unpackFlags and uses no bit. Neither side gives them a name. gen_live_data still computes len(config.peers), config.closest_peer, config.furthest_peer, the ms since config.last_peer_msg and gc.mem_free() and never packs them, which suggests these slots once carried such statistics. (The app labels flag bits 2–4 isDimLeds, isGnssLock and isLowBatt; the firmware sets bit 2 at full brightness.)

Static Data — (0x01, 0x02) gen_static_data

buff[2] = total_len & 255; buff[3:9] = MAC (6 bytes); then struct <biHBBBbBBBbhhiiibbb at offset 9 (calcsize = 34); then three UTF-8 strings concatenated from offset 43 (no per-string prefix; the lengths are fields 17–19). 19 packed values: Then, from offset 43, the three strings back to back: device_name, git branch ('N/A' if none), wifi_ssid (''). Field 9 is a capability byte. 4.1.3, which has no send_data_v2, packs 0; 5.x packs 1. The official app (2.3.0, useStaticDataParser) reads bit 0 as isHalfDuplex and switches to the TX handoff when it is set. The app reads fields 1 and 11–16 and discards them; every published firmware packs them as 0.

Peer Ping — (0x06, 0x02) gen_peer_ping

buff[0:2] = (0x06, 0x02); buff[3:9] = peer MAC (hex_to_bin, 6 bytes); buff[9] = peer.mesh_hops clamped to 0..255. A 40-byte record struct <ffbbh4BHbb4BiihBbf (calcsize = 40) is then packed at offset 10, followed by the UTF-8 peer name (its byte length is field #11 below, name bytes from offset 50) and a trailing struct <bH = (batt_pct, release_id). buff[2] holds the total length, written last. This is a full ~53 + name-length-byte peer record, not just MAC + two flag bytes — the two peer-flag bytes are fields 6 and 20 inside this struct. (Confirmed from the struct.pack_into opcode stream; field labels are the LOAD_ATTR names, so semantics are Confirmed where a name is given.) Log: Sending BLE | DataTransfer (0x06, 0x02) for {}.

Peer Sync — (0x06, 0x07) gen_peer_sync

A peer-MAC list: [0x06, 0x07, total_len & 255, peer_count, mac0(6), mac1(6), …] (6-byte MACs, first at offset 4, +6 each).

ESP-NOW registry: TOTEM_MSG_MAP

Earlier docs decoded the qstr-immediate values with >>2, yielding bogus “handler names” (disconn_animation, device_power, dev_info, dev_total_lightsleep_ms, disabled, …). Those were decode artifacts and are wrong. ESP32 MicroPython (REPR_A) tags qstr-immediates as (o & 7) == 2 with value o >> 3; under the correct >>3 decode every TOTEM_MSG_MAP value is a struct format string.
TOTEM_MSG_MAP in espnow_conn_v2.py is a (cat_id, cmd_id)-keyed map (2-byte bytes key) whose values are the ESP-NOW / mesh payload struct formats. Recovered in full with the corrected >>3 decode (Confirmed): EXTENDED (schema-versioned variants, (cat, cmd) → {schema_id → fmt}): Every <BB… format leads with the echoed (cat_id, cmd_id). The (0,*) 23-B family (<BBffbbhbbb6B) is the mesh peer beacon: two floats (lat/lon) + flags + 6-byte MAC — the ESP-NOW analogue of the BLE Peer records. A BLE-only client ignores this table. Field-by-field mesh semantics are Partial (not fully labelled).

ESP-NOW frame envelope

On the radio these payloads sit inside an ESP-NOW application frame with a fixed SyncWord prefix and no CRC (Confirmed):
Frame validation = SyncWord 0xA7 0x74 match + len ≥ 4 + per-(cat, cmd) payload-size check. There is no checksum or CRC on the ESP-NOW frame. (The rodata string Invalid Checksum value for: {} belongs to the u-blox UBX GNSS parser, not this path.) A mesh dedup UID (uint16) sits at mesh-frame offset 20.

Message identity & lifetime

Mesh/peer messages are de-duplicated and expired: This lets a message flood the mesh once and be dropped as a duplicate on re-receipt (Demi-god command ignored, already received is the same idea in the command path).

Chunking

Files larger than one GATT write are moved by a common chunking layer (f_ble/chunking.py, used by BLE OTA and, since v5.0.3, by the log uploader). Two headers are involved:
In v5.0.2 f_ble/file_upload.py imported CHUNK_HDR_FMT and CHUNK_HDR_SZ from chunking, but chunking never defined them, and nothing imported the uploader. Earlier versions of this page inferred CHUNK_HDR_FMT = '<HBBBiHiB' from that. v5.0.3 defines both names explicitly, and they describe the 12-byte chunk header. <HBBBiHiB is the transfer header packed inline by gen_transfer_buff.

Transfer header (gen_transfer_buff)

buff[0] = 2, buff[1] = 2, then struct.pack_into('<HBBBiHiB', buff, 2, …) (Confirmed, both versions): After the 16-byte header: buff[18:50] = 32-byte SHA-256 (when set); buff[50] = name_len (v5.0.3 writes 0 when there is no name), then the UTF-8 file name, then a trailing err_no byte.

Chunk header (v5.0.3, FileUploader._stream)

[0x00, 0x02] + struct.pack_into('<HHiH', buff, 2, file_id, n, byte_pos, chunk_no) + n data bytes, written to …-0003 with send_update=True. n is the bytes read for this chunk and byte_pos its offset (Confirmed from the operands); chunk_no counts from 1 (Inferred from the counter variable). The payload per chunk is min(ble.mtu_payload(), 247) - CHUNK_HDR_SZ (20 − 12 before the MTU is known). v5.0.3 tracks the negotiated MTU in BleLite.mtu for this purpose.

Header enums (Confirmed)

file_id is derived from the file’s SHA-256: file_id = sha256[0] | sha256[1] << 8 (u16). The transfer destination enum is separate: SAVE_TO_VFS = 1, SAVE_TO_OTA = 2, and v5.0.3 adds UPLOAD_TO_APP = 3.

File transfer

f_ble/file_upload.py (FileUploader) pushes device log files (events-* files) to the app so the app can forward them to the cloud. It is new in practice in v5.0.3: v5.0.2 shipped the module but never imported it. In v5.0.3 ble_manager starts it as the ble_file_upload task, and it only runs when every gate is open: Uploads mark themselves as the half-duplex non-critical owner (noncrit_owner = 'upload'), which may delay the next TX handoff by up to 2 s. After MAX_FILE_RETRIES failures a file is suspended until BLE is next turned on. v5.0.3 also toggles nav-log fast rotation depending on whether a backlog exists.

App→device reply header (FileUploader.on_header)

The app replies with an 18-byte header, struct <BBHbBBiHiB (Confirmed, guarded by len(data) >= 18), starting (0x02, 0x03). In v5.0.3 it is written to …-0001: recv_status_msgs routes (2,3) to uploader.on_header. Used fields: file_id = t[2], status = t[3], action = t[4], chunk = t[7]. Log: [Upload] App header | status: {} | action: {} | chunk: {} | err: {}.

Control flow

Result codes (Confirmed)

RES_DONE = 1, RES_NO_APP = 2, RES_RETRY = 3, RES_FAILED = 4, RES_CANCEL = 5, RES_ABORT = 6 (logged as result: {} (1=done 2=no app 3=retry 4=failed 5=cancel 6=abort)).