zat.dev
atproto utils for zig
zat.dev
atproto
sdk
zig
feat: configurable CAR size limits + devlog 005
CAR parser now accepts max_size and max_blocks options for large repo
verification. export jwt module for verify tool. devlog 005 covers the
three-way trust chain comparison (zig vs go vs rust).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zzstoatzz.io
+75
-3
+67
devlog/005-three-way-verify.md
+67
devlog/005-three-way-verify.md
···
1
+
# three-way trust chain verification
2
+
3
+
the previous devlogs covered decode throughput and signature verification as isolated benchmarks. this one puts it all together: given a handle, resolve identity, fetch the full repo, and cryptographically verify everything — zig vs Go vs Rust.
4
+
5
+
## what we're measuring
6
+
7
+
the full AT Protocol trust chain:
8
+
9
+
```
10
+
handle → DID → DID document → signing key
11
+
↓
12
+
repo CAR → commit → signature ← verified against key
13
+
↓
14
+
MST root CID → walk nodes → rebuild tree → CID match
15
+
```
16
+
17
+
all three implementations do the same work: resolve the handle, resolve the DID, extract the signing key, fetch the repo CAR, parse every block with SHA-256 CID verification, verify the commit signature, walk the MST to count records, and (where possible) rebuild the MST to verify the root CID.
18
+
19
+
## the implementations
20
+
21
+
**zig (zat)** — uses zat's own primitives end to end: `HandleResolver`, `DidResolver`, `car.read()` with CID verification, `jwt.verifySecp256k1`, `mst.Mst` for walk + rebuild.
22
+
23
+
**go (indigo)** — uses bluesky's official Go SDK: `identity.BaseDirectory` for handle/DID resolution, `repo.LoadRepoFromCAR` for parsing, `commit.VerifySignature` for sig verify, `MST.Walk()` + `MST.RootCID()` for MST.
24
+
25
+
**rust (RustCrypto)** — manual implementation since no indigo-equivalent exists in Rust. HTTP + DNS TXT handle resolution, plc.directory DID resolution, hand-rolled CAR parser with SHA-256, k256/p256 for ECDSA, recursive CBOR MST traversal. skips MST rebuild (no crate for it).
26
+
27
+
## the O(n) bug
28
+
29
+
first run against pfrazee.com (192k records, 243k blocks): zig's MST walk took **79 seconds**. go finished in 6ms.
30
+
31
+
the cause: `findBlock()` was doing a linear scan through 243k blocks on every lookup. MST walk calls `findBlock()` once per node (~50k nodes). that's ~12 billion comparisons.
32
+
33
+
Go's `TinyBlockstore` uses a `map[string]blocks.Block` — O(1) by CID key. replaced the flat block slice with `std.StringHashMapUnmanaged([]const u8)` in zig and `HashMap<Vec<u8>, Vec<u8>>` in rust.
34
+
35
+
result: 79s → 48ms (zig), 14s → 125ms (rust).
36
+
37
+
## results
38
+
39
+
_pfrazee.com — 192,144 records, 243,470 blocks, 70.6 MB CAR, macOS arm64 (M3 Max)_
40
+
41
+
| SDK | CAR parse | sig verify | MST walk | MST rebuild | compute total |
42
+
|-----|----------:|----------:|---------:|------------:|-------------:|
43
+
| zig (zat) | 81.6ms | 0.6ms | 45.5ms | 172.6ms | **300.4ms** |
44
+
| go (indigo) | 403.8ms | 0.4ms | 5.8ms | 0.0ms | **410.0ms** |
45
+
| rust (RustCrypto) | 301.0ms | 0.2ms | 120.9ms | N/A | **422.1ms** |
46
+
47
+
network time (handle + DID resolution + repo fetch) dominates total wall clock — 8-20 seconds depending on PDS response time. compute is under 500ms for all three.
48
+
49
+
the story is different from the decode benchmarks. there, zig was 19x faster than Go. here, the gap is ~1.4x. the reason: signature verification is a single ECDSA verify (sub-millisecond for everyone), and CAR parsing on a 70 MB file is less dominated by per-block overhead than the firehose's thousands of small CARs. the MST rebuild (zig-only) is the biggest single cost — serializing 192k entries into a fresh tree and hashing.
50
+
51
+
go's MST walk is fastest (5.8ms vs zig's 45.5ms) because indigo's `LoadRepoFromCAR` builds the MST in memory during CAR parse. walking it is just pointer chasing. zig and rust decode MST nodes from raw CBOR on each visit.
52
+
53
+
## the chart tool
54
+
55
+
added a script (`scripts/verify_chart.py`) that parses `just verify` output and generates SVG charts — stacked horizontal bars with a dark theme. two variants: compute-only (the interesting comparison) and total (dominated by network). see the [atproto-bench README](https://tangled.sh/@zzstoatzz.io/atproto-bench) for the charts.
56
+
57
+
```sh
58
+
just chart pfrazee.com # run verify + generate SVGs
59
+
```
60
+
61
+
## what's in this release
62
+
63
+
- **go-verify**: full trust chain verification using indigo
64
+
- **rust-verify**: full trust chain verification using RustCrypto + hand-rolled CAR/MST
65
+
- **zig verify**: O(1) block lookup fix (HashMap instead of linear scan)
66
+
- **scripts/verify_chart.py**: SVG chart generation from verify output
67
+
- **justfile**: `just verify`, `just chart` recipes
+6
-2
src/internal/repo/car.zig
+6
-2
src/internal/repo/car.zig
···
45
45
/// this is the correct behavior for untrusted data (e.g. from the network).
46
46
/// set to false only for trusted local data where you want raw decode speed.
47
47
verify_block_hashes: bool = true,
48
+
/// max total CAR size in bytes. null = use default (2 MB).
49
+
max_size: ?usize = null,
50
+
/// max number of blocks. null = use default (10,000).
51
+
max_blocks: ?usize = null,
48
52
};
49
53
50
54
/// parse a CAR v1 file from raw bytes
···
54
58
55
59
/// parse a CAR v1 file from raw bytes with options
56
60
pub fn readWithOptions(allocator: Allocator, data: []const u8, options: ReadOptions) CarError!Car {
57
-
if (data.len > max_blocks_size) return error.BlocksTooLarge;
61
+
if (data.len > (options.max_size orelse max_blocks_size)) return error.BlocksTooLarge;
58
62
59
63
var pos: usize = 0;
60
64
···
105
109
try verifyBlockHash(cid_bytes, content);
106
110
}
107
111
108
-
if (blocks.items.len >= max_block_count) return error.TooManyBlocks;
112
+
if (blocks.items.len >= (options.max_blocks orelse max_block_count)) return error.TooManyBlocks;
109
113
110
114
try blocks.append(allocator, .{
111
115
.cid_raw = cid_bytes,
+2
-1
src/root.zig
+2
-1
src/root.zig
···
23
23
pub const json = @import("internal/xrpc/json.zig");
24
24
25
25
// crypto
26
-
pub const Jwt = @import("internal/crypto/jwt.zig").Jwt;
26
+
pub const jwt = @import("internal/crypto/jwt.zig");
27
+
pub const Jwt = jwt.Jwt;
27
28
pub const multibase = @import("internal/crypto/multibase.zig");
28
29
pub const multicodec = @import("internal/crypto/multicodec.zig");
29
30