zzstoatzz.io
bsky feeds about music
music-atmosphere-feed.plyr.fm/
bsky
feed
zig
add zig atproto sdk wishlist doc
+272
+272
docs/zig-atproto-sdk-wishlist.md
+272
docs/zig-atproto-sdk-wishlist.md
···
1
+
# zig atproto sdk wishlist
2
+
3
+
notes from building a bluesky feed generator in zig. what would make life easier.
4
+
5
+
## the pain points
6
+
7
+
### 1. json navigation is brutal
8
+
9
+
every single field access looks like this:
10
+
11
+
```zig
12
+
if (record.get("embed")) |embed_val| {
13
+
if (embed_val == .object) {
14
+
if (embed_val.object.get("external")) |external_val| {
15
+
if (external_val == .object) {
16
+
if (external_val.object.get("uri")) |uri_val| {
17
+
if (uri_val == .string) {
18
+
// finally, the actual value
19
+
}
20
+
}
21
+
}
22
+
}
23
+
}
24
+
}
25
+
```
26
+
27
+
this is 6 levels of nesting to get `embed.external.uri`. it's error-prone, hard to read, and easy to mess up.
28
+
29
+
**wish**: typed structs generated from lexicons. just give me:
30
+
31
+
```zig
32
+
const post = try Post.fromJson(data);
33
+
if (post.embed) |embed| {
34
+
if (embed.external) |ext| {
35
+
doSomething(ext.uri);
36
+
}
37
+
}
38
+
```
39
+
40
+
### 2. no lexicon types at all
41
+
42
+
we're working blind. the at-protocol has a full lexicon system with typed schemas, but in zig we get raw `json.Value` and have to know the structure from memory or docs.
43
+
44
+
**wish**: codegen from lexicons. run a build step that reads `app.bsky.feed.post.json` and outputs `Post`, `Facet`, `Embed`, `ExternalEmbed`, etc. with proper zig types.
45
+
46
+
even better: ship pre-generated types for the core `app.bsky.*` and `com.atproto.*` namespaces.
47
+
48
+
### 3. no xrpc client
49
+
50
+
making api calls means manually:
51
+
- constructing urls
52
+
- handling auth headers
53
+
- parsing responses
54
+
- dealing with rate limits
55
+
- cursor pagination
56
+
57
+
**wish**: typed xrpc client:
58
+
59
+
```zig
60
+
const client = try AtProto.init(allocator);
61
+
try client.login("handle", "password");
62
+
63
+
// typed request, typed response
64
+
const feed = try client.call(.app_bsky_feed_getFeed, .{
65
+
.feed = "at://did:plc:.../app.bsky.feed.generator/my-feed",
66
+
.limit = 50,
67
+
});
68
+
69
+
for (feed.posts) |post| {
70
+
// post is already typed
71
+
}
72
+
```
73
+
74
+
### 4. no firehose/jetstream client
75
+
76
+
we had to build our own websocket handler from scratch:
77
+
- tls connection setup
78
+
- websocket frame parsing
79
+
- reconnection logic
80
+
- backpressure handling
81
+
- cursor management for resumption
82
+
83
+
**wish**: built-in stream client:
84
+
85
+
```zig
86
+
const stream = try Jetstream.connect(allocator, .{
87
+
.collections = &.{"app.bsky.feed.post"},
88
+
.cursor = saved_cursor,
89
+
});
90
+
91
+
while (try stream.next()) |event| {
92
+
switch (event) {
93
+
.commit => |commit| {
94
+
// commit.record is already typed based on collection
95
+
},
96
+
.identity => |id| { ... },
97
+
.account => |acc| { ... },
98
+
}
99
+
}
100
+
```
101
+
102
+
### 5. tid parsing is non-obvious
103
+
104
+
tids encode timestamps in base32-sortable format. we had to figure out the algorithm and implement it:
105
+
106
+
```zig
107
+
pub fn parseTidTimestamp(tid: []const u8) ?i64 {
108
+
if (tid.len < 13) return null;
109
+
var timestamp: u64 = 0;
110
+
for (tid[0..13]) |c| {
111
+
const val: u64 = switch (c) {
112
+
'2'...'7' => c - '2',
113
+
'a'...'z' => c - 'a' + 6,
114
+
else => return null,
115
+
};
116
+
timestamp = (timestamp << 5) | val;
117
+
}
118
+
return @intCast(timestamp / 1000);
119
+
}
120
+
```
121
+
122
+
**wish**: `Tid` type with utilities:
123
+
124
+
```zig
125
+
const tid = Tid.parse("3jui7...") orelse return error.InvalidTid;
126
+
const created_at = tid.timestamp(); // returns i64 milliseconds
127
+
const clock_id = tid.clockId();
128
+
129
+
// also: generate tids
130
+
const new_tid = Tid.now();
131
+
```
132
+
133
+
### 6. at-uri parsing
134
+
135
+
at-uris are everywhere: `at://did:plc:xyz/app.bsky.feed.post/abc123`
136
+
137
+
we need to extract did, collection, rkey constantly. currently doing string splits manually.
138
+
139
+
**wish**: `AtUri` type:
140
+
141
+
```zig
142
+
const uri = try AtUri.parse("at://did:plc:xyz/app.bsky.feed.post/abc123");
143
+
uri.did // "did:plc:xyz"
144
+
uri.collection // "app.bsky.feed.post"
145
+
uri.rkey // "abc123"
146
+
147
+
// construct
148
+
const new_uri = AtUri.init("did:plc:xyz", "app.bsky.feed.post", "abc123");
149
+
new_uri.toString() // "at://did:plc:xyz/app.bsky.feed.post/abc123"
150
+
```
151
+
152
+
### 7. feed generator scaffolding
153
+
154
+
building a feed generator requires:
155
+
- http server for xrpc endpoints
156
+
- `describeFeedGenerator` endpoint
157
+
- `getFeedSkeleton` endpoint with cursor handling
158
+
- well-known did document serving
159
+
160
+
we built all this from scratch.
161
+
162
+
**wish**: feed generator framework:
163
+
164
+
```zig
165
+
const FeedGenerator = @import("atproto").FeedGenerator;
166
+
167
+
const MyFeed = struct {
168
+
pub fn filter(post: Post) bool {
169
+
// return true to include in feed
170
+
return post.hasLink("soundcloud.com");
171
+
}
172
+
173
+
pub fn sort(posts: []Post) void {
174
+
// custom sort, or use default chronological
175
+
}
176
+
};
177
+
178
+
pub fn main() !void {
179
+
var generator = try FeedGenerator.init(allocator, .{
180
+
.did = "did:web:my-feed.example.com",
181
+
.feeds = &.{
182
+
.{ .name = "my-feed", .handler = MyFeed },
183
+
},
184
+
});
185
+
try generator.serve(3000);
186
+
}
187
+
```
188
+
189
+
### 8. did resolution
190
+
191
+
resolving `did:plc:*` and `did:web:*` to did documents requires http calls and json parsing. needed for verifying identities, getting service endpoints.
192
+
193
+
**wish**: did resolver:
194
+
195
+
```zig
196
+
const resolver = DidResolver.init(allocator);
197
+
const doc = try resolver.resolve("did:plc:xyz");
198
+
doc.alsoKnownAs // ["at://handle.bsky.social"]
199
+
doc.service // pds endpoint, etc.
200
+
```
201
+
202
+
### 9. cbor/dag-cbor support
203
+
204
+
the actual at-proto repo format uses dag-cbor. if you want to verify signatures or work with raw repo data, you need cbor.
205
+
206
+
**wish**: cbor codec, at least for reading commit data from firehose.
207
+
208
+
### 10. labeler integration
209
+
210
+
reading and applying labels (nsfw, etc.) from labeler services.
211
+
212
+
**wish**: label types and utilities:
213
+
214
+
```zig
215
+
const labels = post.labels orelse &.{};
216
+
if (Label.hasAny(labels, &.{"porn", "sexual", "nudity"})) {
217
+
// exclude
218
+
}
219
+
```
220
+
221
+
### 11. facet helpers
222
+
223
+
facets (links, mentions, tags in post text) have a specific structure. extracting links means navigating:
224
+
225
+
```zig
226
+
for (facets) |facet| {
227
+
for (facet.features) |feature| {
228
+
if (feature.type == .link) {
229
+
// feature.uri
230
+
}
231
+
}
232
+
}
233
+
```
234
+
235
+
**wish**: facet utilities:
236
+
237
+
```zig
238
+
const links = post.extractLinks(); // [][]const u8
239
+
const mentions = post.extractMentions(); // []Did
240
+
const tags = post.extractTags(); // [][]const u8
241
+
```
242
+
243
+
### 12. rich text building
244
+
245
+
creating posts with links/mentions requires building facet byte ranges correctly.
246
+
247
+
**wish**: rich text builder:
248
+
249
+
```zig
250
+
var rt = RichText.init(allocator);
251
+
rt.text("check out ");
252
+
rt.link("this track", "https://soundcloud.com/...");
253
+
rt.text(" by ");
254
+
rt.mention("@artist.bsky.social");
255
+
256
+
const post = rt.toPost(); // has .text and .facets set correctly
257
+
```
258
+
259
+
## summary
260
+
261
+
the ideal sdk would have:
262
+
263
+
1. **typed lexicons** - generated zig structs for all at-proto record types
264
+
2. **xrpc client** - typed api calls with auth, pagination, rate limiting
265
+
3. **stream client** - jetstream/firehose with reconnection, typed events
266
+
4. **primitives** - Tid, AtUri, Did, Cid types with parsing/generation
267
+
5. **feed generator framework** - scaffolding for common feed patterns
268
+
6. **facet utilities** - extract/build links, mentions, tags
269
+
7. **label handling** - read and apply moderation labels
270
+
8. **did resolution** - resolve did:plc and did:web
271
+
272
+
basically: let me focus on the feed logic, not the protocol plumbing.