An encrypted personal cloud built on the AT Protocol.

Update docs for three-crate architecture

- README: appview section, three-crate description, updated roadmap
- ARCHITECTURE.md: system diagram with appview + Jetstream, full
appview crate tree, Ed25519 in public key discovery, shared config
path resolution, storage layout includes appview.toml
- CONTRIBUTING.md: appview in architecture and test commands
- Blackbox test plan: appview section (status, health, auth, indexing)
- Extract MICROS_PER_SECOND constant from inline 1_000_000 literals

sans-self.org cf3b53a6 e568b118

Waiting for spindle ...
+213 -27
+110 -1
AGENT-BLACKBOX-TEST.md
··· 404 404 405 405 --- 406 406 407 - ## 7. Cleanup 407 + ## 7. AppView 408 + 409 + The AppView is a separate binary (`opake-appview`) that indexes grants and keyrings from the AT Protocol firehose. These tests require a running Jetstream instance or network access to the public Jetstream relays. 410 + 411 + ### 7.1 Configuration 412 + 413 + Create a minimal config: 414 + 415 + ```bash 416 + cat > /tmp/opake-appview-test/appview.toml <<EOF 417 + jetstream_url = "wss://jetstream2.us-east.bsky.network/subscribe" 418 + listen = "127.0.0.1:6100" 419 + db_path = "/tmp/opake-appview-test/appview.db" 420 + EOF 421 + ``` 422 + 423 + ### 7.2 Status (cold start) 424 + 425 + ```bash 426 + opake-appview --config-dir /tmp/opake-appview-test status 427 + # Cursor: (none — indexer has not run) 428 + # Grants: 0 429 + # Keyrings: 0 430 + ``` 431 + 432 + ### 7.3 Start indexer + API 433 + 434 + ```bash 435 + opake-appview --config-dir /tmp/opake-appview-test run -v & 436 + APPVIEW_PID=$! 437 + sleep 3 438 + ``` 439 + 440 + **Verify:** 441 + - Logs show "opake-appview listening on 127.0.0.1:6100" 442 + - Logs show Jetstream connection established 443 + 444 + ### 7.4 Health endpoint 445 + 446 + ```bash 447 + curl -s http://127.0.0.1:6100/api/health | jq . 448 + # { 449 + # "indexerConnected": true, 450 + # "cursorTime": "2026-...", 451 + # "cursorAgeSecs": <small number> 452 + # } 453 + ``` 454 + 455 + **Verify:** 456 + - `indexerConnected` is `true` 457 + - `cursorAgeSecs` is small (< 60) 458 + 459 + ### 7.5 Inbox and keyrings require auth 460 + 461 + ```bash 462 + curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:6100/api/inbox?did=did:plc:test 463 + # 401 464 + 465 + curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:6100/api/keyrings?did=did:plc:test 466 + # 401 467 + ``` 468 + 469 + ### 7.6 Share triggers indexing 470 + 471 + With the AppView still running, create a share grant using the CLI (from section 4.2): 472 + 473 + ```bash 474 + A$ opake share shared-file.txt <B-handle> 475 + ``` 476 + 477 + Wait a few seconds for the firehose to deliver the event. 478 + 479 + ```bash 480 + opake-appview --config-dir /tmp/opake-appview-test status 481 + # Grants: should be ≥ 1 482 + ``` 483 + 484 + ### 7.7 Status (after indexing) 485 + 486 + ```bash 487 + opake-appview --config-dir /tmp/opake-appview-test status 488 + # Cursor: 2026-03-02T... 489 + # Lag: <small number>s 490 + # Grants: <non-zero> 491 + # Keyrings: <number> 492 + ``` 493 + 494 + ### 7.8 Config dir matches CLI 495 + 496 + Both binaries should resolve the same config directory: 497 + 498 + ```bash 499 + # Both use OPAKE_DATA_DIR 500 + OPAKE_DATA_DIR=/tmp/opake-appview-test opake-appview status 501 + # should find /tmp/opake-appview-test/appview.toml 502 + 503 + # --config-dir flag works the same way 504 + opake-appview --config-dir /tmp/opake-appview-test status 505 + ``` 506 + 507 + ### 7.9 Cleanup 508 + 509 + ```bash 510 + kill $APPVIEW_PID 2>/dev/null 511 + rm -rf /tmp/opake-appview-test 512 + ``` 513 + 514 + --- 515 + 516 + ## 8. Cleanup 408 517 409 518 ```bash 410 519 # remove test files
+19 -11
CONTRIBUTING.md
··· 14 14 - `cargo fmt` before every commit (enforced by CI and pre-commit hook) 15 15 - `cargo clippy -- -D warnings` must pass 16 16 - No `unwrap()` in library code — use `?` and proper error types 17 - - `opake-core` uses `thiserror` for typed errors; `opake-cli` uses `anyhow` for application errors 17 + - `opake-core` uses `thiserror` for typed errors; `opake-cli` and `opake-appview` use `anyhow` for application errors 18 18 - Prefer `&str` parameters, `String` for owned data 19 19 - Avoid `.clone()` unless necessary 20 20 21 21 ## Architecture 22 22 23 23 ``` 24 - opake-core platform-agnostic library (compiles to WASM) 25 - - encryption/decryption (AES-256-GCM, x25519 key wrapping) 26 - - XRPC client with automatic token refresh 27 - - document operations (upload, download, list, delete, resolve) 28 - - AT Protocol record types and lexicon constants 24 + opake-core platform-agnostic library (compiles to WASM) 25 + - encryption/decryption (AES-256-GCM, x25519 key wrapping) 26 + - XRPC client with automatic token refresh 27 + - document operations (upload, download, list, delete, resolve) 28 + - AT Protocol record types and lexicon constants 29 + - shared config path resolution (paths.rs) 29 30 30 - opake-cli thin CLI wrapper 31 - - clap command definitions 32 - - config/session/identity persistence 33 - - user interaction (prompts, formatting) 31 + opake-cli thin CLI wrapper 32 + - clap command definitions 33 + - config/session/identity persistence 34 + - user interaction (prompts, formatting) 35 + 36 + opake-appview indexer + REST API for grant/keyring discovery 37 + - Jetstream firehose consumer 38 + - SQLite storage (WAL mode) 39 + - Axum API with DID-scoped Ed25519 auth 40 + - rate limiting via tower_governor 34 41 ``` 35 42 36 - `opake-core` must never depend on filesystem, stdin, or any platform-specific API. All I/O happens at the CLI layer. 43 + `opake-core` must never depend on filesystem, stdin, or any platform-specific API. All I/O happens in the binary crates. 37 44 38 45 See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the detailed crate structure and encryption model, and [docs/FLOWS.md](docs/FLOWS.md) for sequence diagrams of every operation. 39 46 ··· 50 57 cargo test # all tests 51 58 cargo test -p opake-core # core only 52 59 cargo test -p opake-cli # CLI only 60 + cargo test -p opake-appview # appview only 53 61 cargo test -- --test-output # show println output 54 62 ``` 55 63
+19 -5
README.md
··· 18 18 → store metadata as app.opake.cloud.document record 19 19 ``` 20 20 21 - No middleware, no AppView, no modifications to the PDS. All crypto happens on your machine. 21 + No modifications to the PDS. All crypto happens on your machine. 22 22 23 23 ## Build From Source 24 24 ··· 30 30 cargo build --release 31 31 ``` 32 32 33 - The binary lands at `target/release/opake`. 33 + Produces two binaries: `target/release/opake` (CLI) and `target/release/opake-appview` (indexer/API server). 34 34 35 35 ## Usage 36 36 ··· 88 88 89 89 The `--as` flag works with document commands (`upload`, `download`, `ls`, `rm`, `share`, `shared`, `revoke`) and accepts a handle or DID. 90 90 91 + ## AppView 92 + 93 + The AppView is a separate binary (`opake-appview`) that indexes grants and keyrings from the AT Protocol firehose and serves them via a REST API. It enables grant discovery — "what's been shared with me?" — without scanning every PDS in the network. 94 + 95 + See [docs/appview.md](docs/appview.md) for configuration, authentication, API endpoints, and deployment. 96 + 91 97 ## Architecture 92 98 93 - Two crates: `opake-core` (platform-agnostic library, compiles to WASM) and `opake-cli` (thin CLI wrapper). See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the encryption model, crate structure, and design decisions. See [docs/FLOWS.md](docs/FLOWS.md) for sequence diagrams of every operation. 99 + Three crates: 100 + 101 + - **`opake-core`** — platform-agnostic library (compiles to WASM). Encryption, records, XRPC client, document operations. 102 + - **`opake-cli`** — thin CLI wrapper. Config, session, identity persistence. 103 + - **`opake-appview`** — Axum-based indexer and REST API. Jetstream firehose consumer, SQLite storage, DID-scoped Ed25519 auth. 104 + 105 + See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the encryption model, crate structure, and design decisions. See [docs/FLOWS.md](docs/FLOWS.md) for sequence diagrams of every operation. 94 106 95 107 ## Roadmap 96 108 ··· 104 116 - [x] Direct file sharing between DIDs 105 117 - [x] Cross-PDS shared file download (via --grant flag) 106 118 - [x] Grant listing (shared command) 107 - - [ ] Grant discovery (inbox command) 119 + - [x] AppView indexer (grants + keyrings from firehose) 120 + - [x] AppView REST API with DID-scoped Ed25519 auth 121 + - [ ] Grant discovery (inbox command — queries AppView) 108 122 - [ ] Keyring-based group sharing 109 123 - [ ] Folder hierarchy 110 - - [ ] Web UI (Rust/Axum AppView + SPA) 124 + - [ ] Web UI (SPA frontend) 111 125 112 126 ## Development 113 127
+2 -2
crates/opake-appview/src/api/health.rs
··· 5 5 use axum::response::{IntoResponse, Json}; 6 6 use serde::Serialize; 7 7 8 - use crate::db::cursor; 8 + use crate::db::cursor::{self, MICROS_PER_SECOND}; 9 9 use crate::state::AppState; 10 10 11 11 #[derive(Serialize)] ··· 23 23 24 24 let (cursor_time, cursor_age_secs) = match cursor_us { 25 25 Some(us) => { 26 - let secs = us / 1_000_000; 26 + let secs = us / MICROS_PER_SECOND; 27 27 let now = chrono::Utc::now().timestamp(); 28 28 let time = chrono::DateTime::from_timestamp(secs, 0).map(|dt| dt.to_rfc3339()); 29 29 (time, Some(now - secs))
+2 -1
crates/opake-appview/src/commands/status.rs
··· 2 2 3 3 use crate::config::Config; 4 4 use crate::db; 5 + use crate::db::cursor::MICROS_PER_SECOND; 5 6 6 7 use super::build_state; 7 8 ··· 23 24 24 25 match cursor_us { 25 26 Some(us) => { 26 - let cursor_secs = us / 1_000_000; 27 + let cursor_secs = us / MICROS_PER_SECOND; 27 28 let now_secs = chrono::Utc::now().timestamp(); 28 29 let lag_secs = now_secs - cursor_secs; 29 30
+3
crates/opake-appview/src/db/cursor.rs
··· 2 2 3 3 use crate::error::Result; 4 4 5 + /// Jetstream cursors are unix microsecond timestamps. 6 + pub const MICROS_PER_SECOND: i64 = 1_000_000; 7 + 5 8 /// Save the Jetstream cursor (unix microseconds timestamp). 6 9 /// Uses upsert into the singleton row (id = 1). 7 10 pub fn save_cursor(conn: &Connection, time_us: i64) -> Result<()> {
+58 -7
docs/ARCHITECTURE.md
··· 10 10 Crypto["Client-side crypto<br/>(AES-256-GCM, X25519)"] 11 11 end 12 12 13 + subgraph Server ["AppView (self-hosted)"] 14 + AppView["opake-appview"] 15 + SQLite["SQLite"] 16 + end 17 + 13 18 subgraph Network ["AT Protocol Network"] 14 19 OwnPDS["Your PDS"] 15 20 OtherPDS["Other user's PDS"] 16 21 PLC["PLC Directory"] 22 + Jetstream["Jetstream firehose"] 17 23 end 18 24 19 25 CLI --> Core ··· 21 27 Core -->|XRPC / HTTPS| OwnPDS 22 28 Core -->|unauthenticated| OtherPDS 23 29 Core -->|DID resolution| PLC 30 + CLI -->|inbox query| AppView 31 + 32 + AppView -->|subscribe| Jetstream 33 + AppView --> SQLite 34 + Jetstream -.->|events from| OwnPDS 35 + Jetstream -.->|events from| OtherPDS 24 36 25 37 OwnPDS -.->|federation / sync| OtherPDS 26 38 27 39 style Client fill:#1a1a2e,color:#eee 40 + style Server fill:#0f3460,color:#eee 28 41 style Network fill:#16213e,color:#eee 29 42 ``` 30 43 31 - The CLI talks directly to PDS instances over XRPC. No middleware, no AppView, no PDS modifications. All encryption and decryption happens on your machine. 44 + The CLI talks directly to PDS instances over XRPC. No PDS modifications needed. All encryption and decryption happens on your machine. The AppView is an optional component that indexes grants and keyrings from the firehose for discovery. 32 45 33 46 ## Crate Structure 34 47 ··· 88 101 main.rs Clap app, command dispatch 89 102 config.rs Multi-account config (default DID, account map) 90 103 session.rs Per-account session persistence (JWT tokens) 91 - identity.rs Per-account X25519 keypair persistence 104 + identity.rs Per-account X25519 + Ed25519 keypair persistence 92 105 keyring_store.rs Local group key persistence (per-keyring) 93 106 transport.rs reqwest-based Transport implementation 94 107 utils.rs Test harness, env helpers ··· 106 119 accounts.rs List accounts 107 120 logout.rs Remove account 108 121 set_default.rs Switch default account 122 + 123 + opake-appview/ Indexer + REST API for grant/keyring discovery 124 + src/ 125 + main.rs Clap app (run/index/serve/status subcommands) 126 + config.rs AppView config (appview.toml) 127 + state.rs AppState (Database, indexer status, key cache) 128 + error.rs Typed error hierarchy (thiserror) 129 + indexer.rs Event loop: firehose → parse → store 130 + api/ 131 + mod.rs Axum router (public + protected routes, rate limiting) 132 + auth.rs DID-scoped Ed25519 auth middleware 133 + key_cache.rs Signing key cache with TTL 134 + health.rs GET /api/health (unauthenticated) 135 + inbox.rs GET /api/inbox (grants by recipient DID) 136 + keyrings.rs GET /api/keyrings (memberships by DID) 137 + types.rs API response types 138 + db/ 139 + mod.rs Database wrapper (SQLite, WAL mode) 140 + schema.rs Table definitions 141 + cursor.rs Firehose cursor persistence 142 + grants.rs Grant upsert/query/delete 143 + keyrings.rs Keyring member upsert/query/delete 144 + firehose/ 145 + mod.rs Re-exports 146 + subscribe.rs WebSocket connection to Jetstream 147 + events.rs Event parsing → IndexableEvent 148 + commands/ 149 + mod.rs Shared helpers (build_state, serve_http) 150 + run.rs Indexer + API (default) 151 + index.rs Indexer only 152 + serve.rs API only 153 + status.rs Print cursor + stats 109 154 ``` 110 155 111 - The boundary is strict: `opake-core` never touches the filesystem, stdin, or any platform-specific API. All I/O happens in `opake-cli`. This keeps `opake-core` compilable to WASM for the future web UI. 156 + The boundary is strict: `opake-core` never touches the filesystem, stdin, or any platform-specific API. All I/O happens in the binary crates. This keeps `opake-core` compilable to WASM for the future web UI. 112 157 113 158 ## Encryption Model 114 159 ··· 142 187 143 188 ### Public Key Discovery 144 189 145 - AT Protocol DID documents only contain signing keys (secp256k1/P-256), not encryption keys. Opake publishes X25519 encryption public keys as `app.opake.cloud.publicKey/self` singleton records on each user's PDS. Key discovery is an unauthenticated `getRecord` call — no auth needed to look up someone's public key. 190 + AT Protocol DID documents only contain signing keys (secp256k1/P-256), not encryption keys. Opake publishes `app.opake.cloud.publicKey/self` singleton records on each user's PDS containing: 191 + 192 + - **X25519 encryption public key** — used for key wrapping (sharing) 193 + - **Ed25519 signing public key** — used for AppView authentication 146 194 147 - The key is published automatically on every `opake login` via an idempotent `putRecord`. 195 + Key discovery is an unauthenticated `getRecord` call — no auth needed to look up someone's public key. Both keys are published automatically on every `opake login` via an idempotent `putRecord`. 148 196 149 197 ## Data Model 150 198 ··· 208 256 - X25519 keypair 209 257 - PDS URL and handle 210 258 259 + Both binaries resolve their config directory through the same chain: `--config-dir` flag → `OPAKE_DATA_DIR` env → `XDG_CONFIG_HOME/opake` → `~/.config/opake`. Resolution logic lives in `opake-core/src/paths.rs`. 260 + 211 261 Storage layout: 212 262 213 263 ``` 214 264 ~/.config/opake/ 215 - config.toml Global config (default DID, account map) 265 + config.toml CLI config (default DID, account map) 266 + appview.toml AppView config (jetstream URL, listen addr, db path) 216 267 accounts/ 217 268 <did>/ 218 269 session.json JWT tokens 219 - identity.json X25519 keypair (plaintext for MVP) 270 + identity.json X25519 + Ed25519 keypairs (plaintext for MVP) 220 271 keyrings/ 221 272 <rkey>.json Group keys for each keyring (per-rotation) 222 273 ```