a rough sketch for a communal task orchestrator built on AT
meshlet.md edited
65 lines 3.0 kB view raw view code

(summarized by Claude)

A decentralized, consent-based container orchestration system where commune members contribute compute resources and run each other's workloads. Built on AT infrastructure with mutual trust and fair-share principles.

Identity Model

  • Owner DIDs: Real people (e.g., did:plc:alice)
  • Meshlet DIDs: Each meshlet gets its own DID (e.g., did:plc:alice-mesh)
  • Two-way linkage:
    • Owner's repo: commune.scheduler.meshlet { meshletDid: "did:plc:alice-mesh" }
    • Meshlet's repo: commune.scheduler.meshletOwner { ownerDid: "did:plc:alice" }

Cluster Formation

meshlet --did=did:plc:alice-mesh \     # ← this would likely be implicit? maybe this could even just be a did:web:mesh.alice.com
        --owner=did:plc:alice \
        --cluster-members=did:plc:bob,did:plc:carol  # owner DIDs

Cool thing about this is Alice's meshlet can be in multiple clusters -- insofar as the other meshlet nodes recognize her. There can be cluster overlap.

Startup process:

  1. For each member owner DID, resolve to their meshlet DID via their commune.scheduler.meshlet record
  2. Verify mutual membership - check if their meshlet also lists your owner DID
  3. Verify meshlet ownership - confirm meshlet→owner linkage
  4. Form active cluster with only mutually-verified members
  5. Members can participate in multiple overlapping clusters

Workload Scheduling (Decentralized)

No central scheduler - consensus via deterministic selection:

  1. Workload request: Alice publishes commune.scheduler.workload to her owner repo

    • Resource requirements (CPU, memory, storage)
    • Container image
    • Replica count
    • Constraints
  2. Bid publication: Meshlets see workload on firehose, evaluate capacity/fairness, publish commune.scheduler.bid to their meshlet repos

    • Time window for bids (e.g., 5 seconds)
    • Bids include priority score based on fair-share balance
  3. Deterministic selection: All meshlets independently:

    • Collect all bids from firehose
    • Run identical selection algorithm (sort by priority + DID, take top N)
    • Everyone reaches same conclusion about who runs it
  4. Execution: Selected meshlets pull image and start containers

  5. Status updates: Meshlets write commune.scheduler.workloadStatus to their repos

Communication Patterns

  • Firehose: Workload discovery, bid coordination, status updates (all via atproto repos)
  • XRPC: Direct meshlet operations
    • farm.meshlet.getCapacity
    • farm.meshlet.listWorkloads
    • farm.meshlet.getWorkloadStatus
    • farm.meshlet.deleteWorkload
    • etc. for pause/stop/shutdown meshlet etc.

Open Questions

  • Secrets management: Sealed secrets per meshlet? Shared vault? Secret sharing?
  • Fair-share accounting: Local tracking vs distributed ledger?
  • Failure handling: What happens when meshlets go offline mid-workload?
  • Governance: Handling free-riders, minimum contributions, dispute resolution?
  • Appview: What would this look like?