skywatch.blue
A tool for parsing traffic on the jetstream and applying a moderation workstream based on regexp based rules
Add some documentation
+322
-21
+5
-5
.env.example
+5
-5
.env.example
···
3
3
OZONE_PDS=
4
4
BSKY_HANDLE=
5
5
BSKY_PASSWORD=
6
-
HOST=127.0.0.1
7
-
PORT=4000
8
-
METRICS_PORT=4001
9
-
FIREHOSE_URL=
10
-
PLC_URL=plc.wtf
6
+
HOST=0.0.0.0
7
+
METRICS_PORT=4101
8
+
FIREHOSE_URL=wss://jetstream1.us-east.fire.hose.cam/subscribe
11
9
CURSOR_UPDATE_INTERVAL=10000
12
10
LABEL_LIMIT=2900 * 1000
13
11
LABEL_LIMIT_WAIT=300 * 1000
12
+
LOG_LEVEL=info
13
+
PLC_URL=plc.wtf
+41
-14
README.md
+41
-14
README.md
···
1
-
# skywatch-tools
1
+
# skywatch-automod
2
2
3
-
This is a rewrite of the original skywatch-tools project in TypeScript. The original project was written in Bash. The purpose of this project is to automate the moderation by the Bluesky independent labeler skywatch.blue
3
+
Automated moderation tooling for the Bluesky independent labeler skywatch.blue. Monitors the Bluesky firehose and applies labels based on configured moderation rules.
4
4
5
-
## Installation and Setup
5
+
## Setup
6
6
7
-
To install dependencies:
7
+
Configure environment:
8
8
9
9
```bash
10
-
bun i
10
+
cp .env.example .env
11
+
# Edit .env with your credentials and configuration
11
12
```
12
13
13
-
Modify .env.example with your own values and rename it to .env
14
+
Required environment variables:
15
+
- `BSKY_HANDLE` - Bluesky account handle
16
+
- `BSKY_PASSWORD` - Account password
17
+
- `MOD_DID` - Moderator DID
18
+
- `OZONE_PDS` - Ozone PDS URL
19
+
- `FIREHOSE_URL` - Jetstream firehose URL
20
+
21
+
Create cursor file (optional but recommended):
22
+
23
+
```bash
24
+
touch cursor.txt
25
+
```
26
+
27
+
## Running
28
+
29
+
Production:
14
30
15
31
```bash
16
-
bun run start
32
+
docker compose up -d
17
33
```
18
34
19
-
To run in docker:
35
+
Development mode with auto-reload:
20
36
21
37
```bash
22
-
docker build -pull -t skywatch-tools .
23
-
docker run -d -p 4101:4101 skywatch-autolabeler
38
+
docker compose -f compose.yaml -f compose.dev.yaml up
24
39
```
25
40
26
-
## Brief overview
41
+
The service runs on port 4101 (metrics endpoint). Redis and Prometheus are included in the compose stack.
42
+
43
+
## Authentication
44
+
45
+
The application authenticates with Bluesky on startup and retries up to 3 times on failure. If all attempts fail, the application exits. Sessions are cached in `.session` (gitignored).
46
+
47
+
## Testing
48
+
49
+
```bash
50
+
bun test # Watch mode
51
+
bun test:run # Single run
52
+
bun test:coverage # With coverage
53
+
```
27
54
28
-
Currently this tooling does one thing. It monitors the bluesky firehose and analyzes content for phrases which fit Skywatch's criteria for moderation. If the criteria is met, it can automatically label the content with the appropriate label.
55
+
## How It Works
29
56
30
-
In certain cases, where regexp will create too many false positives, it will flag content as a report against related to the account, so that it can be reviewed later.
57
+
Monitors the Bluesky firehose via Jetstream and analyzes posts, profiles, and handles against configured moderation rules. When criteria are met, applies appropriate labels or creates moderation reports.
31
58
32
-
For information on how to set-up your own checks, please see the [developing_checks.md](./src/developing_checks.md) file.
59
+
For developing custom checks, see [developing_checks.md](./rules/developing_checks.md).
+2
-2
package.json
+2
-2
package.json
+274
rules/developing_checks.md
+274
rules/developing_checks.md
···
1
+
# Developing Moderation Checks
2
+
3
+
This guide explains how to configure moderation rules for skywatch-automod.
4
+
5
+
## Overview
6
+
7
+
Moderation checks are defined in TypeScript files in the `rules/` directory. Each check uses regular expressions to match content and specifies what action to take when a match is found.
8
+
9
+
## Check Types
10
+
11
+
### Post Content Checks
12
+
13
+
File: `rules/posts.ts`
14
+
15
+
Monitors post text and embedded URLs for matches.
16
+
17
+
```typescript
18
+
import type { Checks } from "../src/types.js";
19
+
20
+
export const POST_CHECKS: Checks[] = [
21
+
{
22
+
label: "spam",
23
+
comment: "Spam content detected in post",
24
+
reportAcct: false,
25
+
commentAcct: false,
26
+
toLabel: true,
27
+
check: new RegExp("buy.*followers", "i"),
28
+
},
29
+
];
30
+
```
31
+
32
+
### Handle Checks
33
+
34
+
File: `rules/handles.ts`
35
+
36
+
Monitors user handles for pattern matches.
37
+
38
+
```typescript
39
+
export const HANDLE_CHECKS: Checks[] = [
40
+
{
41
+
label: "impersonation",
42
+
comment: "Potential impersonation detected",
43
+
reportAcct: true,
44
+
commentAcct: false,
45
+
toLabel: false,
46
+
check: new RegExp("official.*support", "i"),
47
+
},
48
+
];
49
+
```
50
+
51
+
### Profile Checks
52
+
53
+
File: `rules/profiles.ts`
54
+
55
+
Monitors profile display names and descriptions.
56
+
57
+
```typescript
58
+
export const PROFILE_CHECKS: Checks[] = [
59
+
{
60
+
label: "spam-profile",
61
+
comment: "Spam content in profile",
62
+
reportAcct: false,
63
+
commentAcct: false,
64
+
toLabel: true,
65
+
displayName: true, // Check display name
66
+
description: true, // Check description
67
+
check: new RegExp("follow.*back", "i"),
68
+
},
69
+
];
70
+
```
71
+
72
+
### Account Age Checks
73
+
74
+
File: `rules/accountAge.ts`
75
+
76
+
Labels accounts created after a specific date when they interact with monitored content.
77
+
78
+
```typescript
79
+
import type { AccountAgeCheck } from "../src/types.js";
80
+
81
+
export const ACCOUNT_AGE_CHECKS: AccountAgeCheck[] = [
82
+
{
83
+
monitoredDIDs: ["did:plc:abc123"],
84
+
anchorDate: "2025-01-15",
85
+
maxAgeDays: 7,
86
+
label: "new-account-spam",
87
+
comment: "New account replying to monitored user",
88
+
expires: "2025-02-15", // Optional expiration
89
+
},
90
+
];
91
+
```
92
+
93
+
### Account Threshold Checks
94
+
95
+
File: `rules/accountThreshold.ts`
96
+
97
+
Applies account-level labels when an account accumulates multiple post-level violations within a time window.
98
+
99
+
```typescript
100
+
import type { AccountThresholdConfig } from "../src/types.js";
101
+
102
+
export const ACCOUNT_THRESHOLD_CONFIGS: AccountThresholdConfig[] = [
103
+
{
104
+
labels: ["spam", "scam"], // Trigger on either label
105
+
threshold: 3,
106
+
accountLabel: "repeat-offender",
107
+
accountComment: "Account exceeded spam threshold",
108
+
windowDays: 7,
109
+
reportAcct: true,
110
+
commentAcct: false,
111
+
toLabel: true,
112
+
},
113
+
];
114
+
```
115
+
116
+
## Check Configuration Fields
117
+
118
+
### Basic Fields (Required)
119
+
120
+
- `label` - Label to apply (string)
121
+
- `comment` - Comment for the moderation action (string)
122
+
- `reportAcct` - Create account report (boolean)
123
+
- `commentAcct` - Add comment to account (boolean)
124
+
- `toLabel` - Apply the label (boolean)
125
+
- `check` - Regular expression pattern (RegExp)
126
+
127
+
### Optional Fields
128
+
129
+
- `language` - Language codes to restrict check to (string[])
130
+
- `description` - Check profile descriptions (boolean)
131
+
- `displayName` - Check profile display names (boolean)
132
+
- `reportPost` - Create post report instead of just labeling (boolean)
133
+
- `duration` - Label duration in hours (number)
134
+
- `whitelist` - RegExp to exclude from matching (RegExp)
135
+
- `ignoredDIDs` - DIDs to skip checking (string[])
136
+
- `starterPacks` - Filter by starter pack membership (string[])
137
+
- `knownVectors` - Known attack vectors for tracking (string[])
138
+
139
+
## Examples
140
+
141
+
### Language-Specific Check
142
+
143
+
```typescript
144
+
{
145
+
language: ["spa"],
146
+
label: "spam-es",
147
+
comment: "Spanish spam detected",
148
+
reportAcct: false,
149
+
commentAcct: false,
150
+
toLabel: true,
151
+
check: new RegExp("comprar seguidores", "i"),
152
+
}
153
+
```
154
+
155
+
### Temporary Label
156
+
157
+
```typescript
158
+
{
159
+
label: "review-needed",
160
+
comment: "Content flagged for review",
161
+
reportAcct: true,
162
+
commentAcct: false,
163
+
toLabel: false,
164
+
duration: 24, // Label expires after 24 hours
165
+
check: new RegExp("suspicious.*pattern", "i"),
166
+
}
167
+
```
168
+
169
+
### Whitelist Exception
170
+
171
+
```typescript
172
+
{
173
+
label: "blocked-term",
174
+
comment: "Blocked term used",
175
+
reportAcct: false,
176
+
commentAcct: false,
177
+
toLabel: true,
178
+
check: new RegExp("\\bterm\\b", "i"),
179
+
whitelist: new RegExp("legitimate.*context", "i"),
180
+
}
181
+
```
182
+
183
+
### Ignored DIDs
184
+
185
+
```typescript
186
+
{
187
+
label: "blocked-term",
188
+
comment: "Blocked term used",
189
+
reportAcct: false,
190
+
commentAcct: false,
191
+
toLabel: true,
192
+
check: new RegExp("\\bterm\\b", "i"),
193
+
ignoredDIDs: [
194
+
"did:plc:trusted123",
195
+
"did:plc:verified456",
196
+
],
197
+
}
198
+
```
199
+
200
+
## Global Configuration
201
+
202
+
### Allowlist
203
+
204
+
File: `rules/constants.ts`
205
+
206
+
DIDs in the global allowlist bypass all checks.
207
+
208
+
```typescript
209
+
export const GLOBAL_ALLOW: string[] = [
210
+
"did:plc:trusted123",
211
+
"did:plc:verified456",
212
+
];
213
+
```
214
+
215
+
### Link Shorteners
216
+
217
+
Pattern to match URL shorteners for special handling.
218
+
219
+
```typescript
220
+
export const LINK_SHORTENER = new RegExp(
221
+
"bit\\.ly|tinyurl\\.com|goo\\.gl",
222
+
"i"
223
+
);
224
+
```
225
+
226
+
## Best Practices
227
+
228
+
### Regular Expressions
229
+
230
+
- Use word boundaries (`\\b`) to avoid partial matches
231
+
- Test patterns thoroughly to minimize false positives
232
+
- Use case-insensitive matching (`i` flag) when appropriate
233
+
- Escape special regex characters
234
+
235
+
### Action Selection
236
+
237
+
- `toLabel: true` - Apply label immediately (use for clear violations)
238
+
- `reportAcct: true` - Create report for manual review (use for ambiguous cases)
239
+
- `commentAcct: true` - Create comment on account (probably can be depreciated)
240
+
241
+
### Performance
242
+
243
+
- Keep regex patterns simple and efficient
244
+
- Use language filters to reduce unnecessary checks
245
+
- Leverage whitelists instead of complex negative lookaheads
246
+
247
+
### Testing
248
+
249
+
After modifying rules:
250
+
251
+
```bash
252
+
bun test:run
253
+
```
254
+
255
+
Test specific rule modules:
256
+
257
+
```bash
258
+
bun test src/rules/posts/tests/
259
+
```
260
+
261
+
## Deployment
262
+
263
+
Rules are mounted as a volume in docker compose:
264
+
265
+
```yaml
266
+
volumes:
267
+
- ./rules:/app/rules
268
+
```
269
+
270
+
Changes require automod rebuild:
271
+
272
+
```bash
273
+
docker compose up -d --build automod
274
+
```