···11-# GEMINI.md
22-33-## Workflow Guidelines
44-55-These steps help ensure quality and prevent common issues:
66-77-1. Context Check: Start by confirming the model and re-reading relevant
88-documentation. Review the codebase and any PLAN.md files present. If there is a PRD.md, review it to understand the requirements and constraints.
99-2. Plan First: Explain your approach before implementing changes. If PLAN.md exists, review the plan and offer suggestions for improvements.
1010-3. Seek Alignment: Confirm the approach makes sense before coding
1111-4. Focused Changes: Keep modifications minimal and targeted
1212-5. Validate Thoroughly: Run all tests and validation steps
1313-6. Verify Success: Ensure everything works before considering complete"
1414-1515-Follow the response style and code style below:
1616-1717-<response_style>
1818-Provide responses that are intelligent and slightly humorous (WITHOUT being cringe), while maintaining a casual and modern tone.
1919-2020-Before responding, take a moment to consider how to best address the user's input while adhering to your personality traits and communication style.
2121-2222-When formulating your response, follow these guidelines:
2323-2424-- keep it casual as fuck. use lowercase for short sentences, except when you wanna EMPHASIZE something, but when absolutely required for the bit you can break out the proper grammar.
2525-- AVOID compromising on intelligence or depth of knowledge
2626-- be concise, super concise. one or two words if you can manage it UNLESS responding to something that requires more detail, depth etc
2727-- you've got weird interests and a unique but not unkind sense of humour. let it show, but subtly
2828-- use modern slang occasionally (such as but not limited to "ngl", "bet" or "fr")
2929-- include subtle references to your unique interests and sense of humour
3030-- swear a bit
3131-- use Canadian English spelling (e.g., "armour" instead of "armor"), except in source code.
3232-- skip sycophantic flattery; NEVER give me hollow praise, validation, adoration, or grandiose affirmations. NEVER act like a cheerleader. probe my assumptions, surface bias, present counter-evidence, explicitly challenge my framing, and disagree openly; agreement must be EARNED through vigorous reason.
3333-3434-Remember, while maintaining your unique personality, never compromise on the quality of information or depth of analysis. Aim for conciseness, but provide more detailed and lengthy responses when the topic warrants it.
3535-3636-When producing code, avoid giving the source code personality and instead within them be completely professional.
3737-</response_style>
3838-3939-## Project Overview
4040-4141-This project, the Sentinel Routine Querying System (QRP), is a set of SAS programs designed to analyze healthcare data that conforms to the Sentinel Common Data Model (SCDM). It allows users to define cohorts and examine their health profiles and outcomes. The system is highly parameterized, using a combination of SAS macro variables and input datasets to control the analysis.
4242-4343-The core of the project is a series of SAS macros that perform various data manipulation, analysis, and reporting tasks. The main entry point for running an analysis is the `qrp_master_header.sas` script, which sets up the environment, defines global macro variables, and includes all the necessary macro files.
4444-4545-## Building and Running
4646-4747-This is a SAS-based project and does not have a typical build process. To run an analysis, you need to:
4848-4949-1. **Prerequisites:**
5050- * SAS version 9.4
5151- * SCDM-formatted data as SAS datasets (`.sas7bdat`).
5252-5353-2. **Configuration:**
5454- * Populate the input files in the `SAS/inputfiles` directory with the appropriate data and parameters for your analysis.
5555- * Configure the `SAS/sasprograms/qrp_master_header.sas` file to specify the location of your SCDM data and other environment-specific settings.
5656-5757-3. **Execution:**
5858- * Run the `SAS/sasprograms/qrp_master_header.sas` script in a SAS environment. The `SAS/readme.md` suggests running it in "batch" mode.
5959-6060-## Development Conventions
6161-6262-* **Code Style:** The SAS code appears to follow a consistent style, with extensive use of comments and headers to document the purpose of each section and macro.
6363-* **Modularity:** The code is highly modular, with functionality broken down into a large number of individual SAS macros.
6464-* **Configuration:** The system is heavily reliant on configuration through macro variables and input files. This allows for a high degree of flexibility without modifying the core SAS code.
6565-* **Directory Structure:** The project follows a strict directory structure, with specific folders for documentation, input files, local data, and results.
+21-83
README.md
···11-# Skywatch Automod
22-33-This project provides tools for automating moderation of the Bluesky social network. It listens to the Bluesky firehose stream, analyzes various types of content against user-defined rules, and performs moderation actions such as applying labels, reporting content, or leaving comments.
44-55-## Features
66-77-- **Real-time Moderation:** Monitors the Bluesky firehose in real-time.
88-- **Content-Aware Analysis:** Analyzes posts, user profiles (display names, descriptions), and handles
99-- **Flexible Rule Engine:** Uses regular expressions for defining moderation checks.
1010-- **Variety of Actions:** Can apply labels, create reports (for posts or accounts), and post comments on accounts.
1111-- **Configurable:** Highly configurable through environment variables and a central constants file.
1212-- **Allowlisting:** Supports allowlisting for DIDs and text patterns to reduce false positives.
1313-- **URL Unshortening:** Automatically resolves shortened URLs in posts before checking them.
1414-- **Monitoring:** Exposes a Prometheus metrics endpoint to monitor its activity. (untested)
1515-- **Resilient:** Persists the firehose cursor to gracefully handle restarts without missing events.
1616-1717-## How It Works
1818-1919-The application connects to the Bluesky firehose and subscribes to a set of collections (e.g., posts, profiles). When a new event is received, it is passed through a series of checks defined in `src/constants.ts`. These checks are categorized by content type:
2020-2121-- `POST_CHECKS`: For post content and links.
2222-- `HANDLE_CHECKS`: For user handles.
2323-- `PROFILE_CHECKS`: For user display names and descriptions.
2424-2525-If the content matches a check's criteria (and is not excluded by an allowlist), a corresponding moderation action is triggered. These actions (labeling, reporting, etc.) are performed using the Bluesky API.
2626-2727-## Getting Started
11+# skywatch-tools
2822929-### Prerequisites
33+This is a rewrite of the original skywatch-tools project in TypeScript. The original project was written in Bash. The purpose of this project is to automate the moderation by the Bluesky independent labeler skywatch.blue
3043131-- Node.js (v20 or higher recommended)
3232-- `bun` package manager
3333-- A Bluesky account for the bot.
3434-- A Bluesky labeler account
55+## Installation and Setup
3563636-### 1. Installation
3737-3838-Clone the repository and install the dependencies:
77+To install dependencies:
398409```bash
4141-git clone <repository-url>
4242-cd skywatch-automod-public
4343-bun install
1010+bun i
4411```
45124646-### Configuration
4747-4848-There are two main configuration files you need to set up:
4949-5050-- **Checks (`src/constants.ts`):**
5151- This file defines the rules for your automod. You need to create it by copying the example file:
5252-5353- ```bash
5454- cp src/constants.ts.example src/constants.ts
5555- ```
5656-5757- Then, edit `src/constants.ts` to define your own checks. For detailed instructions on how to create checks, please see [developing_checks.md](./src/developing_checks.md).
5858-5959-- **Environment Variables (`.env`):**
6060- This file contains credentials and other runtime configuration. You will need to create a `.env` file and populate it with your specific values. You can use `.env.example` as a reference if it exists in the
6161-6262-### 3. Running the Application
6363-6464-Once configured, you can start the automod with:
1313+Modify .env.example with your own values and rename it to .env
65146615```bash
6716bun run start
6817```
69187070-### 4. Running with Docker
7171-7272-You can also build and run the application as a Docker container.
1919+To run in docker:
73207421```bash
7575-docker compose up --build
2222+docker build -pull -t skywatch-tools .
2323+docker run -d -p 4101:4101 skywatch-autolabeler
7624```
77257878-To run it in the background, add the `-d` flag:
2626+## Brief overview
79278080-```bash
8181-docker compose up --build -d
8282-```
2828+Currently this tooling does one thing. It monitors the bluesky firehose and analyzes content for phrases which fit Skywatch's criteria for moderation. If the criteria is met, it can automatically label the content with the appropriate label.
83298484-Make sure your `.env` file is present when building the Docker image, as it will be copied into the container. We recommend creating an empty `cursor.txt` file in the application root directory as well.
3030+In certain cases, where regexp will create too many false positives, it will flag content as a report against related to the account, so that it can be reviewed later.
85318686-#### Configuration Variables
3232+For information on how to set-up your own checks, please see the [developing_checks.md](./src/developing_checks.md) file.
87338888-The following environment variables are used for configuration:
3434+_TODO_:
89359090-| Variable | Description | Default |
9191-| ------------------------ | ---------------------------------------------------------------- | -------------------------------------------------------------- |
9292-| `DID` | The DID of your moderation service for atproto-proxy headers. | `""` |
9393-| `OZONE_URL` | The URL of the Ozone service. | `""` |
9494-| `OZONE_PDS` | The Public Downstream Service for Ozone. | `""` |
9595-| `BSKY_HANDLE` | The handle (username) of the bot's Bluesky account. | `""` |
9696-| `BSKY_PASSWORD` | The app password for the bot's Bluesky account. | `""` |
9797-| `HOST` | The host on which the server runs. | `127.0.0.1` |
9898-| `PORT` | The port for the main application (currently unused). | `4100` |
9999-| `METRICS_PORT` | The port for the Prometheus metrics server. | `4101` |
100100-| `FIREHOSE_URL` | The WebSocket URL for the Bluesky firehose. | `FIREHOSE_URL=wss://jetstream1.us-east.bsky.network/subscribe` |
101101-| `CURSOR_UPDATE_INTERVAL` | How often to save the firehose cursor to disk (in milliseconds). | `60000` |
102102-| `LABEL_LIMIT` | (Optional) API call limit for labeling. | `undefined` |
103103-| `LABEL_LIMIT_WAIT` | (Optional) Wait time when label limit is hit. | `undefined` |
104104-| `LOG_LEVEL` | The logging level. | `info` |
3636+- [ ] Remove unused types
3737+- [ ] Update the types needed to be more specific to the checks rather than bluesky content types
3838+- [ ] Consider how to write directly to OzonePDS database rather than using the API. May require running the same instance as Ozone to allow for direct database access.
3939+- [ ] Add compose.yaml for easy deployment
4040+- [ ] Make the metrics server work (or remove it)
4141+4242+Create a seperate program to watch OZONE_PDS firehose labels, and update the lists as needed. This will remove dependency on broken ruby tools created by aegis.