# NSID Autocomplete Implementation Plan

Redis-based autocomplete for lexicon NSIDs using `FT.SUGADD`/`FT.SUGGET` commands.

## Overview

- **Index command**: `FT.SUGADD autocomplete {nsid} 1.0 PAYLOAD "{did} {nsid}"`
- **Query command**: `FT.SUGGET autocomplete "{query}" FUZZY WITHPAYLOADS MAX 10`
- **API endpoint**: `GET /api/autocomplete-nsid?q={query}`
- **UI**: Search input in navbar with dropdown autocomplete

## Prerequisites

- **Redis Stack** or **RediSearch module** required (standard Redis does not support `FT.SUG*` commands)
- Existing `REDIS_CACHE_ENABLED` and `REDIS_URL` configuration will be reused

---

## Implementation Steps

### 1. Extend Redis Cache Module

**File**: `src/cache/redis.rs`

Add autocomplete methods to `RedisCache`:

```rust
// Constants
const AUTOCOMPLETE_KEY: &str = "lexicon:autocomplete";

// New struct for results
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AutocompleteSuggestion {
    pub nsid: String,
    pub did: String,
}

impl RedisCache {
    /// FT.SUGADD autocomplete {nsid} 1.0 PAYLOAD "{did} {nsid}"
    pub async fn add_nsid_suggestion(&self, nsid: &str, did: &str) { ... }

    /// FT.SUGDEL autocomplete {nsid}
    pub async fn remove_nsid_suggestion(&self, nsid: &str) { ... }

    /// FT.SUGGET autocomplete "{query}" FUZZY WITHPAYLOADS MAX {limit}
    pub async fn get_nsid_suggestions(&self, query: &str, limit: Option<usize>) -> Vec<AutocompleteSuggestion> { ... }

    /// FT.SUGLEN autocomplete
    pub async fn get_suggestion_count(&self) -> Option<i64> { ... }
}
```

Use `deadpool_redis::redis::cmd()` for raw commands since `FT.SUG*` aren't in `AsyncCommands`.

**Update**: `src/cache/mod.rs` - export `AutocompleteSuggestion`

---

### 2. TAP Consumer Integration

**File**: `src/tap_consumer.rs`

Add autocomplete indexing inside the `if is_authoritative` block (after graph edge insertion, around line ~260):

```rust
// Inside the `if is_authoritative { ... }` block, after graph edge processing:

// Add NSID to autocomplete index (best-effort, authoritative only)
if let Some(ref cache) = self.redis_cache {
    cache.add_nsid_suggestion(nsid, did).await;
}
```

This ensures only authoritative schemas are indexed for autocomplete, matching the behavior of `handle_admin_autocomplete_rebuild`.

Note: No need to remove on delete since the rebuild can clean up stale entries.

---

### 3. Admin Backfill Endpoint (Authoritative Only)

**File**: `src/http/handle_admin.rs`

Add handler following the same pattern as `handle_admin_graph_rebuild` - only indexes authoritative, non-deleted schemas:

```rust
/// POST /admin/autocomplete/rebuild
/// Only indexes authoritative schemas (where the NSID authority matches the record DID)
pub async fn handle_admin_autocomplete_rebuild(
    State(web_context): State<WebContext>,
) -> Result<impl IntoResponse, WebError> {
    let redis_cache = web_context.redis_cache.as_ref()
        .ok_or_else(|| WebError::Internal(anyhow::anyhow!("Redis not configured")))?;

    // Get all distinct NSIDs (reuse existing function)
    let nsids = schema_list_distinct_nsids(&web_context.pool).await?;
    tracing::info!(nsid_count = nsids.len(), "Rebuilding autocomplete index from authoritative schemas");

    let mut indexed = 0usize;
    let mut authority_failures = 0usize;

    // For each NSID, resolve authority and index only if authoritative schema exists
    for nsid in &nsids {
        // Resolve the authority DID for this NSID
        let authority_did = match resolve_schema_authority(
            &web_context.pool,
            web_context.dns_resolver.as_ref(),
            nsid,
            web_context.redis_cache.as_ref(),
        )
        .await
        {
            Ok(did) => did,
            Err(e) => {
                tracing::debug!(nsid = %nsid, error = %e, "Failed to resolve authority during autocomplete rebuild");
                authority_failures += 1;
                continue;
            }
        };

        // Construct the AT-URI for the authoritative schema
        let aturi = format!("at://{}/com.atproto.lexicon.schema/{}", authority_did, nsid);

        // Get the latest schema from the authoritative source
        let schema = match schema_get_latest(&web_context.pool, &aturi).await? {
            Some(s) if s.deleted_at.is_none() => s,
            _ => {
                tracing::debug!(
                    nsid = %nsid,
                    authority_did = %authority_did,
                    "No authoritative schema found during autocomplete rebuild"
                );
                continue;
            }
        };

        // Add to autocomplete index
        redis_cache.add_nsid_suggestion(&schema.nsid, &schema.did).await;
        indexed += 1;
    }

    tracing::info!(
        indexed = indexed,
        authority_failures = authority_failures,
        "Autocomplete index rebuild complete"
    );

    Ok(Redirect::to("/admin"))
}
```

**File**: `src/http/server.rs` - Add routes under admin nest:
```rust
.route("/autocomplete", get(handle_admin_autocomplete))
.route("/autocomplete/rebuild", post(handle_admin_autocomplete_rebuild))
```

**File**: `templates/admin/autocomplete.html` - Admin page template showing:
- Indexed NSIDs count
- Redis connection status
- Rebuild Index button

**Note**: No new storage function needed - reuses existing `schema_list_distinct_nsids` and `schema_get_latest`.

---

### 4. Autocomplete API Endpoint

**File**: `src/http/handle_autocomplete.rs` (new)

```rust
#[derive(Debug, Deserialize)]
pub struct AutocompleteQuery {
    pub q: String,
    pub limit: Option<usize>,
}

#[derive(Debug, Serialize)]
pub struct AutocompleteResponse {
    pub suggestions: Vec<AutocompleteSuggestionItem>,
}

#[derive(Debug, Serialize)]
pub struct AutocompleteSuggestionItem {
    pub nsid: String,
    pub did: String,
    pub url: String,  // e.g., "/lexicon/{did}/{nsid}"
}

/// GET /api/autocomplete-nsid?q={query}&limit={limit}
pub async fn handle_autocomplete_nsid(
    State(web_context): State<WebContext>,
    Query(query): Query<AutocompleteQuery>,
) -> impl IntoResponse {
    // Validate: return empty if q < 2 chars
    // Cap limit at 20
    // Call redis_cache.get_nsid_suggestions() if available
    // Return JSON response
}
```

**File**: `src/http/mod.rs` - Add `pub mod handle_autocomplete;`

**File**: `src/http/server.rs` - Add route:
```rust
.route("/api/autocomplete-nsid", get(handle_autocomplete_nsid))
```

---

### 5. Navbar Search UI

**File**: `templates/base.html`

Replace nav section (lines 100-108):

```html
<header class="container">
    <nav>
        <ul>
            <li><strong><a href="/">Lexicon Garden</a></strong></li>
        </ul>
        <ul>
            <li style="position: relative;">
                <input type="search" id="nsid-search" placeholder="Search NSIDs..."
                       autocomplete="off" style="margin-bottom: 0; min-width: 200px;">
                <ul id="autocomplete-dropdown" class="autocomplete-dropdown"></ul>
            </li>
            <li><a href="/lexicon">Graph</a></li>
        </ul>
    </nav>
</header>
```

Add CSS for dropdown (in `<style>` block):

```css
.autocomplete-dropdown {
    position: absolute;
    top: 100%;
    left: 0;
    right: 0;
    z-index: 1000;
    background: var(--pico-background-color);
    border: 1px solid var(--pico-muted-border-color);
    border-radius: 0 0 var(--pico-border-radius) var(--pico-border-radius);
    list-style: none;
    margin: 0;
    padding: 0;
    max-height: 300px;
    overflow-y: auto;
    display: none;
}
.autocomplete-dropdown li { padding: 0.5rem 1rem; cursor: pointer; }
.autocomplete-dropdown li:hover, .autocomplete-dropdown li.selected {
    background: var(--pico-primary-background);
    color: var(--pico-primary-inverse);
}
.autocomplete-dropdown .nsid { font-weight: 500; }
.autocomplete-dropdown .did { font-size: 0.75rem; color: var(--pico-muted-color); }
```

Add JavaScript before `{% block scripts %}`:

```html
<script>
(function() {
    const input = document.getElementById('nsid-search');
    const dropdown = document.getElementById('autocomplete-dropdown');
    if (!input || !dropdown) return;

    let timer, selected = -1, items = [];

    async function search(q) {
        if (q.length < 2) { dropdown.style.display = 'none'; return; }
        const res = await fetch(`/api/autocomplete-nsid?q=${encodeURIComponent(q)}`);
        const data = await res.json();
        items = data.suggestions || [];
        render();
    }

    function render() {
        if (!items.length) { dropdown.innerHTML = '<li>No results</li>'; }
        else {
            dropdown.innerHTML = items.map((s, i) =>
                `<li data-i="${i}" data-url="${s.url}">
                    <div class="nsid">${s.nsid}</div>
                    <div class="did">${s.did}</div>
                </li>`).join('');
        }
        dropdown.style.display = 'block';
        selected = -1;
    }

    input.addEventListener('input', e => {
        clearTimeout(timer);
        timer = setTimeout(() => search(e.target.value.trim()), 200);
    });

    input.addEventListener('keydown', e => {
        if (dropdown.style.display === 'none') return;
        if (e.key === 'ArrowDown') { selected = Math.min(selected + 1, items.length - 1); update(); e.preventDefault(); }
        if (e.key === 'ArrowUp') { selected = Math.max(selected - 1, 0); update(); e.preventDefault(); }
        if (e.key === 'Enter' && selected >= 0) { window.location.href = items[selected].url; e.preventDefault(); }
        if (e.key === 'Escape') { dropdown.style.display = 'none'; }
    });

    function update() {
        dropdown.querySelectorAll('li').forEach((li, i) => li.classList.toggle('selected', i === selected));
    }

    dropdown.addEventListener('click', e => {
        const li = e.target.closest('li[data-url]');
        if (li) window.location.href = li.dataset.url;
    });

    document.addEventListener('click', e => {
        if (!input.contains(e.target) && !dropdown.contains(e.target)) dropdown.style.display = 'none';
    });
})();
</script>
```

---

## Files to Modify

| File | Changes |
|------|---------|
| `src/cache/redis.rs` | Add `AutocompleteSuggestion`, `add_nsid_suggestion`, `remove_nsid_suggestion`, `get_nsid_suggestions`, `get_suggestion_count` |
| `src/cache/mod.rs` | Export `AutocompleteSuggestion` |
| `src/tap_consumer.rs` | Call `add_nsid_suggestion` after schema insert (authoritative only) |
| `src/http/handle_autocomplete.rs` | New file - autocomplete API endpoint |
| `src/http/handle_admin.rs` | Add `handle_admin_autocomplete` and `handle_admin_autocomplete_rebuild` |
| `src/http/mod.rs` | Add `pub mod handle_autocomplete` |
| `src/http/server.rs` | Add routes for `/api/autocomplete-nsid`, `/admin/autocomplete`, and `/admin/autocomplete/rebuild` |
| `templates/base.html` | Add search input, CSS, and JavaScript |
| `templates/admin/autocomplete.html` | New file - admin page for autocomplete management |

**Note**: No new storage functions needed - reuses existing `schema_list_distinct_nsids`, `schema_get_latest`, and `resolve_schema_authority`.

---

## Graceful Degradation

- If Redis is not configured (`redis_cache: None`), API returns empty results
- If RediSearch commands fail (no module), errors are logged but application continues
- Search input remains visible but returns no suggestions

---

## Testing

1. Start Redis Stack locally: `docker run -p 6379:6379 redis/redis-stack-server`
2. Run app with `REDIS_CACHE_ENABLED=true REDIS_URL=redis://localhost:6379`
3. Trigger admin backfill: `POST /admin/autocomplete/rebuild`
4. Test API: `curl "http://localhost:4800/api/autocomplete-nsid?q=app.bsky"`
5. Test UI: Type in navbar search input