# v0.12.1 Migration: Extract Performance + Migration Timeout Fix

This release is a pure performance bug fix. **No manual steps are needed for most
users** — re-run `gbrain init --migrate-only` and re-run `gbrain extract all` if
desired. Both should now complete successfully on any brain size.

## What changed

Two production-blocking bugs are fixed:

1. **`gbrain extract` no longer hangs on large brains.** The N+1 dedup pre-load
   that ran 47K serial `getLinks()` calls before any work started is gone. Both
   engines already enforced uniqueness at the SQL layer; the in-memory dedup was
   redundant. Combined with new batched 100-row INSERTs, a full re-extract on a
   47K-page brain drops from "10+ min hang then ~minutes more" to "immediate
   work, ~30-60s total."

2. **v0.12.0 schema migration no longer times out on duplicate-heavy brains.**
   Migration v9 (timeline_dedup_index) and v8 (links uniqueness) now pre-create
   a btree helper index before the `DELETE ... USING` self-join, then drop it
   after dedup. Turns O(n²) dedup into O(n log n). On 80K+ duplicate rows the
   migration completes in under a second instead of timing out at 60 seconds.

## What you need to do

### If your v0.12.0 upgrade succeeded — you're already done

The migration is idempotent. The fix only matters if your migration FAILED on
the v0.12.0 upgrade attempt. Run `gbrain init --migrate-only` once to confirm
your schema is at version 10, then run `gbrain extract all --dir <brain>` if
you want to reuse it now that it's fast.

### If your v0.12.0 upgrade FAILED on `idx_timeline_dedup` creation

You may have the brain in a partial-migration state with duplicate rows in
`timeline_entries` (or `links`). Run:

```bash
gbrain init --migrate-only
```

Migration v9 will pre-create the helper index, dedup any existing duplicates
(now sub-second instead of timing out), drop the helper, and create the unique
index. Idempotent — safe to re-run.

If you previously ran the manual `CREATE TABLE _clean AS SELECT DISTINCT ON
... + table swap` workaround Garry posted, your schema should already be at
version 10. Confirm with:

```bash
gbrain config get version
```

If it shows `10`, you're done.

### If you previously did manual SQL surgery on duplicate timeline rows

The unique index `idx_timeline_dedup` should now be present after the workaround.
Re-running `gbrain init --migrate-only` is a no-op for v9 (uses
`CREATE UNIQUE INDEX IF NOT EXISTS`). The new migration code adds the helper
btree on the dedup columns first — but the helper is dropped at the end of the
migration, so even if v9 re-ran (it won't, version is already at 10), it would
leave your schema unchanged.

### Re-running `gbrain extract` on a previously-stuck brain

This is the most common case. Run:

```bash
gbrain extract all --dir <brain-dir>
# or for live brains with no local checkout:
gbrain extract all --source db
```

Expect immediate output (`Links: created N from M pages` lines streaming as files
process), not a 10-minute hang. On a re-run of a fully-extracted brain you should
see `Done: 0 links, 0 timeline entries from N pages` — that's the truthful counter
at work, confirming nothing changed.

## New engine API (informational, optional)

For plugin authors building integrations on the `BrainEngine` interface, two
new methods are available:

- `addLinksBatch(LinkBatchInput[]) → Promise<number>`
- `addTimelineEntriesBatch(TimelineBatchInput[]) → Promise<number>`

Both return the count of rows actually inserted (excluding ON CONFLICT no-ops
and JOIN-dropped rows whose slugs don't exist). Existing per-row `addLink` /
`addTimelineEntry` are unchanged — no migration required for plugin code.

## Verification

After the migration completes:

```bash
# Confirm schema version
gbrain config get version
# Expect: 10

# Confirm the unique indexes exist (Postgres / Supabase only)
gbrain stats
# Expect: link_count and timeline_entry_count both populated, no duplicates
```

If you hit any issue, file at https://github.com/garrytan/gbrain/issues with the
output of `gbrain init --migrate-only` and `gbrain config get version`.