AI Character Index · Use the tool

Use the tool

A tool for researchers working in the model-spec space. Point a panel of LLM judges at a spec and give the panel a behaviour you care about. You get back a passage-level coverage map: every place the spec addresses that behaviour, scored by each judge and quoted verbatim.

Get it on GitHub ↗

What people use it for

Assess coverage. How well does a spec address a behaviour you are interested in? Is it a defining commitment, a passing mention, or a gap?
Locate the exact text. Every verdict anchors to a specific passage, so you see precisely where in the spec a behaviour is addressed.
Select passages for downstream work. Every citation is a stable, re-resolvable locator plus a verbatim quote, in plain JSON -- ready to feed automated adherence evals testing whether models actually follow the passages you selected.
Experiment with spec edits. Register your own spec versions locally, run the panel against each, and compare how coverage shifts as you rewrite.

Get it

Open a terminal and copy-paste:

git clone https://github.com/AndresCotton/ai-character-index.git
cd ai-character-index
python3 -m http.server 8080 --directory site

If you'd like to run new judges, simply add an OpenRouter API key. Follow the README for more details.

Or hand it to an agent

Everything can be done by a coding agent. Open one in your clone and say “register a behaviour about X and run the panel” -- the repository ships agent instructions (AGENTS.md) carrying the steps and their gotchas, so no by-hand setup is needed.

Credits

Developed by Andrés Cotton in collaboration with Matt Stults. This work was supported by Generator Residency (Kairos & Constellation) and BlueDot Impact.