Field guide / primary domain
28
sources in this field
Updated July 16, 2026
Current thesis
The shortest path to orientation.
Autoresearch—the Karpathy-originated hill-climbing loop of small changes, binary testing, and iterative refinement—has matured from ML optimization into a generalizable pattern for prompt tuning, GPU experiments, web automation, and full research workflows. Claude-based agents now autonomously walk citation graphs, pull datasets, reformat data, launch training jobs, and retrain on failure (ml-intern beat Claude Code on GPQA 32% vs 22.99%); Feynman generates cited meta-analyses in 30 minutes and audits claims against code. The loop scales to planning (Sean Geng's plan-optimizer using Claude Fable 5 to break scoring ceilings) and adversarial research (grounding development in literature before building). Operationalized variants include the Evo plugin (open-source for Claude Code, auto-discovers metrics and runs tree search) and monthly wiki health checks that flag contradictions and unsourced claims. Power users are investing 1,200+ hours into Claude-based research workflows, indicating Claude is becoming a primary knowledge-work tool. At ~100 articles and ~400K words, LLMs handle complex Q&A against personal wikis via auto-maintained index files; filing results back creates compounding knowledge bases where every interaction enhances future queries rather than disappearing into chat history. The long-term trajectory involves synthetic data generation plus finetuning so LLMs "know" the data in weights rather than relying on context windows.
Evidence board
Claims worth carrying forward01Install the plan-optimizer skill with one command: `curl -fsSL https://seangeng.com/plan-optimizer-skill.zip -o /tmp/plan-optimizer-skill.zip && unzip -o /tmp/plan-optimizer-skill.zip -d ~/.claude/skills/ && rm /tmp/plan-optimizer-skill.zip`. Restart Claude Code; it then triggers automatically on prompts like 'improve this plan' or 'make this bulletproof'.
02The rubric is the loop's ceiling: a vague rubric produces a low, noisy score ceiling regardless of how many iterations run. Seven default criteria (goal clarity, completeness, sequencing, feasibility, risks, success metrics, specificity) should be domain-adapted and weighted before the first iteration begins.
03Scoring and rewriting must be separate deliberate acts. Merging them produces lateral drift; the explicit critique step (list weaknesses ordered by score cost, concrete not vague) converts blind rewriting into directed search that climbs toward a target.
04Plateau detection uses a sliding window of K=3 rounds: stop when best score hasn't improved by a meaningful margin (≥+2/100). Hard cap at 6-8 rounds regardless. Before declaring convergence on hill-climb, attempt one best-of-N=3 round with structurally different plan framings to escape local optima.
05Running the improve and score steps on a stronger model is described as 'the single biggest lever' in the loop. A stronger model proposes structurally better rewrites AND perceives rubric-relevant flaws a weaker scorer misses, raising the score ceiling itself—not just the plan quality.
06Three named pitfalls: (1) reward hacking—scores climb but plans don't improve, fix by re-anchoring to original rubric; (2) weak rubric capping scores—plateau at a low ceiling signals the rubric can't see the dimension that matters; (3) reformatting masquerading as improvement—substance changes (missing risk, fixed dependency) must be distinguished from structural prettification.
07Output format specifies four deliverables: (1) best plan clean and ready to use, (2) final score with per-criterion breakdown, (3) score trajectory e.g. '62→74→81→83→83', (4) 2-3 biggest substantive changes from first draft to final. The trajectory is evidence the plan was hardened, not reformatted.
08Full personal-agent stack: Hermes on Hetzner/DO/Hostinger VPS + GBrain (or QMD+SQL) memory vault + Codex auth routed to GPT-5.5 fast/no-reasoning mode + Orca IDE on macOS and iOS via Tailscale for cross-device access.
Adjacent fields
Latest evidence
Sean Geng
Sean Geng packaged @goodalexander's iterative plan-improvement technique as an installable Claude Code skill. The skill treats planning as a search problem: generate a plan, score it 0-100 against a rubric, critique specific weaknesses, rewrite, keep the best, and stop when scores plateau over a sliding window. One curl command installs it into ~/.claude/skills/plan-optimizer/.
Jonata Santos
A concrete personal-agent stack: Hermes on a VPS (Hetzner/DO/Hostinger), GBrain or QMD+SQL memory vault, GPT-5.5 fast mode via Codex auth, Orca IDE on macOS/iOS via Tailscale. The workflow gates every project with deep research, `/grill-me` unknowns surfacing, written evals, and iterative runs until all evals pass — saving learnings to the vault throughout.
Brace
> @matt_feroz Ya by Google! It’s overall pretty simple which is why I’m inclined to adopt early. Would recommend checking out, only like a 10 min read > > https://t.co/Vt8AjTAbjw
Nabeel Hyatt
> Asking Fable to "read the relevant academic literature on this idea before building - then think adversarially" is my favorite thing in the last 48 hours. - Fable can be prompted to "read the relevant academic literature on this idea before building" to ground development in existing research — re
Jonata Santos
> how to build anything rn: > > - get a hetzner, do, or hostinger vps
Alok Bishoyi
> @NicolasZu hey nicholas - had opensourced a autoresearch orchestrator recently that you may find interesting. do give it a look / try and let me know ! > > https://t.co/NfVpYk7Sej
Jeff Grimes
> The best hedge fund PMs compare views from different research analysts. Now you can do that in 2 minutes with Equity Research Council in Perplexity Computer. > > This workflow pulls in research from brokers like GS, JPM, MS, and Evercore - and shows you where they agree vs. disagree.
TBPN
> OpenAI member of product staff @embirico describes the evolution of "Lord Bottleneck," an internal Codex loop developed by a single staff member that ultimately ended up creating a tight feedback and improvement loop for new user experiences: > > "This person on the growth team needed to figure ou
Graeme
> There’s one Hermes use case for everyone, and if you're not using it, you're already behind. > > Do yourself a favour and build a research agent as I outline below; it will change the way you work.
Ellaa
> Self-improving Claude Code skills are f*cking ridiculous 🤯 > > One loop → 10 test runs, scored against an eval, prompt rewritten, retested, winner kept.