Projects

Ship the Paper

Ship the Paper: The Judge That Looked Everything Up

A calibrated LLM judge, rebuilt on a laptop. Its cap on wrong verdicts held where it was calibrated, and quietly broke on a kind of question it had never seen.

• Python
• Risk Control
• Ollama
• Unittest

Generative AI

Generative AI: Ninety Percent Sure, in English

One tiny model trained three ways on a refund desk, changing only the reward. Only a scored confidence was safe to act on, until the queue spoke more Portuguese.

• Python
• Calibration
• Unittest

Ship the Paper

Ship the Paper: Fresh Is Not Right

A replay of FinCacheServe's reuse check over amended filings. It never served a stale answer, but it did serve fresh answers to sibling questions, and its own metric called them safe.

• Python
• Answer Cache
• Unittest

Generative AI

Generative AI: The Second Email Never Asked

A miniature of the approval guard around Meta's Muse agent, run on ten scripted errands. Checking where each action goes caught little; reading every field caught nearly all.

• Python
• Approval Guard
• Unittest

Ship the Paper

Ship the Paper: Discovery Tree Replay

Replay scores other exploration policies on discovery trees you already logged. A frozen-width policy cannot see a branch it never opened.

• Python
• Replay Simulator
• Unittest

Generative AI

Generative AI: They Called It Improvement in the Morning

An AI advisor for Indian investors ranked names for tomorrow, then promoted a mix that lucked one night. Pair it against itself and a tie looks like a failure.

• Python
• Promotion Loop
• Unittest

Generative AI

Generative AI: The Quiet Rooms Kept Saying Thank You

WhatsApp voice notes from a night a phone went missing. Windowed English and German transcription, with YouTube thank-yous dropped as ambience rather than kept as clues.

• Python
• Whisper
• mlx-whisper
• Unittest

Generative AI

Generative AI: Prompt Is Not the Sandbox

The same Northwind memo, four harnesses. When the intended host was down, a no-internet prompt still fetched the lookalike customer record.

• Python
• Containment Harness
• Unittest

Generative AI

Generative AI: Memory Is Not a Brief

The same Northwind topic files, packed six ways into three agent tasks. Live-update fixed the date. The speaker email still left with a brainstorm listed as a sponsor.

• Python
• Memory Packing
• Unittest

Ship the Paper

Ship the Paper: Context Compaction Verifier

An offline test of TRACE's compaction verifier, which scores a context summary by the tool errors it causes next. It removed the failures it could hear and none of the ones it could not.

• Python
• Closed-Loop Verifier
• Pass² Metrics
• Unittest

Generative AI

Generative AI: Model Is Not the Assistant

One scripted model inside four different apps, asked the same questions about one expense sheet. Two apps tied on score, with one failing loudly and the other confidently wrong.

• Python
• Harness Ablation
• Unittest

Generative AI

Generative AI: Chat Is Not the Application

A launch brief wrapped four ways. Fluency and valid JSON both shipped the grandfather price, and the note date left with it.

• Python
• Claim Gate
• Unittest

Generative AI

Generative AI: Folder Is Not the Data

A mixed-age HR folder packed three ways. The stale FAQ won on keywords, and preferring the newest file then missed two other answers.

• Python
• Retrieval Harness
• Unittest

Generative AI

Generative AI: Leaderboard Is Not the Job

Same two jobs on three models. All three said do not switch on a two-point index gap, and all three matched on the token bill.

• Python
• Evaluation Harness
• Unittest

Ship the Paper

Ship the Paper: Compute-Aware Robustness

The cost model and metrics from Risk Under Pressure, rebuilt over a synthetic guard task to find out which of them a nightly evaluation job could actually gate a release on.

• Python
• Cost Accounting
• Resampling
• Unittest

Ship the Paper

Ship the Paper: Coding Agent Harness

An offline test of how a coding agent's work reaches the evaluator, built from Claw-SWE-Bench. Same edits, three adapters, and a 50-point spread in what the agent looks capable of.

• Python
• Unified Diff
• Ablation Harness
• Unittest

Ship the Paper

Ship the Paper: Falsifiable Commitments

An offline commitment tester for browser agents, built from FCPAgent. It writes down what would prove each step wrong, then checks for it before the browser is touched.

• Python
• Evidence Matching
• Evaluation Harness
• Unittest

Ship the Paper

Ship the Paper: Browser Agent Interface

An AgentOccam-inspired observation and action compactor for browser agents, tested across eight synthetic workflows for what a smaller interface quietly drops.

• Python
• HTML Parsing
• Benchmarking
• Unittest

Orchestrator

A modular personal assistant CLI that routes natural language requests across Slack triage, research, legal review, and meeting analysis workflows.

• Python
• CLI
• Pydantic
• Rich
• Pytest

FIRE

A multi-agent AI financial advisor for Indian investors where specialist agents collaborate on portfolio, spending, tax, and market signals to produce actionable briefings.

• Next.js
• TypeScript
• Prisma
• SQLite
• Claude Code CLI

CVAT + Segment Anything

A customized CVAT-based computer vision annotation platform with Segment Anything and auto-labeling support for faster image and video dataset workflows.

• TypeScript
• Python
• JavaScript
• SCSS
• Docker