The headline result of this project is negative, and the system is the one that reported it: every prop signal type was pre-registered with a threshold, graded against settlement outcomes, and retired by its own scorecard. No edge survived honest measurement. Building the machine that could reach that verdict — and trusting its answer over my own enthusiasm — taught me more about data-integrity discipline than anything else I’ve shipped. The measurement spans two codebases — the collection layer described here and the downstream validator that inherited its grading semantics — and this page tells that story end to end.

Underneath the verdict is the machine: a multi-book consensus-outlier system for MLB betting markets. It watches around nine sportsbooks in parallel — major US-regulated books, a few prediction markets, and a couple of prop-deep sources — and surfaces moments when one of them is sitting meaningfully off where the rest of the market has converged.

The thesis

Sportsbooks are sharp. Individually, betting against any one of them is a hard way to make money, because each one prices its book using sharper inputs than I have. But “sharp” doesn’t mean “right” — it means “calibrated to its own customer base.” Books shade their numbers based on who they need to balance, who they want to attract, which way the public is leaning, and what their risk department is worried about. Different books are sharp in different directions.

If you watch one book, you see one book’s number. If you watch enough books at once, you see the market — a soft consensus that emerges from the joint pricing of every participant. The signal isn’t whether any single number is right. It’s whether one book has landed somewhere the rest of the market clearly hasn’t. That gap is the edge. It tends to close fast once the outlier’s risk desk notices what its peers are doing, but it does open, and it opens often. Whether anything survives once you grade it honestly is a separate question — and answering that question is what this system turned out to actually be for.

That observation is the whole project in one paragraph. The rest of the work is the infrastructure that makes it operable.

The shape of the system

Conceptually there are three layers:

  1. Collectors. One per data source. Some collectors are Playwright-driven scrapers running through a VPN container on the Linux box — behind an egress check that refuses to launch a browser whose traffic would leave on the house IP. Some are REST polls against authenticated APIs. Some are WebSocket streams from prediction markets. Each collector is isolated in its own module with its own schema; the API-driven collectors share a common circuit-breaker and retry utility. The test weight deliberately sits downstream of the collectors, where the risk is — the event matcher, the grader, and the normalization rules — because a collector that breaks fails loudly, while a bad join or a wrong grade fails silently. Collectors write to a normalized event store keyed on a canonical event identity.
  2. Normalization. A small set of services that joins raw rows from different sources onto a shared canonical event — same game, same market, same line — so that downstream queries can compare prices apples-to-apples. Team-name normalization alone is a hundred-plus-rule layer in its own right; sportsbooks do not agree on whether the team in Los Angeles is the Rams, the LA Rams, or Los Angeles. Time-window matching, line-bucketing, and home/away alignment all live here.
  3. Surfaces. A consolidated lines board over the canonical event store — each market with its best price, its consensus context, and an outlier badge when one book sits meaningfully off the field — built as a React/Vite dashboard talking to a Postgres-backed API.

The thing the architecture diagram does not show is that every layer above the collectors makes assumptions about the data in the layers below. And those assumptions are exactly where everything fell apart.

The audit story

There was a point, well into the project’s life, where I tried to re-grade the historical signal corpus against an updated version of the grading code. The result was zero rows passing the EV floor under verified code. Not “fewer than before.” Zero.

That was not a fixable bug. That was the project telling me the integrity work had been deferred too long and the corpus was no longer trustable. So I stopped feature work and ran three sequential audits.

Audit one — event matching. When two collectors emit a row for “the same game,” are they actually referring to the same game? Or have we joined the prediction-market contract for one team’s series to the regular sportsbook’s single-game line, or matched two games with different home/away orderings, or matched a postponed game to its replay? The audit added test coverage against the matcher’s phases — exact match, fuzzy match, variation match — pinned the team-name normalization rules with full coverage, and added a canonical-events table that became the single point of truth for “same game.” Secondary collectors were demoted to match-only mode; only the primary source could create new canonical events. The downstream effect: every join past this layer is provably operating on the same game, not a coincidentally-similar one.

Audit two — grading. This was the deepest one. The old grader was a fifteen-way if/else if chain of substring matches against uncontrolled stat-label strings from external books. Correctness depended on branch order. Pitcher-specific branches had to precede batter-generic ones. New stat types had to be inserted at exactly the right position or they would silently land in the wrong branch and grade against the wrong column. Pitcher “Runs Allowed” props were quietly grading against the batter’s runs scored. ProphetX’s underscore-form labels (player_total_outs_recorded) didn’t match the grader’s space-form substring checks (outs recorded), and the signals fell through to a last-resort branch.

The replacement is a DB-backed crosswalk grader. A (bookmaker, normalized_stat_label) → canonical_stat table, cached in memory, validated against the canonical-stat enumeration at startup. The grader takes a signal, normalizes the label, looks up the crosswalk, dispatches to a pure decision function keyed on the canonical stat, returns a result. Unknown (bookmaker, label) pairs go to an unmapped-signals inbox for human review; they do not silently land in a default branch. Bookmaker is a first-class input to dispatch, which closes the entire class of “same label means different stats on different books” bugs that the substring chain couldn’t see.

Before merging the refactor, I ran both the old grader and the new one against two hundred randomly-sampled historical signals and compared outputs. The divergence rate was one in two hundred, and the single divergence was the bug the refactor was meant to fix. The comparison harness stayed in the repo, and re-running it is part of my discipline before a grader change lands — it exits nonzero on any divergence.

Audit three — timezone bucketing. Less dramatic but the kind of quiet bug that’s expensive when it happens. Some collectors emit timestamps in UTC, some in eastern time, some in book-local time. A game whose first pitch is 7:05 PM ET might land in two different date buckets depending on which timestamp the bucketing logic trusted. The audit pinned every timestamp to TIMESTAMPTZ UTC at ingestion, pushed timezone-aware rendering all the way to the UI, and added tests that caught the off-by-one-day failures the prior code would have shipped silently.

The three audits took roughly two weeks. Re-grading the corpus under the post-audit code restored a nonzero EV signal set, and — more importantly — the corpus is now one I can change the grader and re-grade against, repeatedly, without worrying that the next change will silently break a class of historical rows.

The realEdge metric

The metric the system optimizes for is realEdge:

realEdge = trueProb × decimal(candidate_price) − 1

where trueProb = impliedProb(second_best_price_across_market).

Two pieces are worth explaining.

The first is decimal(candidate_price) − 1, which is just the payoff multiplier of the candidate book’s price, expressed in decimal form. Standard.

The second is trueProb = impliedProb(second_best_price). The “best price” on a market is by definition the candidate I’m considering; using it as trueProb would be circular. The “second-best price” is the runner-up across all other books. That number is more robust to coordinated mispricing than the median, because the median can be dragged by a stale book or by a clump of books with correlated risk inputs. Second-best is dragged only when the runner-up itself is the outlier — a much rarer condition. It’s a deliberately conservative anchor.

realEdge is the operational version of the market-structure observation this page opens with — and it is exactly the number whose survival the grading machinery was built to judge.

The verdict

Detection was never the hard part. The hard part was refusing to believe detection. Two grading systems sit on top of the signal corpus, one in each codebase.

In the collector, signal capture is wired into the live scheduler: every fired signal logs the price seen at that moment, and CLV grading later compares it against the closing line. Two honest caveats bound what CLV can prove here: the closing price comes from a single book, not a market-wide close, and the ml_outlier signal type structurally can’t join to closing observations at all — it grades NULL. CLV is a screen, not a verdict.

The verdict came from settlement, because box scores don’t argue. In the downstream validator, every signal type declares its threshold before data collection, graded rows are immutable, and per-type scorecards decide: win rate with a Wilson interval, expectancy with a confidence interval, PASS only when the entire interval clears zero. When the settlement grades came back, the scorecards retired the prop signal types — negative realized ROI across the board. No prop edge survived. The code itself frames settlement as the verdict CLV can’t give on thin props.

The system had already acted on its own answer once before that. The projection-based prop “EV” feature — an entire endpoint — was deleted after CLV measured it as noise (beat-close: 3.9%), replaced by real market-vs-market comparisons only. A feature removed because the measurement said it wasn’t real is this whole page in one commit.

What it looks like today

The current state is roughly nine books being watched, normalized into a single canonical event store, with dedicated frontend views for the three things I care about — outliers, arbitrage, and line shopping. The grader is the DB-backed crosswalk version; the corpus under it is internally consistent; the test suite covers the fifteen-branch coverage the substring chain used to need plus regression guards for every bug the audit surfaced.

There’s a frontend design system underneath it that matches the rest of my personal projects — JetBrains Mono, near-black background, green/gold/red accent system, grid-row tables, dense by intent. It is not a marketing site. It is an operational service: today it runs as the collection and settlement-archive layer feeding a downstream decision system, which is a promotion, not a demotion — the data layer survived its own audit well enough that a second system now builds on it.

What I learned, mostly

The data integrity discipline is the project. The modeling work is five percent of the value; the audit infrastructure is the rest. The moment I treated the signal corpus as a database that could be re-graded under any current version of the code — instead of a historical artifact frozen under whatever code was running the night it was written — the project became operable in a way it wasn’t before.

The other lesson is the unmapped-signals inbox pattern. Any time you dispatch on uncontrolled external strings — book stat labels, tournament names, market types, anything you don’t control the schema for — the right default is to surface the unknown to a human-review queue, not to fall through to a best-guess branch. The silent-mis-grade class of bug is the worst kind, because the system keeps producing plausible-looking output forever. The unmapped-inbox pattern makes the failure noisy on first contact, which is exactly where you want noise.

The clearest sign the discipline held: when I built the downstream system’s validator, its grading semantics were lifted from the settlement code here — the docstring says so. Log the price you saw, let ground truth grade it later, void over guessing. That convention was earned in this codebase, one audit at a time, and it outlived the codebase’s original job. The foundation is the part that mattered, and the foundation is the part the audit story is about.