UOA Exact-Contract Backtests: Strong PnL Was Not Enough

Repository reference: cutebacktests

Abstract

The May 8 research pass looked exciting at first. The unusual-options-activity branch finally moved away from stock proxies and into exact option contracts, which is the right direction for a strategy that claims to trade option flow.

The best quote-validated shortlist rows were green. The hiVol_hiPrem_early_both_x1 and hiVol_hiPrem_10am_both_x1 profiles each selected 22 bar trades, closed 20 quote-priced trades, made 17310.0 total quote PnL, posted a 55% win rate, and showed daily Sharpe 4.5526. The broader midVol_hiPrem_early_both_x1 profile closed 25 quote-priced trades and made 17907.0.

That was the good news. The bad news was more important: the robustness checks and holdout evidence did not support promotion.

Question

The research question on May 8 was simple: did the new contract-exact UOA path produce a promotable options model family, or only a strong-looking local artifact?

The distinction matters. A UOA strategy that only works when contract selection is loose, quote coverage is partial, or the best period dominates the result is not ready for paper trading. The model has to survive exact contracts, quote-aware fills, PBO/DSR checks, and a separate holdout without quietly turning into a different object.

Method

The May 8 pass focused on exact contracts whose intraday option volume and premium were unusually large. The selected profiles used early-session windows, high cumulative option volume, high estimated premium, short DTE, and quote validation on the option itself.

The shortlist then went through two stricter checks:

1. A quote-validated local summary.
2. A separate holdout pass.

The bar-family PBO search also tested threshold-grid families so the result was not judged only by the best-looking profile.

Evidence

The headline local numbers were strong:

But the same summary reported base PBO 0.8810 across 5 strategies and 11 periods. That is not a small warning. It says the selection process was highly fragile.

The family-level scan looked better in places, but not enough. The best early-call threshold families showed PBO around 0.2143 to 0.2381, yet the best selected DSR was only 0.3680, with 14 trades and 11401.0 PnL. That is promising as a research lead, not a promotion case.

Then the separate holdout was blunt. One high-volume QQQ holdout found 0 bar candidates. A quote-priced calls holdout selected 0 bar trades, closed 0 quote trades, and produced 0 total quote PnL.

What Worked

What worked was the shift in measurement. This was no longer a stock-proxy UOA fantasy. The strategy was evaluated on exact option contracts, quote-priced entries and exits, and explicit coverage reasons such as missing_exit_quote and exit_fetch_error.

That is a real improvement. It forced the research object closer to what a paper bot would actually trade.

What Failed

The May 8 result failed because the strongest local PnL did not survive the promotion logic. The PBO result was too high, the selected DSR was too low in the cleaner family scan, and the separate holdout produced no executable evidence.

This is the kind of result that can fool a researcher if they stop after the PnL table. The quote-validated rows were green, but the robustness and holdout checks said the family was not ready.

On Paper Trading

This kind of result is useful because it explains issues we often see other systematic traders run into when they move from research to paper trading. A paper module should not simply replay the highest-PnL research row and hope the live path matches it.

Several issues commonly appear between research and paper:

• a signal can be valid, but the selected contract may not have a fresh executable quote at the decision time;
• a holdout can go empty even when the local sample looked strong;
• missing exit quotes can make a result look cleaner than the live process would be;
• PBO and DSR can reject a strategy that still has attractive headline PnL.

Those are the kinds of mechanics we want to explain deeply and build directly into the integrated paper trading module we are working on: timestamped signal records, quote freshness gates, holdout-aware promotion, explicit reject reasons, and paper/backtest parity checks before a strategy is allowed to look successful.

Takeaway

May 8 was a useful false start. It showed that contract-exact unusual-options research could produce real-looking option PnL, but it also showed why exact fills are only the first gate.

The next step was not to paper this exact branch. The next step was to ask which mechanics could keep exact-contract cleanliness while reducing selection fragility and holdout collapse. That question shaped the following week's broader model-family search.