How to Backtest a Kalshi Strategy (Without Fooling Yourself)

A backtest's job is not to make you feel confident. It's to try to kill your idea in a spreadsheet before the market kills it with your money. Most backtests do the opposite — they're built, consciously or not, to confirm a strategy the author already believes in, and they reliably produce a beautiful equity curve that evaporates on contact with live trading. This guide is about doing it the honest way: getting real Kalshi data, putting the real fee in the math, and recognizing the five specific ways a backtest will lie to you.

This is the validation step that belongs between a strategy idea and a live bot. Pair it with Kalshi bot risk management for what to do once an edge survives testing, and the complete guide to Kalshi trading bots for the bigger picture.

What a backtest can and can't tell you

A good backtest can tell you that a rule would have been profitable on a specific historical sample, net of realistic costs. That's genuinely useful — it's how you reject most ideas cheaply. What it cannot tell you is that the rule will work going forward. The future is out-of-sample by definition, edges decay as others find them, and a backtest is always vulnerable to having been fit to noise. Hold both truths at once: backtest to reject bad ideas confidently, and to promote survivors cautiously.

The data problem on Kalshi

Backtesting equities is easy because clean historical price data is everywhere. Prediction markets are harder, for structural reasons you have to respect:

• History is thin and young. Many Kalshi markets are recent, and each market resolves exactly once. You don't get years of ticks on one instrument — you get one outcome per market, so your sample size is the number of markets, not the number of days.
• Resolved markets can vanish from convenient views. If you want to test against history, you have to record it as it happens. The most reliable approach is to snapshot the order book and prices yourself, point-in-time, going forward — or pull what you can from the API's historical and settlement endpoints and accept the gaps.
• You need the book, not just the last price. A strategy that assumes it can buy 200 contracts at the last printed price is fiction if the book only had 18 there. Realistic backtests model the depth that actually existed.

The practical upshot: start logging the markets you care about now. A month of self-recorded, point-in-time data beats a year of data you can't trust the timing of.

The backtest loop

Every honest backtest has the same skeleton. The non-negotiable rule is point-in-time discipline: at each decision moment, your strategy may only see data that existed at that moment.

for each historical decision_point t:
    context = data_available_strictly_before(t)   # no peeking past t
    signal  = strategy(context)
    if signal.act:
        fill_price = realistic_fill(book_at(t), signal)  # model the spread/depth
        fee        = kalshi_fee(signal.contracts, fill_price)
        record_position(t, signal, fill_price, fee)

# Resolve every position at its market's known outcome, then:
total_pnl = sum(payout - cost - entry_fee - exit_fee for each position)

If your code can "see" the resolution when it makes the entry decision, your backtest is science fiction. The single most common backtest bug is some subtle form of looking into the future.

Put the real fee in the math

This is where most public Kalshi backtests quietly cheat: they test gross, not net. Kalshi's trading fee is a dome that peaks at 50¢:

fee = round_up_to_next_cent( 0.07 × contracts × price × (1 − price) )

Charged on entry, and again on exit if you trade out rather than holding to settlement. On coin-flip-priced contracts a round trip is roughly 3.5¢ on a contract that can only move 100¢ — so a "3% edge" strategy that churns near 50¢ is a net loser, full stop. A backtest that omits fees doesn't have a small error; it tests a different, fictional strategy. The full mechanics are in Kalshi fees explained, and the way the dome erases supposed free money is in the arbitrage guide.

The five ways a backtest lies

1. Lookahead bias. Using information that wasn't available at decision time — a resolved outcome, a revised data print, a price from one second in the future. The cardinal sin, and the easiest to commit accidentally.
2. Survivorship. Testing only on markets that still exist or that you happened to remember. The markets that quietly resolved against the thesis are exactly the ones missing from a careless dataset.
3. Overfitting. Tuning thresholds until the curve looks perfect on the sample. If your rule has five knobs and you turned all of them to fit the past, you've memorized noise. Reserve out-of-sample data the optimizer never sees, and prefer fewer parameters.
4. Ignoring fees and slippage. See above. Gross backtests are fantasies; model the fee dome and assume you don't always get the last price.
5. Tiny sample. Ten resolved markets is an anecdote, not evidence. Because each market resolves once, you need many of them before a positive result is distinguishable from luck. Be suspicious of any conclusion drawn from a handful of trades.

Sizing only makes sense once the edge survives all five — and even then you size the real edge, not the curve-fit one. That's the link to Kelly position sizing: precise sizing of a fake edge is precise nonsense.

Metrics that matter — and ones that mislead

A single profit number hides more than it shows. The ones worth looking at:

• Expectancy per trade — average net P&L per trade, after fees. The honest core metric.
• Sample size — how many resolved markets the result rests on. Ten is an anecdote; hundreds start to mean something.
• Maximum drawdown — the worst peak-to-trough dip. A strategy you can't psychologically sit through is one you'll abandon at the bottom.
• Win rate in context — 40% winners can be highly profitable and 70% can lose money. Win rate is meaningless without the average win versus the average loss.

Be most suspicious of the metric that looks best. A spectacular return on a tiny sample is usually luck or a lookahead bug, not a discovery.

Walk-forward: the closest thing to honesty

The strongest defense against overfitting is out-of-sample testing done seriously. Split your data by time: tune the strategy on an earlier period, then test it — completely untouched — on a later period it never informed. Better still, walk that window forward repeatedly across your history. If the edge holds on data the optimizer never saw, you may have something real. If it only shines on the exact period you tuned, you fit noise and the live market will tell you so, expensively.

A concrete example

Suppose you believe favorites in a certain market type are systematically underpriced. You record point-in-time prices for 200 such markets, apply your entry rule using only pre-resolution data, net out the real fee on each, and resolve each position at its known outcome. The result: a small positive expectancy of, say, 1.5¢ per contract over 200 markets. Now interrogate it. Is that bigger than the noise you'd expect from 200 essentially random draws? Does it survive on the second 100 markets after being tuned on the first? Only if it clears those bars does it earn paper trading. That is the entire discipline — produce a number, then make every honest attempt to disprove it before you trust it with capital.

From backtest to paper to live

A surviving backtest earns a strategy the right to the next test, not a live deployment. The honest sequence is:

1. Backtest to reject bad ideas and net out costs.
2. Paper forward test to inspect checks, blocks, signals, source behavior, and modeled orders, fees, positions, and P&L against live quotes in a separate virtual account. Bot Builder Paper mode never submits a Kalshi order, and it cannot reproduce queue position, latency, partial fills, rejection, slippage, or live performance.
3. Live, small. Start with sizes where being wrong is tuition, not a catastrophe, and scale only if live results track the backtest.

For a concrete example of why closed and open observations must stay separate, read our 369-entry paper-test postmortem.

Backtesting is for rejecting, not believing

End where you started: the goal of a backtest is not to fall in love with an equity curve. It's to cheaply eliminate the many ideas that don't work so you spend real capital only on the few that survive honest scrutiny. Treat a great backtest as a hypothesis that has not yet been disproven — not as a promise. The market is the only out-of-sample test that ultimately counts, and it charges tuition. A disciplined backtest, with the real fee in the math and the future kept firmly out of the past, simply makes that tuition smaller.