alert backtest — fires, forward moves and false-signal rate, no look-ahead

Alert backtest: what price did after each fire

Take a price or indicator recipe — "RSI crosses below 35", "price crosses up its 20-day average" — and AtlasVector replays it bar by bar over history. A "fire" is counted once when the condition flips false→true, the same de-duplication a live alert applies; the forward move is then measured over the next 1 / 5 / 21 bars: hit rate, average forward move, and how often the fire reverses (the false-signal rate). No look-ahead. The replay runs on real price history only.

AAPL⚑ price crosses down its 20-day average

⚑ The recipe: price crosses down its 20-day average — the price breaks below its 20-day moving average — a momentum-rolling-over / breakdown trigger. expects price to fall

BACKWARDS · triggered 9× over 260 bars

5-bar hit rate 38%avg forward move +1.48%median +1.98%edge vs baseline +0.55%false-signal rate 88%

Fires9

Hit rate38%

Avg move (5b)+1.48%

False signals88%

What happened after each fire, by horizon

HorizonResolved firesHit rateAvg moveMedianFalse signal

1-bar838%-0.20%+0.19%63%

5-bar · headline838%+1.48%+1.98%88%

21-bar850%+0.96%-0.88%100%

A fire at bar t is only scored at horizon h if bar t+h exists — outcomes that cannot yet be observed are dropped (censored), never guessed. A hit rate near 50% is indistinguishable from chance; meaningfully above it, in the direction the recipe expects, is the start of a measurable edge.

This bearish alert fired 9×; over 5 bars it went your way 38% of the time (avg +1.48%), head-faking 88% of the time.

How often does it fire?

Your alert fired 9 times over 260 bars.

We replay your exact condition bar by bar. A "fire" is counted once at the moment the condition flips from false to true — the same way a live alert de-dupes so a condition that stays true for ten bars pings you once, not ten times.

→ A workable sample size.

Did the move go your way? (5-bar hit rate)

Over the next 5 bars after each fire, price moved in the expected (down) direction 38% of the time, averaging +1.48% (median +1.98%).

Hit rate is the share of fires where the forward return's SIGN matched what the trigger implies — a "bearish" alert wants price to fall. A coin flip is ~50%; meaningfully above (for the direction you want) is the start of a real edge. Mean can be skewed by one big move, so we show the median too.

→ Below chance for your direction — the trigger may be backwards.

How often does it head-fake? (false-signal rate)

88% of fires went against you at some point within 5 bars before the horizon was up.

False-signal rate measures the "it triggered, then immediately reversed" experience — the share of fires where, at any bar inside the horizon, price moved adversely to your intent. A high number means lots of whipsaw even when the endpoint eventually works.

→ Whippy — expect to sweat through drawdowns.

Glossary

Fire: One moment the alert condition flips from false to true; counted once per flip, like a de-duped live alert.
Hit rate: Share of fires whose forward-return sign matched the alert's directional intent. ~50% is a coin flip.
Forward return: close[t+h] / close[t] − 1 — the percentage price change over the next h bars after a fire.
False-signal rate: Share of fires that moved adversely to your intent at any bar within the horizon (whipsaw).
Horizon: How many bars forward we measure the outcome (here 1, 5, and 21 bars).