Results are never interpreted after the fact. Before deployment the reaction ranking and dynamics judgment are recorded and sealed; 14 days later the market grades them. The client receives the scorecard first.
The next round’s per-creative reaction ranking and dynamics judgment are recorded, and the file hash is sent to the client. No edits after that.
Creatives are deployed by market, platform and time slot. Every condition is logged and sealed with the forecast.
Reaction collection closes. Volume and order of views, completions, likes, comments, saves and shares are aggregated at cohort level.
The seal is opened and the forecast–observed rank agreement (correlation) and the judgment hit rate are computed, reported against the baseline.
Redeploy the same creatives under the same conditions and the reactions still shift. We measure that natural variance first and call it the baseline. Hits inside the baseline are not counted, and no model can score above it.
The baseline differs by market and platform and is re-measured every round.
Agreement is the rank correlation of per-creative reactions (Spearman, draft) shown as a percentage. Figures are hypothetical; the baseline method is agreed with the client in the pilot.
Top 8 creatives from round 3. A hit means rank error within ±2 and the same dynamics judgment. Misses are reported as they are.
| Creative | Predicted rank | Observed rank | Δ rank | Predicted dynamic | Observed dynamic | Verdict |
|---|---|---|---|---|---|---|
| #214 | 1 | 1 | 0 | TRUST | TRUST | Hit |
| #087 | 2 | 4 | +2 | HABIT | HABIT | Hit |
| #142 | 3 | 2 | -1 | EXPLORATION | EXPLORATION | Hit |
| #031 | 4 | 3 | -1 | PROPAGATION | PROPAGATION | Hit |
| #198 | 5 | 9 | +4 | TRUST | EXPLORATION | Miss |
| #066 | 6 | 5 | -1 | HABIT | HABIT | Hit |
| #250 | 7 | 6 | -1 | PROPAGATION | PROPAGATION | Hit |
| #119 | 8 | 12 | +4 | EXPLORATION | HABIT | Miss |
Metrics that look big but say nothing about dynamics are excluded from the score.
Budgets make them. Used only as order and ratio.
The cheapest reaction, and the least informative. Used only as a ratio against saves and comments.
Differences inside the baseline are treated as chance.
A hypothesis not recorded before sealing is never reported as a result.
After 14 days a judgment is flagged “refresh needed” and re-checked with a redeploy.
Forecast vs. observed · against baseline · creative ranking · next hypothesis. One PDF, with the seal hash.
Clients receive the sealed file and the observed aggregates, and can recompute everything themselves.
One D+15 report sample built on hypothetical data. Review the format and baseline math before deciding on a pilot.