Probabilistic EPL
Match Forecasting

Overview

This project presents a unified analysis of probabilistic forecasting for English Premier League match outcomes. Match outcomes are modeled as three mutually exclusive classes — Home Win (H), Draw (D), and Away Win (A) — and evaluated against outcome-specific climatology baselines using proper scoring rules, primarily the Ignorance Score.

Multiple validation strategies are employed to assess robustness under distribution shift. The results show that calibrated and ensemble-based ML models provide consistent improvements over climatology for Away and Draw outcomes, while Home Wins remain the most challenging to forecast reliably.

Outcome Distribution

The three outcomes are imbalanced — Home Win is the dominant class, making Away Win and Draw inherently harder to forecast. This motivates outcome-specific evaluation rather than aggregated accuracy.

Evaluation Framework

Ignorance Score (IGN) is the primary metric — it measures the information content of a forecast in bits: IGN = −log₂(p_true). Lower IGN = better. It strongly penalizes confident but wrong predictions and rewards well-calibrated probability estimates. Climatology — the empirical base rate of each outcome in training data — is used as the baseline. Any model that cannot beat climatology cannot be considered probabilistically informative.

Accuracy is included only as a descriptive metric. It ignores probability magnitudes, treats confident and uncertain predictions equally, and fails to penalize overconfident errors — so identical accuracy can hide very different probabilistic quality.

Validation Strategies

Results by Validation Strategy

Key Findings

• Away Win is hardest — Model loses to climatology in 2/3 validation strategies. Away performance is inherently variable and difficult to predict robustly across seasons.
• Home Win & Draw — Model consistently beats climatology in both interleaved strategies (Even→Odd and Odd→Even), demonstrating genuine probabilistic skill beyond historical averages.
• Temporal validation is hardest — Distribution shift between 2014–2022 training and 2023 test hurts performance. 2023 Home Win rate (46.8%) differed significantly from training (50.9%).
• Best improvement — +0.0086 bits on Away Win (Temporal, Logistic Regression) and +0.0032 bits on Home Win (Odd→Even, Isotonic LogReg).
• Accuracy is misleading — All models achieve identical accuracy to climatology (46.8–51.3%) yet differ substantially in IGN. Proper scoring rules are essential for probabilistic evaluation.
• Consistent results across interleaved splits — Even→Odd and Odd→Even both show model wins on Home Win and Draw, confirming results are not partition-dependent.

Conclusion

This project demonstrated a comprehensive framework for evaluating probabilistic football forecasts across three match outcomes using proper scoring rules. Models sometimes outperformed climatology for Home Win and Draw, indicating meaningful probabilistic skill beyond historical averages.

Away Wins proved most challenging — models lost to climatology in 2/3 strategies, suggesting away performance is inherently more variable. These findings emphasize the importance of proper scoring rules: accuracy alone completely fails to capture calibration and information content differences between models.

Further improvements likely require larger and richer feature sets (player-level data, form indices, injuries) and potentially Poisson-based goal models rather than direct outcome classifiers. Multi-task training on multiple related outcomes may also reduce the specialization-generalization tension observed here.