Sports Prediction and Betting Models
New Avenues Unlocked with the Bittensor Network
The intersection between sports predictions and machine learning has enabled a new era where data-driven analysis may outperform intuition-based approaches, however this comes at the cost of complexity. Advancements in machine learning tools and libraries have allowed for the possibility of analyzing vast amounts of sports data and predicting outcomes with surprising accuracy. This article delves into the trials and tribulations of developing predictive sports models and explores how the Bittensor ecosystem can allow for testing and data generation at scale.
The Rise of Sports Prediction Models
Sports prediction and statistical betting has been a traditionally closed off field of research, with model architecture insights and data processing techniques hard to find. Statistical models, involving predefined assumptions and static equations have been used to assist in player recruitment and to predict game outcomes, among other applications. Machine learning models such as random forest, linear regression, and deep learning have provided a new tool to capture a more generalized understanding of sports data that statistical models cannot achieve. Recent machine learning architecture developments such as long short-term memory and transformer-based models provide new tools for predicting game outcomes and managing betting strategies, which have proven effective in capturing time dependency and long-term interactions between data.
Types of Machine Learning Models
The wide variety of possible model architectures provides many options to choose from. Simple models such as low layer count regression and random forest have been known to perform well, while more complex deep learning models may provide better performance by their ability to cover larger parameter spaces and understand more complex temporal and spatial dependencies in sports data.
Regression models: Predict continuous outcomes such as number of goals or fouls in a game
Classification models: Predict discrete outcomes such as win, draw, or loss
Neural Networks: Architecture based on neurons that can recognize complex patterns
Ensemble models: Combine several focused machine learning models to improve data understanding and prediction accuracy
Transformer models: Attention based architecture proposed in Attention is All you Need by Vaswani et al.
Building models for Bettensor
Developing sports prediction models starts with sourcing and exploring datasets. High quality data is at the core of any accurate model. A common saying is “garbage in, garbage out”. Datasets for sports prediction have to be comprehensive and provide enough individual events to allow for extensive training, validation and testing. The sports model Podos began with a large dataset of 100,000 games covering numerous teams from around the world. The data also includes many game statistics such as fouls, shots on goal, passing percentage, and others. Another important parameter, especially for betting models, is bookmaker odds. Training on these odds can provide the model useful information about when and how to bet. Model architecture selection is highly flexible and dependent on data structure.
Preprocessing
Data preprocessing for the soccer, and upcoming NFL model start with sourcing many games, odds per game, and match statistics. Statistical analysis can be applied to find strong predictors of match outcome, this is an invaluable tool to guide feature selection for training and inform potential gaps in the dataset. Feature engineering, namely forming useful features using existing data can give the model an extra edge in understanding and predicting game results. Parameters such as difference in each team's passing percentage, averaged score, possession time, and defensive metrics are all valuable. Including individual player statistics, and simulating matches based on player performance is an exciting future opportunity, but not currently implemented for the Soccer and NFL models.
Model architecture
The model architecture is flexible, and dependent on the type of data being used. While there are right and wrong decisions in designing a model for sports applications, hyperparameter tuning is somewhat of an artform rather than a clear-cut decision process. For time series data, such as per-game statistics, LSTM, Transformer, or Recurrent Neural Networks are good options to try. The soccer model went through several iterations of recurrent networks, but a transformer-based model performed the best.
The NFL model takes an ensemble approach, with detailed discussion later in the article.
Training and testing
Training machine learning models involves tuning parameters and weights such that the parameters minimize the difference in predicted outcome and actual outcome. Traditional training loops work well for game prediction models, with tools such as gradient descent and variants like adaptive moment estimation providing good performance and fast convergence. The Adam optimizer and criterion loss were used for training the game outcome models with adaptive learning rates and early stopping to prevent overfitting was employed. Validation and testing followed to ensure the models generalized well and did not overfit onto the training data. Predicting game outcomes is hard, and most models achieve lower than 80% accuracy over the long term.
NFL model
The upcoming NFL model uses a hybrid approach and is designed to find valuable bets rather than solely maximize prediction accuracy.
A smaller random forest model was trained on every NFL game over several years to predict game outcome
A four-layer feed-forward PyTorch model trained to predict a fractional kelly criterion
The model outputs are combined to recommend wagers based on model confidence and edge against book odds
Model performance
Random-forest model - 62% prediction accuracy over the entire 2023 NFL season
Pytorch feed forward model - Between -20% to 7% return on initial investment
Both models take various historical team statistics over a sliding window as inputs, return predicted game outcome and kelly fraction. The Kelly criterion outputted by the model determines the optimal percentage of the bettors total bankroll to allocate towards the bet. The philosophy behind this ensemble approach is that task-specific models may perform better than one model trying to fit on both game prediction and betting strategy.
Sports prediction is Hard
Sports betting is a uniquely difficult task, and forming models that outperform experienced betters and industry entities is hard.
In the plot above, random betting, Martingale, Kelly criterion betting, and the ensemble model bankrolls are plotted over the entire 2023 NFL season. Each simulated bettor had a starting bankroll of $1000 and stopped betting either when no more games were available or if they went bankrupt. The model performed better than random betting, was more stable than a statistical kelly approach but was not as consistent in breaking even as the Martingale bettor. The model, and accompanying strategy function decides to bet only on games where it has an edge, or will not lose a lot. This resulted in fewer games being bet on than the random or martingale strategies, however it still performed well, considering an average expected loss of 5% on each bet due to Bookmaker’s margin.
Towards the Future
We aim to provide baseline models that can run in parallel by different miners with different parameter settings to test and iterate on model performance faster than one individual may be able to do. The Bettensor subnetwork provides an environment where models can be run en-masse and generate useful data on model performance tested against human predictions. Our platform can guide human bettors to improve their performance, and for machine learning developers to test their sports prediction models against humans. Open Sourcing the models is an important philosophy, and we aim to provide users the tools they can choose to integrate into their own betting toolset.
You can find our models on our HuggingFace, and our subnet repository on our GitHub.