Formula One and Machine Learning

The power of Machine Learning, based upon artificial neural networks, has become all-too-obvious over the past decade. Early this year, it was announced in Nature that a Deep Learning algorithm, developed by Google Health, is better than human experts at identifying breast cancer in mammograms.

Naturally, there's also been much chatter in recent years about the potential use of such Artificial Intelligence (AI) in Formula One. For example, one can find Jonathan Noble's article, 'Why Artificial Intelligence could be F1's next big thing', on Autosport.com, apparently suggesting that AI could be used by both trackside engineers and those in race-support roles:

"Getting through the mountains of data generated in Formula 1 can be a 'needle in a haystack' process for teams searching for performance. There's technology on the way that could make a huge difference...The AI being talked about right now will be used at first to help better manage access to data. The computers will learn to know which data needs to be saved; which data needs to be prioritised so there can be rapid access to it. Plus it needs to be one step ahead and bring up data that is needed next."

Perhaps the idea is that if AI can spot patterns in a bunch of tits, then it could also be used by a bunch of tits to spot patterns in data. 

From inside the teams, the arrival of the Machine Learning advocates sometimes resembles a flock of seagulls swooping noisily from one landfill site to another, seeking easy pickings from the technically clueless decision-makers, squawking and chirping happily about 'convolutional neural networks', and 'GPUs running in the cloud' as they descend upon unwitting mechanical engineers and aerodynamicists.

Perhaps Formula One needs to carefully scrutinise some of the claims made by the Machine Learning (ML) community, particularly vis-a-vis its capabilities in the fields of forecasting and data-mining. A recent paper published in PLoS by Makridakis, Spiliotis, and Assimakopoulos compares the performance of ML algorithms, versus standard statistical methods, for making future predictions from time-series data. The aggregated errors were quantified using two measures, symmetric Mean Absolute Percentage Error (sMAPE), and the Mean Absolute Scaled Error (MASE). Unfortunately for the Machine Learning advocates, the statistical methods had the lowest error levels, as represented in the chart below.   

So, good news if you're an F1 engineer: you can cling onto your Excel spreadsheets for at least a little longer. Or better still, learn to use the statistical package R.

As Makridakis et al justifiably assert, "the importance of objectively evaluating the relative performance of the ML methods in forecasting is obvious but has not been achieved so far raising questions about their practical value to improve forecasting accuracy and advance the field of forecasting. Simply being new, or based on AI, is not enough to persuade users of their practical advantages over alternative methods."

Also provided in the paper by Makridakis et al is a useful table (below), which can be used as a guide to distinguish those applications where Machine Learning is demonstrably powerful (games, image and speech recognition), from those applications where it isn't (currently) the right tool for the job.

 

Machine Learning advocates can be expected to thrive in an environment lacking technically knowledgeable management. Coincidentally, there are two articles on Autosport.com extolling the virtues of Artificial Intelligence, the aforementioned 'Why Artificial Intelligence could be F1's next big thing', and 'The dangerous AI tool that could dominate F1'. In the latter, Serguei Beloussov, boss of Acronis, asserts: "In F1, there are ultimately three areas that you can apply machine learning - one is the race strategy, [the others are claimed to be logistics/operations and design]. There is some advantage, but not so much - because a race is a highly random activity, so it is relatively difficult to make a sustainable project because there is a lot of randomisation."

Now, Serguei is right about the difficulty of applying Machine Learning to race strategy, but he's completely misunderstood the principal reason. The problem is not the random element, and indeed, the random element (safety-cars and suchlike) is not the factor which dominates the logic of F1 race-strategy. 

To the disappointment of many, F1 race-strategy is a perturbation of deterministic logic: when teams devise their race strategies, they do the deterministic calculations involving tyre-compound offsets, tyre-degradation, pit-losses, fuel-consumption and so forth, and then apply perturbations to the timing of pitstops based on game-theoretic considerations of undercuts and overcuts, and the importance of hedging against (or catching) safety-car and virtual safety-car periods. There's a random element, but it's not the dominant element.

No, what makes Machine Learning so difficult (at present) to apply to F1 race strategy is the fact that F1 is a game in which the rules are constantly changing. The sporting and technical regulations are constantly changing from one year to the next, altering the rules on starting tyre-sets, how many tyre compounds are available or need to be used, how difficult overtaking is, whether refuelling is permitted etc.; moreover, the performance characteristics of the tyres change from one race to the next, and the compounds and construction change from one year to the next. It's much more difficult to train an artificial neural network when the past data is, like this, essentially a collection of similar, but different games. 

For example, you might try and estimate the overtaking difficulty at Paul Ricard based upon one year of data, without taking into account the fact that there was a headwind down the Mistral on that particular weekend, or the fact that the DRS effect was much stronger/weaker under the set of aero regulations in force that year; there might even have been a higher level of tyre degradation that year, which can have a disproportionate effect on traction, reducing the overtaking difficulty more than the pure lap-time deficit alone would indicate.

So, whilst it's difficult to see a long-term future in which all aspects of F1 activity are not influenced by artificial intelligence, in the short and medium-term, perhaps it's best to employ standard engineering practice: look at the nature of the problem, and choose the right tool for the job, rather than grabbing a sexy new tool and trying to find an application for it.