APR Prediction Competition 2024 – Double the Challenge, Double the Excitement!

Introduction

It’s everyone’s favourite time of the year. The sun is out, exams have finished, and the first set of results for APR’s conference predictions competition are released! And luckily for you all, we have a bumper edition this year…

Because of how much fun people had predicting at the inaugural APR Predictions Competition at Life Conference in 2023, we started to feel bad those at GIRO couldn’t get involved. So to make sure they weren’t feeling left out, in 2024 we carried out two simultaneous tournaments – one for each conference!

The rules for the tournaments remained the same as in 2023. Our test subjects contestants were given five metrics that they had to predict at two different time points – 31^st March and 30^th June 2025. To help them, we provided information about the values of the metrics over the past 4 years. The metrics were chosen to cover a range of statistics relevant to actuarial work, with some returning favourites from last year while some were brand new. We also decided to use 5 completely new metrics for our GIRO competition, to make them more relevant to the GI audience.

Metrics and Scoring

For the Life Conference, the first three metrics remained the same as in 2023 – the BoE base rate, the GBP-EUR spot exchange rate, and the 10-year UK gilt yields. These were joined by two new ones; the weekly price of a litre of unleaded petrol and the number of deaths recorded in England and Wales in a 13-week period (Note that this was the number of deaths recorded, not that actually happened – this will be important later).

For GIRO, we started from scratch with 5 brand new metrics. These were the number of electric vehicles registered in the UK in the last month, the Consumer Price Index (CPI), the monthly rainfall in mm, the GBP-USD spot exchange rate, and the UK’s quarter-on-quarter GDP growth.

For consistency, we used the same formula as last year to find participants scores. To find participant n’s score for a given metric, we used the absolute difference between their prediction and the true value and divided it by the average absolute difference between the predictions and the true value across all entrants. To get a participants final score, we added their scores across the 5 metrics together. That is,

\(score = \sum_{i=1}^5\frac{|metric_i – prediction_{n,i}|}{\sum_{k=1}^N|metric_i – prediction_{k,i}|}\)

where metric_i was the true value for metric i and prediction_n,i was participant n’s prediction for metric i. This means a lower score is better. By construction, the average guesser would expect to get a score of 5, whilst a high score is roughly anything three or lower. Our March winner last year scored an impressive 1.427!

The Results

So, what were the results? Drumroll please…

Despite some big changes between October and March, particularly for the rainfall and the number of electric cars registered metrics, there were some good guesses by our participants! By far the most accurate prediction was the BoE Base Rate, with 46% of entrants correctly putting 4.5%. Other well answered questions included the exchange rate questions, the weekly petrol price, and the number of deaths recorded in England and Wales.

It initially appears our Life contestants were much better predictors then their GIRO counterparts, with an average distance from the true value of 7.71% compared to 60.02%. This isn’t the fairest comparison, however, as our GI metrics had a lot more variability (we’re looking at you, monthly rainfall). On the one comparable metric, the exchange rates, the performances were actually very similar and both pretty good – 3.26% for our GI contestants compared to 2.40% at Life.

The Winners

Our March winner for Life Conference was Kathryn Rolfe with a notable 2.100, which outperformed second and third place scores of 2.618 and 3.016. For GIRO, our winner was Natasha Regan with a score of 3.388, followed by second and third place scores of 3.769 and 3.879.

Interesting Trends

We have picked out four particularly interesting metrics where the trend appears to change since October, and provided their plots. In blue, we can see a part of the trends our participants were provided with, whilst in red we can see how it has changed since.

CPI: CPI was showing a decreasing trend in the 8 months before September/October, but that turned out to be the lowest it would reach. Since then, we have seen CPI rebound to the levels seen earlier in 2024, reaching 2.6% by the end of March. Overall, guesses seemed to predict that inflation would rebound, but not by nearly as much as it has done. The mean guess was 1.86%, with an average distance away compared to the true value of 25%.

Rainfall: This was by far hardest to predict, with an average distance away of 198% – and who can blame them? At this point last year, rainfall was 107.8mm, and two years ago it was 134.4m. Guesses therefore sensibly averaged around 115mm – over three times the actual value of 37mm! The best guess we received for this was 80mm, which was still over double the actual value. I will leave the explanation behind this fall to the meteorologists, but I’m sure it isn’t good…

GBP-USD: The GBP-USD exchange rate must have caused a lot of anxiety for our guessers, who mostly predicted it would remain around 1.29, as it continued its fall to a trough of 1.24 in late January. Luckily for them, since then it has rebounded to almost exactly the same level it was at in September. This metric also demonstrated the ‘wisdom of crowds’ we discussed , as the mean of our entrants guesses was in fact closer to the true value than any of the actual guesses, being less than 0.001 away (though as our value was almost exactly the same as at the start, it equally shows our guessers were very uncertain about what direction it would move!).

Deaths: This metric was highly cyclical, due mainly to seasonal effects where deaths are typically higher in winter then they are in summer. Interestingly, we can also see there is a dip in the plot around Christmas. This is likely due to the fact we are measuring the deaths recorded over the 13 week period as opposed to the actual number of deaths. As there are lots of people off work over the Christmas break, some deaths may not get recorded until the next week. This year, the Easter bank holiday weekend fell in April and was not included in the period, whilst the same period last year included Good Friday as Easter fell in late March. This meant there was a day fewer of people working and recording deaths, and possibly explains why the figure this year was almost 8,000 higher than over the same period last year if we had a similar effect as to Christmas. There could be other reasons for this too though, such as population growth.

The AI of it all

Last year, we fitted an ARIMA (autoregressive integrated moving average) model to try and predict some of the values and compare against our participants. It did not do well. With a score of 8.779 it performed well below average, but this was significantly affected by something we highlighted at the time – it lacked context. It thought that the 10-year gilt yields would keep rising, whilst our humans understood it was likely to keep tailing off, and it also predicted the base rate as something that wasn’t a multiple of 0.25% – the horror!

This year, we decided to put AI (in this case Copilot) to the test to see if it could do any better. How did it fare?

Clearly Copilot fared much better than the ARIMA model from last year, as it scored in the top four of both competitions and would have won the GIRO one! Copilot appeared to understand context much better than the ARIMA model did, predicting the base rate as a multiple of 0.25 (restoring order to the universe) and making some impressive other guesses along the way, especially for the exchange rates where it outperformed everyone else for the GBP-EUR rate. Who needs currency swaps when you can just ask your computer?

Don’t Panic! There are reasons to believe we won’t all be replaced by our computers in the next year. Copilot was working with a lot more data than our participants, who were given just one source for each metric. We also asked Copilot for its predictions at a later date than our contestants, asking for Copilot’s input at the end of the conference period, which meant it had better information on certain trends such as gilt yields and rainfall, which had already started to rise/fall respectively.

Conclusion

All the content that we’ve revealed so far represents only the first half of the tournament; we will be back later this year, where we will cover the results for June 30^th, reveal our final winners, and, most importantly, see if Copilots success was just a fluke. We look forward to seeing you then and wish all of our predictors the best of luck for the second round!

Footnotes

[1] Bank Rate history and data | Bank of England Database (07/05/2025)

[2] Weekly road fuel prices – GOV.UK (07/05/2025)

[3] Deaths registered weekly in England and Wales, provisional – Office for National Statistics (07/05/2025)

[4] GBP exchange rates | Bank of England | Database (07/05/2025)

[5] Yield curves | Bank of England (07/05/2025)

[6] We have changed our source as the original source had not been updated. We believe the source uses the same data, as the figure for March 2024 was the same in both sources. The updated source only provides the information for the last month, so we have provided a secondary source which referenced the March figures (equally feel free to check the wayback machine!) https://www.smmt.co.uk/vehicle-data/car-registrations/ (07/05/2025), https://www.msn.com/en-gb/cars/news/new-car-sales-accelerate-124-with-record-volume-of-ev-deliveries-but-it-is-still-well-below-government-targets/ar-AA1ChUHf?ocid=BingNewsVerp (07/05/2025)

[7] https://www.ons.gov.uk/economy/inflationandpriceindices (07/05/2025)

[8] https://www.metoffice.gov.uk/pub/data/weather/uk/climate/datasets/Rainfall/date/UK.txt (07/05/2025)

[9] https://www.bankofengland.co.uk/boeapps/database/Rates.asp (07/05/2025)

[10] Gross Domestic Product: Quarter on Quarter growth: CVM SA % – Office for National Statistics(07/05/2025)

Ben Tocher

May 2025