Avoiding Actuarial Embarrassment
It’s every actuary’s worst nightmare. You’ve calculated some figures, put your name to them, distributed them far and wide, only to later discover that the numbers aren’t correct. Next comes the grovelling email retracting the numbers and promising to supply corrected figures as soon as possible.
How can this horror show be avoided I hear you ask? Sadly, there’s no sure-fire way, but there are steps that can be taken to make it far less likely:
Sense Checks
When doing any calculation, it’s helpful to have an idea in your head of roughly where the result should be. For example:
- If calculating lapse rates on a book of business, your answer must be between 0% and 100%, and probably well within the lower quarter of that range.
- Annual investment returns on a fund are generally small in magnitude. If you’re calculating annual returns of 5000%, it’s likely something has gone wrong.
Often, calculation errors lead to results so far from the sensible range that even the briefest visual inspection of results will reveal something has gone wrong. Taking a few minutes to step back and consider the sensibility of the results can save a lot of time and hassle later.
Sense checks can also be made with reference to a separate calculation. For example, if the equivalent calculation has been done during a previous year or quarter, that result can be viewed as a ballpark figure against which to verify an updated result. Further still, the sensibility of any movement between the two results can be assessed considering changes in conditions or inputs over the intervening period.
Going back to our investment return example above, a good check may involve looking at how a similar market index has performed. If the fund mainly invests in UK equities, the movement in the FTSE100 index should be a useful comparison.
Inbuilt checks
Inbuilt checks are automated systems that are designed to identify errors or inconsistencies in actuarial calculations. These checks can take many forms, including validation rules around inputs, checks on intermediate calculation steps, and checks on the results.
The purpose of inbuilt checks is to ensure the accuracy and reliability of calculations. They can help to identify errors at an early stage, which can save time and effort in the long run. For example, if a calculation model includes a check on intermediate steps, it can alert the actuary if an error is made at any point in the process, allowing the error to be corrected before the result is produced.
In some cases, seemingly small errors early in the calculation process will lead to questionable output and require significant time investment to find the root cause. Investing the time to create inbuilt checks will reduce the risk of errors and significantly reduce the time required to investigate any calculation issues. Models which involve multiple calculation steps or complex formulae are prime candidates where inbuilt checks will add value.
Doer/checker process
Have you ever spent hours scouring code for a mistake that is causing your programme to error, only for a colleague to spot the issue in minutes after looking? This is a common example of where an extra pair of eyes can be so useful in the actuarial world.
The doer/checker process is a method used to ensure the accuracy and reliability of actuarial calculations. In this process, one team member, known as the doer, performs the calculation and produces the result. Another team member, known as the checker, reviews the calculation process followed by the doer and verifies that they are satisfied with the result.
Having two pairs of eyes on the calculation can significantly reduce the likelihood of errors occurring. This is because the checker can identify any mistakes or inconsistencies in the calculation process, as well as ensure that all necessary steps have been followed. The checker can also verify that the result is reasonable and consistent with expectations.
The doer/checker process is often used in complex or high-risk calculations, where the consequences of errors could be significant. It can also be helpful in cases where the calculation process is particularly lengthy or involves multiple steps, as it can help to ensure that all steps have been completed correctly.
Audit
Audit is an important tool for ensuring the accuracy and integrity of actuarial calculations. It involves a review of the work performed by actuaries by an independent party, either within the organization or externally. The purpose of the audit is to assess the quality of the work, identify any errors or areas of improvement, and provide assurance that the work meets professional standards and regulatory requirements. Audits can be performed on a variety of actuarial work, including reserve calculations, financial projections, and insurance pricing.
There are several benefits to performing an audit on actuarial work. Firstly, it helps to ensure the accuracy and reliability of the calculations, which is essential for informed decision-making and risk management. This is especially important in cases where the calculations are used to make important business decisions or to inform regulatory reporting. Secondly, it helps to build confidence and trust in the actuarial profession, as it demonstrates that the work has been thoroughly reviewed and meets high standards of quality. This can be particularly important when dealing with clients or stakeholders who rely on the accuracy of the actuarial work.
There are several different approaches to conducting an audit. An auditor may review a sample of the work or may review all the work in detail. Alternatively, the auditor may review the documentation and records related to the actuarial work and may interview the actuaries who performed the work to better understand their process and assumptions.
Overall, audit is an important tool for ensuring the quality and reliability of actuarial work. It can help to identify errors and areas for improvement and provide assurance that the work meets professional standards and regulatory requirements. By seeking audit and incorporating any recommendations for improvement, actuaries can enhance the quality of their work and better serve their clients.
Version control
It’s 6pm, you’ve spent all day making tweaks to your results in line with feedback from your manager and finally have a set of final actuarial results. You’ve followed this guide and performed a range of checks to ensure they are sensible: it’s time to hit send and enjoy the evening.
Fast forward to the next morning and you have a message from your manager asking if your results include the late adjustment discussed yesterday. After some careful checking, you realise that you’ve sent the ‘Actuarial results 0.2 (3) final AH amendments’ file instead of the correct ‘Actuarial results 0.2 (3) final FINAL AH amendments’ file.
You’ve fallen at the final hurdle, and a quick ‘updated results’ email to the long distribution list is sent within minutes.
Good version control is essential and can help you avoid issues like the one above, with little extra effort:
- Choose a naming convention for your results and stick to it. One possible option is to use the ‘Model name Vx (Y)’ convention. Where x is the version number of the model (which will only change where a model development is made) and Y is the version number of your results (which changes as a result of altering an input in the model or adding an adjustment to the output).
- Create an archive folder and move previous versions into this folder to avoid a cluttered folder with multiple versions. This is particularly important where the number of versions is high.
- Where results are shared as final or for review, create a sub folder called ‘shared with X on DD/MM/YYYY’ and save a copy of the email sent to keep track of what has been shared with who.
Conclusion
Such is the tough, unforgiving nature of the actuarial world that there is no way to make sure every calculation you perform is correct. Even the best actuaries drop a clanger sometimes. However, by bearing in mind the methods of checking discussed above, you give yourself the best chance of keeping errors to a minimum.
George Fairbrother and Matthew Gregg
March 2023
