29.07.2025

The AI Risk Landscape

AI and the insurance industry

The insurance industry stands on the cusp of profound change. As with any period of transformation, new opportunities will emerge — but so too will new risks. The rise of Artificial Intelligence (AI) is already influencing every aspect of our industry and is set to accelerate over the coming decade. From reshaping pricing methodologies and transforming claims management to redefining valuation processes, AI’s reach will be extensive.

A 2023 survey by EIOPA found that 50% of non-life and 24% of life insurers already use AI, with another 30% and 39% of respondents planning adoption within three years. As AI embeds itself across insurance operations, it brings both exciting opportunities and new risks [1].

While AI is fast becoming a tool for identifying and mitigating risk, it also introduces its own category of emerging risks – including biases, cyber vulnerabilities, operational risks. The irony? The very system designed to detect risks might struggle to recognise the risks it introduces. Insurers must position themselves proactively to identify, monitor, and manage these emerging risks while navigating future evolution of regulatory frameworks.

Ben Fletcher, Director of Financial Crime at LV=, discussed ethical considerations in AI deployment:

“At the moment, there’s a big focus on how do we make use of the tech – the ethics conversation of ‘could we, should we’ is going to escalate quite significantly [2].”

In this article, we will look at the different types of AI being used in the industry and some of the risks introduced as a consequence of its use. We then consider some of the ways insurers can manage and mitigate these risks, before taking a look at the regulatory landscape as it relates to AI.

Understanding the types of AI in insurance

So far, insurer’s use of AI has focussed on Narrow AI, such as the use of machine learning designed for specific tasks with a narrow domain with no “thinking” involved. But Generative AI (GenAI) is quickly gaining traction. Unlike narrow AI, GenAI creates new content or outputs, including text, code, or even synthetic data, using large language models (LLMs).

For example:

Narrow AI might use machine learning to assess underwriting risk based on structured data
GenAI could generate underwriting reports, suggest pricing models, or even write code for new risk models.

Each type of AI brings different risks and governance challenges.

AI use introduces new risk…

The risks introduced by AI use are often complex, hard to detect and often poorly understood. They pose fresh new challenges and the first step towards managing them is understanding what the risks are. Below we consider some of the most material risks to the insurer emerging through increased AI use.

Cyber risks

Cyber risks are perceived as the main risk arising from increased AI use. A cyber-attack on AI infrastructure could have significant ramifications for an insurer. The more embedded and automated the processes are, the more material the potential impacts of a cyber-attack – not least due to the vast amounts of personal data that may be used.

As insurers begin to employ AI agents – goal-directed GenAI systems that act autonomously to perform tasks and make decisions without direct and constant human input, the threat of bad actors taking advantage of this increases.

The use of AI provides a larger attack surface for cyber criminals to act on. It introduces new entry points for cyber criminals to act on as the number of systems increases. It also introduces new attack vectors for them to use. These include:

Adversarial examples: An attacker may be able to get specially crafted incorrect inputs (adversarial examples) into existing AI systems to achieve a malicious goal, such as generating false results.
Model poisoning: An attacker may introduce malicious data into training data set used to develop the AI before it is deployed. This could fundamentally change the behaviour the model. [3]
Data breaches: AI models require huge datasets which often contain personal or financial information. Breaches or leaks to such data could violate data protection regulations.
Shadow AI: Use of GenAI without an organisation’s knowledge or approval (“shadow AI”) could lead to sensitive data ending up in unsecured platforms.
Model theft risk: Bad actors may attempt to steal or clone proprietary models.

Complex AI models such as deep neural networks may operate without providing clear reasoning as to decisions made or the outputs achieved. A lack of explainability is a risk caused by the technical advancement of AI models, whereby it may be difficult to understand how the models are arriving at some conclusions.

This “black box” problem creates challenges for:

Explaining decisions to regulators
Justifying assumptions to boards and risk committees
Complying with regulatory requirements under Solvency II and IFRS 17…

Without explainability, validating AI-driven assumptions or decisions becomes a material risk.

Hallucinations and errors

Generative AI systems can produce hallucinations – where LLMs produce misleading, false or nonsensical outputs that may look plausible without careful interrogation. This risk could manifest in:

Modelling: recommending unsuitable methods or distributions
Underwriting: making inappropriate coverage decisions
Claims: rejecting valid claims

Unchecked, such errors could lead to financial loss, regulatory breaches, and reputational harm.

“Once we have stopped hallucinations, generative AI could become a formidable underwriting tool for group risk [4].”

Josh Hart, Co-founder and Chief Technology Officer at YuLife

AI to AI interactions

As AI systems proliferate across different functions, AI-to-AI interactions become more common. This can result in unintended outcomes or automated decisions with harmful, biased, or even unlawful effects—particularly without human oversight [5].

Bias and Discrimination risks

AI models trained on historical data may inadvertently perpetuate or amplify biases, leading to discriminatory outcomes in pricing, underwriting or claims. This could expose insurers to regulatory penalties and reputational damage.

Managing AI Risks: Practical steps for insurers

Having established many of the additional risks that come with increased use of technology, and particularly GenAI, it begs the question: what can insurers do to mitigate and manage these risks?

It is important to have the right policies, systems, and frameworks in place to manage these risks, and to ensure company-wide awareness of the technology—its capabilities, limitations, and risks.

Increased cyber security controls

A combination of increased technical controls, governance and cultural change can be employed to mitigate the cyber risks coming from increased AI use. To combat some of the specific risks such as adversarial inputs or model poisoning, an insurer could deploy specific systems to detect such AI targeted attacks.

Using advanced security technologies such as firewalls and intrusion detection systems and ensuring sensitive data is encrypted can help protect against cyber threats. Additionally, some of the other monitoring and mitigation techniques mentioned below such as having robust governance frameworks and well-trained employees can help a company protect against and respond to cyber threats.

Risk assessment and testing

AI systems should undergo proportionate risk assessments and robust testing before using. This might include:

Simulating potential failure modes.
Stress testing AI outcomes under adverse scenarios.
Monitoring AI decisions over time to detect model drift or anomalies.

EIOPA regulation says that AI systems should be risk assessed and governance measures should be proportionate to the risk level [6].

AI governance framework

Development of a robust AI governance framework is one of the most effective ways in insurer can responsibly manage the growing use of AI across their models and processes. It allows insurers to

Proactively identify and monitor AI risks.
Establish accountability for AI-driven outcomes through clear ownership.
Ensure transparency and explainability.
Align AI use with regulatory requirements.

The framework should be cross functional, involving legal, risk, IT, business stakeholders etc. We consider some key principles underpinning a robust governance framework below.

AI governance framework: Maintaining human accountability /ownership /sign-off

Where AI systems are making decisions or interacting with other AI systems within an insurer’s operations, maintaining human oversight and requiring sign-off at key stages can help mitigate many of the risks associated with unintended consequences. Striking the right balance between autonomy and oversight is key: enough autonomy for efficiency gains, but enough control to ensure accountability.

AI governance framework: AI training for employees

Insurers should equip employees with the skills and knowledge to ensure that they can effectively leverage these technologies, and be aware of the risks and the ways in which it can go wrong.

Training should be provided for:

Actuaries: to design and audit AI models with statistical rigour and compliance, and to understand how the models work and arrive at decisions.
IT & data teams: to ensure secure, scalable and ethical implementation.
Compliance & legal teams: To stay ahead of evolving AI regulations and ensure transparency.
Underwriters: To interpret AI-generated risk scores and integrate them with human judgement.
Executives: To make informed decisions about AI investments and business strategy.

Best practice in fairness, transparency and accountability should be taught to those using AI, particularly given that the stakes of AI misuse are so high. Embedding AI literacy throughout the company will help safeguard against systemic risk and can help drive innovation while maintaining trust, compliance and control.

Model validation framework

The capabilities (and hence uses) of GenAI completely differ to those of narrow AI. We therefore need to consider from scratch how we might tackle the new risks. If we are using GenAI for modelling purposes, e.g. to help us build a cashflow projection model for reporting purposes, then we should break down into detailed stages of the control cycle (e.g. scoping, data collation /assumption choice, data validation, calculation methodology etc) and ask the following questions [7]:

If we take one of the stages of the control cycle as an example:

Stage	How AI might be used	What could go wrong	Managing the error
Calculation Methodology	AI could recommend a common methodology for a particular calculation within the model.	AI may recommend inappropriate methodology which doesn’t meet relevant regulatory requirements.	Input stage You should ensure the inputs into the AI give the relevant information, such as which regulatory body you need to adhere to. Review stage Check the regulatory standard to ensure the methodology does not contravene any requirements. Query the AI as to how it arrived at the particular choice of methodology. For neural networks operating on a probabilistic basis it is important to note input sources due as it may not be possible to reverse engineer these out.

Stage

How AI might be used

What could go wrong

Managing the error

Calculation Methodology

AI could recommend a common methodology for a particular calculation within the model.

AI may recommend inappropriate methodology which doesn’t meet relevant regulatory requirements.

Input stage

You should ensure the inputs into the AI give the relevant information, such as which regulatory body you need to adhere to.

Review stage

Check the regulatory standard to ensure the methodology does not contravene any requirements.
Query the AI as to how it arrived at the particular choice of methodology. For neural networks operating on a probabilistic basis it is important to note input sources due as it may not be possible to reverse engineer these out.

The above table could be extended to cover the full control cycle and there can be multiple ways in which each stage could go wrong. Similarly, there are various ways of addressing each potential pitfall.

Understanding what the AI is being used for and where it can go wrong is crucial for mitigating against such errors.

Explainable AI (SHAP / Lime)

There are techniques to mitigate the black box nature of AI. For those aiming to enhance the transparency of the complex machine learning models introduced, the concept of Explainable AI (XAI) could prove useful.

Given regulatory expectations for model validation and transparency, insurers should adopt Explainable AI (XAI) tools such as:

SHAP (Shapley Additive Explanations): decomposes model predictions into contributions from each feature, enabling actuaries to understand key drivers of predictions. Note: SHAP can be computationally expensive, especially for large datasets and models with many dimensions [8].
LIME (Local Interpretable Model-Agnostic Explanations): approximates complex model behaviour locally around a prediction to explain individual decisions. It can be applied to any machine learning model, including linear models, neural networks and tree-based models, making it versatile for various actuarial applications. [9].

Both of these tools can be used to enhance transparency and interpretability of AI models, acting as a crucial tool for their validation.

Capital requirements

A key question in the mind of the insurer should then be: do we need to hold additional capital against the risk of AI failures? Will such capital buffers eventually be required under evolving regulatory frameworks?

The use of AI introduces a new layer of model risk – particularly where decision-making is automated or influenced by complex algorithms. Holding additional capital against this risk may provide a safeguard against adverse outcomes, including model failures or unintended consequences not anticipated in traditional actuarial models.

Second order impacts of such failures could come from legal action or mass customer complaints where AI has led to unfair outcomes. This leads to additional operational risk against which the insurer may want to hold capital.

The Regulatory landscape

Another consideration to take into account is the introduction of new regulation. The problem here is the difficulty in predicting what may need to be protected against, as there could be unintended consequences of AI use. Additionally, inconsistencies between countries’ attitudes to risk and ethics mean that hopes of consistent application is unlikely. Lack of international and even domestic consensus is unlikely which will hinder the chance of buy-in across the board [10].

In the UK there is no single comprehensive AI regulation, but instead it is proposed that a sector-based principle-led approach will be adopted. Guidance for regulators has been issued in the form of a White Paper which includes a framework underpinned by 5 principles [11]:

Safety, security and robustness
Appropriate transparency and explainability
Fairness
Accountability and governance
Contestability and redress

Article 8.1 (f) of the EIOPA Regulation 1 mandates EIOPA to “monitor and assess market developments in the area of its competence including where relevant, developments relating to trends in insurance, reinsurance and occupational pensions, in particular, to households and SMEs and in innovative financial services duly considering developments relating to environmental, social and governance related factors” [12].

When integrating AI, some existing regulation should be considered even though not AI-specific. For example:

Article 41 of Solvency II already requires insurers to maintain effective governance proportional to operational complexity—applicable to AI systems.
Article 82 requires data used for technical provisions to be “complete, accurate, and appropriate”—applying to AI-driven data and models.

In 2024, the House of Lords committee published a report on LLMs and GenAI where it flagged challenges that their use poses to liability and accountability, particularly where risks are not fully understood or mitigated against.

The Financial Conduct Authority also includes regulation applicable to AI, particularly in the areas of transparency and explainability [13]:

The FCA will use the Senior Managers and Certification Regime (SM&CR) to supervise AI use.
Responsibility for AI should be allocated to a senior manager with the necessary qualifications.

They highlight the importance of transparency in AI models, especially for customer-facing applications.
FCA emphasizes the need for firms to communicate decisions in a meaningful way to consumers,

Conclusion

AI will help reinvent many functions within the insurance industry and lead to unprecedented efficiencies. But we cannot only look at the positives. We have considered a number of potential pitfalls and risks associated with AI use for the insurer, and discussed some of the ways in which we can monitor and mitigate against them.

The industry will have to evolve to from cautious adoption to responsible integration as the pace of change gets ever quicker. Those who are well prepared and with a good understanding of AI’s capabilities, but also where it can go wrong, will be in the best position to harness its benefits while safeguarding against its potential pitfalls.