The Fault in your Algorithms: Who pays for it?

Introduction

AI and algorithms become increasingly embedded in various aspects of business and society, with lot of automated and assisted decision-making process. With the growth of the AI industry (~$600 Bn today), the potential for these products to cause harm — whether through errors, biases, or unintended consequences — has also grown. Meta for example settled a class-action lawsuit of $725 million in 2023 as an outcome of Cambridge Analytica scandal (one of the largest payouts in data privacy case). The plaintiffs argued that Meta’s algorithms had allowed third parties to access personal data without consent.

Large insurers and new players are exploring this niche market to cover these risks as the demand grows. This article throws light on the various types of damages that can be caused by the adoption of the AI in various industries, looks at the how the insurance product design will evolve and how pricing mechanisms can change for actuaries.

Risk landscape

The risks and losses arising from the usage of AI:

Fig. 1 Broad classification of risk

Physical Costs

· Sensors and algorithms in self-driving cars or drones could lead to accidents, causing physical harm to passengers, pedestrians, or other road users. Autonomous cars have faced issues navigating complex construction zones in Arizona. The car’s sensors detected the worker but misclassified them as an “unknown object” and later as a “bicycle,” causing delayed response time.

· Algorithms that control medical devices like insulin pumps, pacemakers, or diagnostic tools could malfunction, leading to incorrect dosages, delayed treatments, or misdiagnoses, resulting in injury or death. Philips has recently faced ventilator software failure issues and the Class I recall forced to pay out more than $1bn in lawsuits just few months back.

· In manufacturing environments, robots or machinery controlled by algorithms could malfunction, rise product feature issues. At a Foxconn electronics plant in China, a robotic arm assembling smartphone components malfunctioned to cause it operate with excessive force. Several workers were injured by flying debris when components were incorrectly handled, leading to downtime and worker compensation claims.

Financial Costs

· Faulty algorithms that miscalculate credit scores can cause financial injury by leading to unfair loan denials, higher interest rates, or other adverse financial consequences. In 2021, certain users reported sudden and significant drops in their FICO credit scores despite no changes in their financial behaviour just owing to a data integration error that caused incorrect payment histories

· A bank’s fraud model can incorrectly freeze accounts based on flawed criteria, causing customers to be unable to access their funds. Affected customers could potentially sue under consumer protection laws. Amazon uses AI-driven fraud detection models to identify suspicious seller activities such as counterfeit product listings or unusual sales patterns. In 2021, a U.S. electronics seller sued Amazon for $100 million, claiming wrongful account suspension caused significant business losses.

• Algorithmic trading systems can trigger large-scale market disruptions if they execute trades based on flawed logic or if they react to market anomalies in a cascading manner. AI-driven trading algorithms, also known as algorithmic trading or high-frequency trading (HFT), played a central role in the flash crash earlier this year.

Social Costs

· Algorithms that determine access to services like loans, insurance, or education can create or reinforce economic disparities if they favour certain groups over others.

· Legal and reputational damage can result from the unethical use of GenAI, such as generating harmful content or deepfakes. GenAI might be in faced used for unethical purposes, such as generating deepfakes or harmful propaganda, leading to legal and reputational damage.

· Protection against breaches of sensitive data that occur due to algorithmic failures or vulnerabilities. A very large data breach can result to a cyber liability for the organization

Defining the covers

Algorithmic liability insurance is a response to these risks. It is designed to protect companies from the financial consequences of errors, omissions, or other failures in their algorithms and AI systems. Traditional liability assumes that people cause accidents, and the litigation involves two humans, hence need to understand who is paying the premiums and who is liable for the losses.

Given the liability sits with the device/manufacturer, the insurance can be embedded that and if an algorithm is part of a product that causes harm, product liability coverage can apply. For example, if a faulty algorithm in a consumer product leads to injury or damage, the insurance could cover related claims.

E&O insurance can be expanded to cover algorithmic liability more specifically. It can protect against claims arising from professional mistakes or failures, including errors in algorithms that result in financial losses, incorrect decisions, or other damages.

When it comes to medical malpractice, the usage of algorithms to detect say cancer for example, brings the risk of human detection down. Hence, overall the frequency of the claims would be intended to come down. However, if it is a miscalculation owing to the algorithm output, it would scale to a very easily to a large extent.

Cyber liability insurance may cover risks associated with algorithms that handle data. This includes coverage for data breaches, unauthorized access, or other cyber-related incidents caused by faulty algorithms.

How is pricing evolving?

Assessing the risks associated with algorithms is complex, as it involves understanding the methodology, the data used to train, where it is embedded, the cutoff thresholds chosen and what is the materiality of the decision being made using the algorithm.

The exposure here is the number of model runs and when multiplied by the probability of giving a false positive or false negative will be the claim frequency. The severity will vary based on the decision making it is used to take — for example in case of reporting a fraud, missing a true fraud can lead to the loss of $value of the transaction, while reporting a false fraud can lead to a loss of $value of legal costs associated with the litigation filed by the customer.

Risk quantification for pricing

Once, the frequency and severity is defined they can be modelled based on the potential loss at various cutoff threshold for a classification problem. The scenarios can be simulated and optimized to minimize the losses and arrive at the right cutoff and corresponding loss cost.

Conclusion

It is a very interesting space to watch out as the adoption is highly dependent on not just the cost & success but also in someone willing to cover the risk associated with negative outcomes of AI. This is where insurance industry has a great role to play and give a spin to the liability insurance the way it is traditionally covered. Industry bodies like ISO and NIST are working on building standard AI frameworks to be followed by various applications. This will pave the path for insurance companies to price and manage the risk better.