GenAI’s emerging risks: How should companies prepare for a world of new exposures?

By James Tuplin, Head of International, Cyber; Jay Vinda, Global CISO & Risk Engineering Lead, Cyber; and Usha Badrinath, Chief Data Officer

A 360-degree view of the cyber, operational, and third-party risks around development and in-house adoption of GenAI models

Generative artificial intelligence (GenAI) is a subset of AI and machine learning that can create, summarize, and analyze unstructured data based on several modes, including text, image and speech and provide outputs in similar formats. It is trained on a large corpus of data that allows it to take on personas in scenarios, share outputs, or think logically, much in the way real-life individuals would. Its ability to generate content, personalize experiences, and analyze data opens a new world of possibilities for businesses—but also potential for rampant abuse of this emerging technology.

Traditionally, accessing AI or machine-learning models required interacting in a language understood by models, thereby limiting access to deeply technical roles such as data scientists and engineers. For years, machine learning languished in the shadow of its more commercially applicable kin—the larger category of predictive analytics—before deriving an identity and applicability of its own and moving into the remit of business analysts and consultants. With the advent of GenAI models using natural-language processing to broaden accessibility, this technology has now moved solidly mainstream, allowing lay users to interact and receive answers in human language.

Navigating the perils and promises of GenAI for cybersecurity

GenAI adoption bears a resemblance to the move several years ago from on-premises solutions to cloud technologies; just like that transition, businesses are now feeling the pressure to integrate new tech into their workflows to remain competitive and realize benefits. In contrast to the widespread adoption of cloud computing, we’re observing a more cautious approach with GenAI. Security and technology teams are taking the lead, carefully implementing the technology even before comprehensive safeguards are in place. However, this caution isn’t universal. Some organizations are rushing to adopt GenAI without prioritizing security, resulting in premature data breaches and misuse. This highlights the critical need for a ‘secure-by-design’ approach in GenAI implementation, with security, privacy, and responsibility at the core. Security architects need to understand the intricacies of how applications using GenAI are built and integrated into their ecosystems and be able to model potential threats and develop secure patterns.

The adoption of GenAI by organizations has significantly amplified their exposure to both existing and emerging operational and third-party risks. This technological shift has created a double-edged sword, on the one hand empowering cybersecurity teams to increase their defensive capabilities, while simultaneously providing threat actors with tools to automate and scale their offensive techniques. The proliferation of public large language models has provided threat actors with the means to increase the speed, efficiency, and sophistication of their attacks. So, there is a real urgency for organizations to address GenAI-related threats and implement robust governance and controls around AI exposure, whether they intend to proactively adopt the technology or not.

The GenAI adoption spectrum and cybersecurity considerations

The exposure of GenAI to organizations can be categorised according to the GenAI adoption spectrum, i.e. the varying levels at which businesses and individuals integrate AI into their workflows:

1. Consumption of public use GenAI models: This involves leveraging publicly available tools like ChatGPT for general use cases
2. Tuning of pre-trained GenAI models: Organizations collaborate with providers like OpenAI to customize ‘Model-as-a-Service’ solutions (e.g. ChatGPT Enterprise) for specific applications
3. Integration into third-party services: GenAI models are embedded into existing applications (e.g. CRM software), used to scale services (such as professional service providers), or incorporated into newly developed proprietary applications

Interactions with GenAI models span three critical elements—the user interface, the model itself, and the data layer—all of which must be considered when developing a threat model. To aid this process, our team at Mosaic has designed a reference architecture (see graphic above) alongside a non-exhaustive list of potential security, technology, and data threats. As GenAI architectures and associated risks evolve, organizations and, indeed, insurers, must continuously refine their threat models to stay ahead of emerging challenges.

Effective GenAI integration hinges on a thoughtful approach, grounded in a deep understanding of an organization’s unique processes. This tailored strategy ensures GenAI is implemented effectively and delivers maximum value. To mitigate the risks, organizations should enhance their third-party risk– processes by addressing the following areas:

• Improve visibility of GenAI usage within the organization: Extend shadow IT identification capabilities to include GenAI tools and incorporate targeted questions during third-party risk assessments
• Embed security and privacy in DevOps: Integrate security and privacy measures into DevOps workflows to protect data during ingestion, processing, and storage
• Enhance third-party risk assessments: Include specific questions in your risk-management processes when evaluating vendors or partners using GenAI:
  • Do you have capability to roll back changes to previous model images if required?
  • How do you ensure my data is not incorporated into models accessible by other customers, and how do you prevent me from inadvertently consuming their data?
  • What measures are in place to proactively prevent intellectual property (IP) leaks, ensuring my competitors or your other customers don’t benefit from my model training or data?
  • How do you safeguard the value and usage of my IP within my organization, especially if there are shifts in your models or ‘pivots’ in your company’s strategic direction?
  • How do you ensure my data is deleted and not persistent in a fine-tuned model following termination of our relationship?

The Intersection of insurance and AI: Coverage gaps and emerging risks

So where does insurance fit into all this? While cyber insurance policies often cover some AI-associated risks, particularly those involving personal or corporate information, significant gaps in coverage remain.

Data protection and GDPR risks: Most GenAI tools used today are owned by third parties and publicly available, posing unique risks. Using personal data in GenAI tools requires explicit consent under GDPR. There have been several court cases in the EU recently around the use of third-party chat bots. Sensitive personal data had to be entered in the chat bot in the course of interactions, which meant data entered the GenAI tool of the third party as well as the intended recipient. Once data is in a GenAI tool, it is very hard, if not impossible, to remove it. It is also possible for third parties other than the intended company to access the data. For sensitive personal data, this would be a data breach under GDPR; for corporate information, it would potentially be a breach of confidentiality. Another scenario, and the focus of multiple court cases, is the training of an AI tool by scraping data off the internet, which requires permission.

Most data-related scenarios, including GDPR fines (subject to insurability), should be covered under cyber policies, whether for GenAI tool users or providers.

The EU AI Act and extended risks: The act introduces further risk factors. Some clearly align with GDPR principles, such as providing transparency, ensuring accuracy, or informing users that they are interacting with an AI tool. Others extend beyond typical cyber policy coverage, such as establishing a risk-management system, logging system activity, or implementing human oversight. Failure to implement some of these elements might not trigger a cyber policy if they don’t breach GDPR or other similar data regulatory guardrails.

Third-party financial loss: Cyber insurance policies are unlikely to cover third-party financial losses incurred by customers due to a company’s use of AI tools, especially when these losses are unrelated to personal data or data regulation. This is not an oversight but an intentional exclusion—cyber policies were never designed to address this type of risk. It doesn’t mean these risks are uninsurable; they simply fall outside the scope of traditional cyber policies. The insurability of risks associated with GenAI tools depends on who made the tool, who used the tool (and for what purpose), and who was affected by use of the tool.

Aligning insurable interests with the GenAI adoption spectrum

For many risks and threats associated with GenAI, the insurable interest falls within the realm of technology errors and omissions (Tech E&O) policies. Traditionally, Tech E&O insurance was the domain of companies designing, building, selling, and licensing technology products or services. They provide a professional technology product to a third party and may be held liable for third-party damages if the product is proven to be defective. In cases where the loss is financial and stems from faulty technology or negligent service, Tech E&O policies typically offer coverage. However, the widespread adoption of GenAI has blurred these lines, with non-tech organizations increasingly resembling tech companies in their capabilities and offerings. For instance, retailers are now leveraging GenAI tools to enhance customer interactions within their business processes. Some are taking this a step further by customizing and fine-tuning the GenAI models’ training data to address specific use cases unique to their operations.

Here’s our perspective on some scenarios where insurable interests align with insurance policies to provide coverage for companies across the GenAI adoption spectrum:

1. Consumption of public use GenAI models

Direct loss: If an end-user (personal or corporate) experiences financial loss due to incorrect output from a public GenAI model, they might seek damages from the model’s creator. While many GenAI providers attempt to exclude liability, successful claims would typically be covered by the technology company’s Tech E&O policy.

Indirect third-party loss: End-users incorrectly using GenAI tools or misapplying their output become liable for third-party losses. Coverage for such losses would fall under the end-user’s own insurance policies (e.g. public liability, professional indemnity, or Tech E&O for technology companies). If losses result from errors in the tool itself despite correct usage, the end-user’s insurance would initially apply, with potential for offsetting against damages from the GenAI provider.

2. Tuning of pre-trained GenAI models

Organizations customizing generic GenAI tools for resale as Model-as-a-Service essentially become technology firms. These entities should consider purchasing Tech E&O coverage, as most of the liability rests with them.

3. Integration into third-party services

The liability risks associated with this adoption model vary, based on how GenAI is integrated into the third-party service. If the third party is customizing a generative AI model within their offerings, insurance considerations should align with those for tuning a pre-trained GenAI model. Conversely, if the third party is simply using a public GenAI model, insurance considerations should mirror those for end-users of such models.

As the GenAI landscape evolves, organizations must carefully evaluate their insurance needs and seek to match insurance coverage to their specific usage scenarios across the GenAI adoption spectrum. To avoid protection gaps, they may expand beyond traditional cyber policies to include specialized Tech E&O coverage that addresses the unique risks associated with AI technology development and deployment.

Even if an organization believes it has adequate insurance coverage for its use or consumption of GenAI, it is prudent to proactively inform their insurance broker or provider about specific AI use. Such communication not only serves to ensure compliance with policy terms and avoid potential dispute on a claim due to misrepresentation or material changes, but also allows the insurer to carry out accurate risk assessment, confirm coverage under the existing policy, and optimize the policy to provide additional or modified coverage.

Connect with our team at Mosaic Insurance if you’d like to explore your cyber and tech E&O exposure to GenAI, or to learn more about how you can use GenAI to support your