Artificial intelligence is transforming virtually every industry, from healthcare and pharmaceuticals to finance and national security. Yet as adoption accelerates, so does evidence of harm. The Organization for Economic Co-operation and Development (OECD) AI Incidents Monitor (AIM), which tracks publicly reported AI incidents and hazards worldwide now contains more than 16,000 documented incidents and hazards, providing one of the largest evidence-based repositories available for understanding real world AI failures.
A Lachman review of the OECD database reveals that current and known AI risks are increasing and may impact the pharmaceutical and biopharmaceutical industry either directly or indirectly. Many are already materializing as operational, ethical, legal, and societal problems. Ignoring these signals may not only hinder adequate mitigations but also impede adherence to established laws and those that are emerging. Four elements that are relevant to this industry are:
- EU AI Act Article 74: Market surveillance and control of AI systems in the Union market (existing)
- ICHQ10: Section 3.2.1 states, “An effective monitoring system provides assurance of the continued capability of processes and controls to produce a product of desired quality and to identify areas for continual improvement” (existing)
- ICHQ9: Introduction states, “The application of digitalization and emerging technologies in the manufacture and control of drug (medicinal) products can lead to risk reduction, when such technologies are fit for their intended use. However, they can also introduce other risks that may need to be controlled.” (existing)
- EMA-FDA Guiding principles of good AI practice in drug development (here): Principle 8 and 9 both refer to performance assessments and scheduled monitoring (existing agreement of principles, planned as future input into planned formal AI guidance).
Existing Thematic Risks
The OECD AI incident monitoring database identifies recurring categories of harm including privacy violations, discrimination and bias, misinformation, security failures, human-rights impacts, reputational damage, and threats to safety. Some of the notable trends are:
- synthetic media and deepfakes
- bias and discrimination
- privacy and data governance
Anticipated Thematic Risks
While these are today’s observed risks, future concerns may be even more significant. This is an opportunity for leaders in pharma/biopharma to see the signals that may affect them in the future. Researchers at MIT’s AI Risk Repository surveyed 272 international experts to identify the most dire emerging AI risks over the next five years. Their findings suggest that many AI risk domains carry a meaningful probability of catastrophic outcomes if current trends continue. Experts concluded that 18 of 24 risk categories have at least a 10% probability of causing catastrophic harm under a business-as-usual scenario (See: Priority AI risks)
The MIT study identified five emerging risks as particularly severe:
-
- Dangerous AI capabilities
- Competitive pressures and AI races
- AI-enabled cyberattacks and weaponization
- Power concentration among a small number of organizations
- False or misleading information at scale
For regulated industries such as pharmaceuticals, healthcare, and biotechnology, these findings offer an important lesson. Traditional compliance frameworks focus on historical failures, while AI risk management must increasingly address emerging risks that have not yet fully materialized as described in ICHQ9. Organizations should therefore monitor both current incident patterns and future risk signals.
The OECD incident data shows where AI is failing today. The MIT expert assessment highlights where AI could fail tomorrow. Together, they suggest that the future of AI governance will depend not only on improving algorithms but also on strengthening oversight, accountability, transparency, and risk management before isolated incidents evolve into systemic failures. The fundamental question remains, “Is your organization self-aware of the signals that exist today?” Lachman Consultants helps companies determine their organizational readiness with robust and advanced Data Governance programs. Accelerate your integration of technology, risk management, and organizational awareness today and reach out to us at LCS@LachmanConsultants.com for a consultation today!
