Category: Clinical Safety and Pharmacovigilance

The mechanics of signal detection are familiar enough. A disproportionality score crosses a threshold, a case series is reviewed,an action is taken. Underneath the mechanics sits the actual work: a
series of judgment calls that no dashboard captures. The following are a few themes I keep coming back to in this work.

1. Inputs are not signals

The following are not signals:

• A disproportionality alert crossing a numerical threshold
• A cluster of ICSRs reporting a similar event
• A literature hit on a class effect

These are inputs. A signal is the output of clinical evaluation applied to those inputs, weighted against biological plausibility, exposure context, background incidence, indication, comparator behaviour, and the totality of available evidence. Conflating inputs with signals produces either over-escalation (noise overwhelms the system) or under-escalation (weak but real patterns are dismissed on statistical grounds).

 

 

2. Standardised thresholds are operationally convenient, scientifically limited

Most signal management SOPs apply uniform numerical cut-offs across the portfolio. The same disproportionality threshold, the same case count minimum, the same statistical significance criterion. Identical numbers can carry very different clinical weight depending on the exposure context behind them.
Example. Two products generate three cases of the same serious event:

• Product A: paediatric biologic, recently launched, limited cumulative exposure. Three cases of severe neutropenia against a small denominator represent a non-trivial proportion of exposed patients. The safety database is immature, comparator data are sparse, and the event is clinically serious. The cluster warrants substantive evaluation regardless of where it falls on a statistical threshold.
• Product B: small molecule, 40 years post-launch, tens of millions of exposures, well-characterised safety profile. Three cases of the same event against that denominator are very likely noise, stimulated reporting, or media-driven case generation. Statistical signals on mature products with known profiles frequently re- surface long-established reactions rather than novel risks Threshold calibration by product maturity, exposure base, indication seriousness, and event severity is therefore not optional refinement. It is a precondition for the methods producing meaningful output.

3. Stratified analysis is the analysis

Primary screens on pooled data routinely mask clinically important subgroup effects. Relevant stratification variables include:

• Age
• Gender
• Indication
• Concomitant medications
• Dose and regimen
• Time on therapy
• Renal and hepatic function
• Formulation and route of administration
• Geography and reporting system

 

Example. A product is approved for two indications: mild hypertension (100,000 exposed patients) and a rare severe condition (1,000 exposed
patients). Acute kidney injury is the event of interest.

• Hypertension cohort: 20 AKI cases in 100,000 patients. Crude reporting rate of 0.02%. Consistent with expected background.
• Rare condition cohort: 15 AKI cases in 1,000 patients. Crude reporting rate of 1.5%, approximately 75 times higher than the hypertension cohort and substantially above plausible background.
• Pooled analysis: 35 cases in 101,000 patients. Crude reporting rate of 0.035%. Unremarkable on its face.

Without stratification, the second cohort signal is invisible. The label remains unchanged. The risk in the smaller, sicker population persists undetected. Stratified analysis is not a follow-up step after the primary screen. The primary screen, run on pooled data alone, is structurally incapable of surfacing this category of signal.

4. The case narrative remains the most diagnostic instrument

Quantitative methods receive most of the methodological attention, but the individual case narrative continues to carry decisive evidential weight. Relevant elements include:

• Temporal relationship between exposure and event
• Dechallenge information
• Rechallenge information where available
• Alternative aetiologies and confounders
• Reporter's clinical reasoning and differential

The components of signal evaluation that have not been automated are the components that resist automation. Narrative review is the principal example.

5. Confounding by indication is persistent

Patients receive medicines because they are unwell. The underlying condition can produce events that resemble adverse reactions. Common examples:

• Thromboembolic events in oncology populations
• Serious infections in immunology populations
• Cardiovascular events in diabetes populations
• Hepatic events in hepatology populations

The diagnostic question is what the expected event rate would be in the absence of any drug effect. Without an answer to that question, attribution is unreliable regardless of the statistical output.

6. Communication of graded confidence is underdeveloped

A signal is rarely binary. Strength of evidence accumulates, shifts, and occasionally reverses. Cross-functional decision-makers require a vocabulary richer than confirmed or refuted. Relevant dimensions include:

• Strength and consistency of evidence across data sources
• Biological plausibility
• Specificity of the association
• Dose-response relationship
• Public health relevance
• Reversibility on discontinuation

Collapsing these dimensions into a single binary output understates what the function is actually producing and weakens the basis for downstream decisions on labelling, risk minimisation, and benefit-risk reassessment.

Closing observation

Signal detection methods will continue to improve. What does most of the work in signal management is clinical judgment: deciding what threshold is meaningful for a specific product, how to stratify the data, what a case narrative is really telling you, whether the indication is driving the event rather than the drug, and how to communicate uncertainty without forcing it into a binary.

• Governance and accountability are foundational to the use of artificial intelligence in pharmacovigilance. AI systems do not exist outside the pharmacovigilance system in which they are deployed. Decisions about their design, deployment, oversight, and continued use must therefore be governed with the same rigour as other safety-critical pharmacovigilance processes.
• Governance provides the framework through which decisions are made, documented, reviewed, and revised across the lifecycle of AI use.
• Accountability ensures that responsibility for those decisions remains clearly assigned, enforceable, and retained by the organisation. Together, they ensure that AI supports pharmacovigilance objectives rather than undermining patient safety or regulatory compliance

Accountability Remains with the Organization

• The use of AI does not transfer responsibility to the technology, the algorithm, or the vendor. Accountability for pharmacovigilance outcomes remains with the organisation deploying the AI system.

This accountability includes responsibility for:

• Defining intended use and boundaries
• Determining acceptable risk
• Approving oversight models and controls
• Deciding whether continued use remains justified

Technical activities may be delegated. Accountability cannot.

Governance Across the AI Lifecycle

Effective governance must be exercised across the full lifecycle of AI use. The nature of governance decisions varies by lifecycle phase, but responsibility and oversight must remain continuous.

1. Collection of Specifications and Requirements

Governance begins before development or configuration. At this stage, the organisation defines why AI is being used and how it is expected to support pharmacovigilance activities.
Key governance responsibilities include:

• Defining the intended pharmacovigilance use case and explicit exclusions
• Assessing whether AI is appropriate given the influence and consequence of error
• Defining expected outputs, limitations, and acceptable error tolerance
• Establishing the human oversight model and escalation pathways
• Defining transparency expectations for users and stakeholders

These decisions set the foundation for all subsequent governance activities

2. Development and Change Management

During development and configuration, governance ensures that implementation choices remain aligned with approved requirements. For systems such as large language models, behaviour is often shaped through configuration rather than model training. Prompt templates, retrieval mechanisms, constraints, and post processing logic therefore require governance oversight.

Key controls at this stage include:

• Approval and version control of prompts and configurations
• Definition of who may make changes and under what conditions
• Documentation of how configuration choices affect system behaviour
• Formal change management for modifications that may affect outputs or risk

Changes introduced at this stage are governance events, not purely technical adjustments.

3. Pre-Deployment Sign-Off

Before deployment, governance must confirm that the system is fit for its intended use and that risks are understood and controlled.

This include:

• Review and approval of risk assessments and mitigation measures
• Confirmation that validation and performance evaluation are appropriate to the use case
• Assessment of failure modes, including false negatives
• Approval of human oversight arrangements and user training
• Formal sign-off by accountable pharmacovigilance leadership

Deployment should only proceed once governance responsibilities at this stage have been fulfilled

4.Post-Deployment “Hyper-Care”

Immediately following deployment or significant change, governance focus shifts to close observation of real-world behaviour. This phase recognises that pre-deployment testing may not capture all operational risks.
Governance activities during hyper-care include:

• Enhanced monitoring of outputs and user interaction
• Rapid identification and escalation of deviations or unexpected behaviour
• Review of incidents and corrective actions
• Adjustment of oversight intensity where needed

This phase is particularly important for systems with variable behaviour, where early detection
of issues is critical.

5. Routine Use and Ongoing Oversight

Once systems move into routine use, governance responsibilities continue rather than diminish.
Key activities include:

• Periodic review of system performance and impact
• Monitoring for drift, emerging failure patterns, or increased reliance
• Review of changes, updates, or reconfiguration
• Reassessment of whether oversight remains proportionate to actual influence
• Decisions to restrict, suspend, or discontinue use where appropriate

These activities should be embedded within existing pharmacovigilance quality systems.

Human-in-Command as the Anchor of Governance

Human-in-command oversight underpins governance across all lifecycle phases. The organisation must retain authority to approve use, modify controls, and intervene when risks become unacceptable.
Without this authority, governance becomes symbolic rather than effective. Human-in command ensures that accountability is not diffused across systems, vendors, or organisational boundaries.

Why LLM “Explanations” Are Not an Audit Trail

Generative language models can be prompted to produce explanations for their outputs. These explanations may appear coherent and convincing, but they do not provide direct insight into how the model arrived at a decision. They are language outputs generated after the fact and may not faithfully reflect the underlying computation.

For pharmacovigilance use, organisations should therefore prefer evidence-anchored justifications, such as explicit links to source text, highlighted narrative spans, or referenced documents. Where LLM-generated rationales are used, their reliability should be assessed through faithfulness checks before they are accepted as part of the audit trail.

Governance Considerations Specific to Large Language Models

Large language models introduce additional governance considerations due to their variable behaviour and reliance on configuration rather than direct model training.
In such systems:

• Prompts and constraints are governed artefacts
• Changes to configuration can materially alter behaviour
• Transparency about AI involvement is essential for effective oversight
• Reliance on outputs may increase gradually and must be monitored

These considerations do not alter accountability. They clarify how governance must be exercised to remain effective.

• AI does not distribute accountability.
• It concentrates it at the points where decisions, changes, and reliance occur.
• Governance that ignores this reality creates blind spots rather than control.
• If an LLM can “explain” its output, that does not mean the explanation is an audit trail.
• Governance fails when convincing language replaces traceable evidence.

In pharmacovigilance, accountability depends on what can be verified, not what sounds reasonable.

This post continues our series on Artificial Intelligence in Pharmacovigilance: What Safety Leaders Need to Know.

 

1. Why Human Oversight Is a Control, Not a Reassurance

• As artificial intelligence becomes embedded in pharmacovigilance workflows, the presence of a human reviewer is often assumed to be a safeguard. The implicit belief is that if a person is
  involved somewhere in the process, risk is automatically controlled. That assumption is flawed.
• Human oversight in AI-enabled pharmacovigilance is not a reassurance mechanism. It is a control. Its purpose is to mitigate known and foreseeable risks associated with automated
  outputs, including missed information, biased interpretation, and over-reliance on system recommendations.
• In safety-critical activities, simply adding a human step does not guarantee meaningful oversight. If the human role is poorly defined, lacks authority, or is limited to confirming system
  outputs, the risk profile remains largely unchanged. In some cases, it may worsen, as apparent human involvement can increase unwarranted trust in the system.
• Effective oversight must therefore be intentional. It requires clarity on when human judgement is expected to intervene, what decisions a human can override, and how that intervention materially affects outcomes. Without this clarity, “human review” becomes a procedural label rather than a genuine risk control.
• For pharmacovigilance organisations, this distinction matters. Oversight exists to manage the consequences of error, not to create comfort that a process appears compliant. Treating human involvement as a checkbox undermines its purpose and leaves critical risks unaddressed.

2. What Human Oversight Actually Means in Practice

• Human oversight in AI-enabled pharmacovigilance is not a single concept. It is best understood through three distinct forms of human involvement, each serving a different control purpose

1. Human-in-the-Loop (HITL):

A qualified human reviews AI outputs and has the authority to accept, modify, or reject them before they influence decisions.

2. Human-on-the-Loop (HOTL):

Humans supervise system performance over time, monitor outputs and trends, and intervene when predefined thresholds or anomalies are detected.

3. Human-in-Command (HIC):

Humans retain authority to decide whether, when, and how an AI system is used, including the ability to limit, suspend, or discontinue its use.

•  These are not interchangeable labels. They represent different oversight mechanisms with different implications for risk control and accountability.
• Human-in-the-loop is most appropriate where incorrect outputs could directly affect regulatory reporting, safety assessment, or patient protection. The defining feature is not the presence of a human, but the requirement that human judgement actively determines the outcome before decisions are finalized.
• Human-on-the-loop applies where AI systems operate with a lower level of immediate decision impact. In this model, humans do not intervene in every individual output but remain
  responsible for supervising performance, detecting drift, and escalating issues when behaviour deviates from expectations. This form of oversight is only effective when monitoring is robust
  and escalation pathways are clear.
• Human-in-command underpins all other forms of oversight. It establishes that responsibility for AI use sits with the organization, not the system. This includes decisions about scope of use,
  acceptable risk, and when continued operation is no longer justified. Without this authority, other oversight mechanisms lack enforceability.
• Selecting the appropriate form of oversight depends on how much influence the AI system has on pharmacovigilance decisions and the consequences of error. Choosing an oversight model that is misaligned with actual system impact can create a false sense of control while leaving material risks unmanaged.
• For Heads of Safety, the key question is not whether one of these oversight models exists, but whether the chosen model is appropriate for the use case and defensible given its impact.

 

Black Swan” Events in AI
A “black swan” event refers to a rare and unexpected failure with disproportionately high impact.
In pharmacovigilance, even infrequent AI errors can matter if consequences affect safety or compliance, which is why average performance metrics alone are insufficient

3. Matching Oversight Models to Pharmacovigilance Use Cases

• Selecting an oversight model is not a theoretical exercise. It must be driven by how an AI system is actually used within pharmacovigilance workflows and by the consequences of error if that
  system fails. In practice, many AI-enabled pharmacovigilance activities influence decisions more directly than initially assumed.
• Case processing is a clear example. AI systems used to extract medical concepts, identify seriousness criteria, or prioritize cases can affect reporting timelines, downstream assessments,
  and escalation decisions. Where such influence exists, human-in-the-loop oversight is typically required, as it ensures that qualified human judgement actively determines outcomes before
  regulatory or safety-relevant decisions are finalized.
• Other use cases may require a different oversight approach, but only where the impact of error is demonstrably low. Examples may include AI systems used for workload balancing or visualization of trends, where outputs do not directly determine regulatory actions on a case-by-case basis.In these scenarios, human-on-the-loop oversight may be appropriate only where errors do not directly influence regulatory or safety decisions. Even then, this approach is acceptable only if
  system performance is continuously monitored, deviations are readily detectable, and clear escalation and intervention mechanisms are in place. The absence of human intervention at the
  level of individual outputs must be explicitly justified through risk assessment and supported by strong supervisory controls.
• Across all use cases, human-in-command oversight remains non-negotiable. The organisation must retain authority over whether an AI system is used at all, under what conditions, and with
  what limitations. This includes decisions to restrict scope, pause operation, or discontinue use when risks outweigh benefits. Human-in-command establishes accountability at the organisational level and ensures that oversight is enforceable rather than symbolic.
• A common failure is selecting an oversight model based on intended use rather than actual operational behaviour. Outputs that are reused downstream, inform prioritisation, or influence
  multiple steps in the workflow can elevate risk over time. Oversight models must therefore be revisited as workflows evolve and reliance on AI increases.
• For Heads of Safety, the key principle is alignment. Oversight must match real system influence, not convenience or vendor positioning. When oversight models are proportionate to impact,
  they function as genuine controls. When they are not, they create a false sense of security.

4. Why Human Review Alone Is Not Sufficient

• Including a human review step in an AI-enabled workflow does not, by itself, ensure effective oversight. In practice, human review can fail to mitigate risk if it is not deliberately designed to
  counter known behavioural effects associated with automated systems.
• One of the most significant risks is automation bias. When AI systems perform well most of the time, reviewers may unconsciously defer to their outputs, even when information is incomplete, ambiguous, or inconsistent. Over time, this deference can reduce critical scrutiny and lead to missed errors, particularly in high-volume environments where efficiency pressures exist.
• This risk is not eliminated simply by requiring a human to “check” outputs. If the reviewer’s role is limited to confirmation, lacks clear authority to challenge results, or is constrained by time or tooling, the presence of human review provides little additional protection. In such cases, human involvement can create the appearance of control without materially changing outcomes.

 

Human vs Machine Roles

Machines should perform tasks they can execute reliably and at scale.
Humans should retain tasks requiring judgement, context, and accountability.

 

• Effective oversight therefore requires more than procedural review. It requires clarity on what the human reviewer is expected to evaluate, when escalation is required, and how
  disagreement with AI outputs is resolved. Reviewers must be supported with appropriate training, decision authority, and system transparency to enable meaningful intervention.
• For pharmacovigilance organisations, this has a practical implication. Oversight models must be designed with an understanding of how humans actually interact with AI systems in real
  workflows. Controls that do not account for automation bias risk becoming ineffective over time, even if they appear robust on paper.
• Human oversight is intended to reduce risk, not to legitimise automation. Where human review is treated as a formality rather than an active control, it fails to serve its purpose and can leave
  critical risks unmanaged.

5. Leadership Accountability and Lifecycle Oversight

• Effective human oversight of AI-enabled pharmacovigilance systems cannot be delegated solely to technical teams or embedded within operational procedures. Decisions about how AI is used, controlled, and monitored are leadership responsibilities.
• Senior pharmacovigilance leadership must understand how AI systems are deployed in practice, how their outputs influence decisions, and where errors could have impact. Reliance on vendor descriptions, average performance metrics, or intended-use statements is insufficient if they do not reflect actual workflow behaviour and decision influence.
• Oversight is not a one-time design decision. AI systems evolve, workflows change, and reliance can increase gradually over time. As systems become more embedded, their influence on safety
  and regulatory outcomes may expand beyond the original scope. Leadership oversight must therefore extend across the full lifecycle of AI use, from initial risk assessment and deployment
  through ongoing monitoring, change management, and, where necessary, decommissioning.
• This lifecycle perspective also applies to human oversight models. A model that is appropriate at introduction may become insufficient as system performance improves, volumes increase, or
  outputs are reused downstream. Regular reassessment is required to ensure that oversight remains proportionate to actual risk and influence.
• Accountability for AI use in pharmacovigilance ultimately rests with the organisation. Human-in-
  command oversight ensures that responsibility for risk acceptance, escalation, and continued use remains clearly assigned and enforceable. Without this, other oversight mechanisms lack
  authority and effectiveness.
• For Heads of Safety, the implication is clear. AI does not reduce responsibility by automating tasks. As its role within pharmacovigilance systems expands, leadership accountability becomes
  more concentrated, not less.

Artificial intelligence in pharmacovigilance should not be assessed by how advanced the technology is. It should be assessed by what happens when the technology is wrong.

Executive Summary

• Artificial intelligence is increasingly being applied across pharmacovigilance activities, particularly in case processing, triage, and assessment. While these technologies are often introduced to improve efficiency and scalability, their use raises important questions about risk, oversight, and accountability.
• In a GxP-regulated environment, AI systems used in pharmacovigilance must meet regulatory expectations for risk management, oversight, and accountability.
• It is important to develop a risk-based perspective on the use of artificial intelligence in pharmacovigilance. AI systems should not be evaluated as standalone productivity tools, but as components of the pharmacovigilance system whose outputs can materially influence regulatory decisions and patient safety outcomes.
• Risk in AI-enabled pharmacovigilance is defined not by algorithmic complexity, but by two factors:

• Degree to which AI outputs influence decisions &
• Consequences of incorrect outputs.

Even technically simple systems may represent high risk when their outputs affect reporting obligations, safety assessments, or downstream signal detection activities.

• We should understand the known and foreseeable failure modes, including false negatives, automation bias, and the compounding effect of downstream reuse of AI outputs. It is important to know why human oversight must be deliberate and proportionate to risk, and why human-in-the-loop models alone are insufficient without clear accountability and safeguards against over-reliance.
• Finally, emphasis should be placed on the role of safety leadership. Classification of AI risk, design of oversight models, and continuous monitoring are leadership responsibilities that cannot be delegated to technology providers or implementation teams. As AI systems become more influential within pharmacovigilance workflows, accountability becomes more concentrated, not diluted.

Organizations should consider applying a structured, inspection-ready approach to AI in pharmacovigilance, grounded in risk awareness, transparency, and patient safety.

Why AI in Pharmacovigilance Must Be Treated as a Risk-Based System

• Artificial intelligence in pharmacovigilance should not be assessed by how advanced the technology is. It should be assessed by what happens when the technology is wrong.
• This is a deliberate position. AI used in pharmacovigilance must be evaluated primarily through the lens of risk, not innovation.
• Risk is driven by two factors:

1.Consequence of an incorrect output &

2.Degree to which the AI system influences pharmacovigilance decisions.

Sophistication or novelty does not reduce risk. Influence and impact define it.

• This framing matters for Heads of Safety. An AI system used for case processing, triage, or assessment is not evaluated as a productivity enhancement. It is evaluated as part of the pharmacovigilance system itself, with direct implications for patient safety, regulatory compliance, and organisational accountability.
• Risk assessment is therefore not a one-time exercise performed at deployment.

It must be applied throughout the lifecycle of AI use.

• This is why AI in pharmacovigilance must be treated as a risk-based system, not a tool.

How Risk Is Defined in AI-Enabled Pharmacovigilance

• Risk in AI-enabled pharmacovigilance is not determined by how complex the technology appears to be. It is determined by how much influence the AI system has on decisions and the consequence of an incorrect decision.
• This distinction is critical. An AI system performing a technically simple task can represent high risk if its output directly affects regulatory reporting, medical judgement, or patient safety. Conversely, a technically complex system may present lower risk if its outputs are clearly limited, well understood, and subject to effective human control.
• Risk increases as AI systems move closer to stand-alone operation. When outputs are consumed without meaningful challenge, or when they materially shape decisions without sufficient human intervention, the potential impact of error increases.

In pharmacovigilance, even small errors can alter the understanding of a medicine’s benefit-risk profile.

• Risk is also contextual. The same AI capability can fall into different risk categories depending on how and where it is used. An extraction model supporting case intake may be lower risk when outputs are fully reviewed, but higher risk when those outputs are reused downstream to drive assessments or reporting decisions.
• For Heads of Safety, this means risk classification is not a technical exercise delegated to vendors or data scientists. It is a business and safety decision that must consider real workflow influence, error detection mechanisms, and actual reliance on the system in practice. This understanding underpins expectations for oversight, monitoring, documentation, and governance.

Explicit Risks That Must Be Anticipated

• The risks associated with AI in pharmacovigilance are not theoretical. Specific failure modes are well understood and particularly relevant to safety-critical activities.

One of the most significant risks is false negatives. AI systems may fail to identify relevant adverse events, seriousness criteria, or safety signals, even when overall performance metrics appear acceptable. In pharmacovigilance, these failures are not benign. Missed information can delay reporting, distort safety assessments, and undermine patient protection.

• Automation bias represents another important risk. When AI systems are perceived as reliable, human reviewers may place undue trust in their outputs. This can lead to reduced vigilance, insufficient challenge, and a gradual erosion of critical review. Human review alone does not eliminate this risk. Without deliberate safeguards, human involvement can become procedural rather than protective.
• A further risk arises from downstream reuse of AI outputs. Information generated early in the workflow is often reused in later steps such as assessment, aggregation, or signal detection. As outputs travel further through the pharmacovigilance process, their influence increases and errors can compound, even if the original use case appeared limited.
• These risks increase whenever AI outputs influence regulatory or clinical judgement. High average accuracy or good intentions are not sufficient safeguards. Known failure modes must be anticipated and actively mitigated.
• For Heads of Safety, the implication is clear. These risks are foreseeable. Failure to address them cannot be justified as unexpected system behaviour.

What This Means for Case Processing and PV Use Cases

• Many AI use cases in pharmacovigilance are introduced as operational support. In practice, their influence often extends far beyond clerical assistance. Case processing is a clear example. AI systems used to extract medical concepts, identify seriousness criteria, or prioritise cases directly influence regulatory timelines and downstream safety evaluation. Even when described as assistive, these outputs frequently shape what is reviewed, what is escalated, and what is ultimately reported.
• Errors in these contexts may be infrequent but consequential. A missed hospitalization, a misinterpreted narrative, or an overlooked adverse event may not significantly affect aggregate metrics, but it can materially affect patient protection and compliance.
• Another compounding factor is downstream reuse. Risk accumulates not because the algorithm changes, but because its outputs travel further through the pharmacovigilance process.
• As a result, many AI-supported case processing activities sit higher on the risk spectrum than initially assumed. Classification cannot be based on convenience.

It must be based on influence and consequence.

Human-in-the-Loop and Proportionate Oversight

• Human oversight is not symbolic review. It is an active control designed to mitigate known risks.
• One important oversight model is human-in-the-loop. In this approach, AI outputs are reviewed and either accepted, modified, or rejected by a qualified human before they influence decisions.
• Human review alone, however, does not guarantee accuracy. Automation bias can undermine oversight when reviewers unconsciously defer to AI outputs. Without deliberate safeguards, human-in-the-loop can degrade into human-on-paper.

Oversight must therefore be proportionate to risk. As AI influence increases, so must the clarity of reviewer responsibility, the depth of review, and the supporting quality controls. Oversight models must account for human behaviour, not just system performance.

What This Means for Safety Leadership and Accountability

• Risk classification of AI use in pharmacovigilance is not a technical decision. It is a leadership responsibility.
• Safety leadership must understand how AI systems are actually used, how outputs influence decisions, and where errors could have impact. Vendor descriptions, average accuracy, or intended use statements are insufficient if they do not reflect real operational behaviour.
• Risk assessment is not static. AI systems evolve, workflows change, and reliance can increase over time. Continuous review, monitoring, and escalation mechanisms are therefore essential.
• This is why expectations around human oversight, performance monitoring, and governance exist. They are not generic best practices. They are responses to known and foreseeable risks.
• AI does not dilute accountability. It concentrates it.

Artificial intelligence will increasingly shape pharmacovigilance activities. The risks are not hypothetical, and ignoring them is not an option.

 

In subsequent posts, we will explore how these risks can be mitigated in practice, including the role of human-in-the-loop oversight, monitoring, and governance.

Ever faced a hepatic signal in a trial that looked like classic drug-induced liver injury (DILI), but turned out to be something entirely different?

Here’s a hypothetical scenario that might resonate, where hepatic physiology,
herbal pharmacology, and cultural practices intersected to produce a pseudosignal that mimicked hepatotoxicity.

A Complex Signal in an Early Phase Trial

During an early Phase trial of a novel drug, one site reported liver enzyme elevations in three participants from the high-dose cohort:

• ALT 2–3 × ULN
• Bilirubin: mildly elevated (1.2 × ULN) in one case
• No clinical symptoms, no jaundice, fatigue, or right upper quadrant pain.

It looked like hepatocellular injury, potentially qualifying as a Hy’s Law signal. But something didn’t add up.

The Hidden Variable: Herbal Co-Exposure

Upon deeper review of case narratives and follow-up interviews, the medical monitor discovered that the participants had been consuming traditional herbal decoctions:

• Andrographis paniculata
• Phyllanthus amarus

These were considered “health tonics” in the local culture and were not disclosed during standard medication history. Their omission was unintentional, these herbs were simply part of daily life.

Why Would These Matter?

Both herbs are pharmacologically active:

• Andrographolide, the key compound in Andrographis paniculata, is a known CYP3A4 inhibitor
• Phyllanthin and hypophyllanthin, from Phyllanthus amarus, inhibit CYP3A4, CYP2C9, and potentially conjugation enzymes like UGT

The mechanism likely involved:

• CYP enzyme inhibition
• Impaired clearance of the parent drug
• Accumulation of reactive intermediates
• Oxidative and mitochondrial stress in hepatocytes
• ALT/AST leakage

This wasn’t classic immune-mediated DILI, it was a pharmacokinetically driven pseudo-DILI, shaped by metabolic interference from herbal products.

Pharmacological Evidence (CYP Modulation) Andrographis paniculata

• Primary active compound: Andrographolide

Pharmacological Evidence (CYP Modulation) Andrographis paniculata

• Mechanism: Inhibits CYP3A4 activity (in vitro studies)

Reference:
Elza Sundhani, Endang Lukitaningsih, Arief Nurrochmad, Agung Endro Nugroho. Potential pharmacokinetic and pharmacodynamic herb-drug interactions of Andrographis paniculata (Burm. f.) and andrographolide:
A systematic review.

Phyllanthus amarus

• Active compounds: Phyllanthin, hypophyllanthin
• Mechanism: Inhibits CYP3A4 and CYP2C9

Reference:
Taesotikul T, Dumrongsakulchai W, Wattanachai N, Navinpipat V, Somanabandhu A, Tassaneeyakul W, Tassaneeyakul W. Inhibitory effects of Phyllanthus amarus and its major lignans on human microsomal cytochrome P450 activities: evidence for CYP3A4 mechanism-based inhibition. Drug Metab Pharmacokinet. 2011;26(2):154-61. doi: 10.2133/dmpk.dmpk-10-rg-107. Epub 2010 Dec 17. PMID: 21178301.

Medical Monitoring Strategy

Once the pattern became clear, the safety team responded rapidly and decisively:
Pharmacokinetic (PK) Substudy
Goal: Determine if herbal co-use altered systemic exposure and metabolite profiles

• Used stored plasma samples
• Compared AUC, Cmax, and metabolite-to-parent ratios between herbalexposed vs. unexposed participants
• Analysis conducted using non-compartmental modeling

Key Results:

• Significant increase in AUC of parent compound in herbal users
• Strikingly higher metabolite-to-parent ratio
• Modestly extended half-life

Conclusion: Exposure was exaggerated due to impaired metabolism, likely contributing to hepatic stress, but not direct hepatotoxicity.

In Vitro Microsome Assay
Goal: Test direct inhibition of hepatic CYP enzymes by the herbal extracts

• Human liver microsomes used (pooled donor panels)
• Co-incubated with midazolam and diclofenac to probe CYP3A4 and 2C9 activity
• Enzyme kinetics analyzed via LC-MS/MS

Key Results:

• Andrographis extract inhibited CYP3A4 activity
• Phyllanthus extract inhibited both CYP3A4 and CYP2C9
• No CYP induction observed in PXR-based reporter assays

Outcome and Protocol Response

Following the findings:

• Dose escalation was paused temporarily
• CRF and ICF were updated to include herbal/traditional supplement disclosure
• A 14-day washout period was implemented for known hepatic modulators
• ALT/AST/bilirubin monitoring was intensified on Days 4, 7, 10, and 14

After herbal intake was discontinued, enzyme levels normalized within 10 days. No further cases were reported. The trial resumed with stronger hepatic safety controls in place.

Still Reflecting….

Even now, this scenario raises important questions:

• Are we doing enough to proactively uncover culturally “invisible” exposures in global trials?
• Are PK/PD insights being used early enough to differentiate real signals from pseudo-DILI?
• Are we integrating hepatic physiology, enzyme modulation, and cultural pharmacology into our signal detection framework?

This wasn’t just a pharmacovigilance issue. It was a clinical, cultural, and mechanistic puzzle; one that we solved by looking beyond lab values.

Let’s Keep the Conversation Going!

• Have you managed hepatic safety signals that turned out to be driven by cultural or dietary confounders?
• How do you screen for herb-drug interactions in early-phase settings?
• Let’s share ideas — because the more perspectives we hear, the better we can safeguard participants and protect the integrity of our science.

Why Medication Errors Matter More Than We Think

Medication errors are a significant public health concern, often occurring during various stages of care, whether preventive, diagnostic, therapeutic, or rehabilitative. These errors not only affect patient outcomes but also highlight critical gaps in the safe use of medicinal products. There is a growing need to strengthen risk mitigation strategies and enhance prevention efforts through existing regulatory mechanisms. Beyond the clinical impact, medication-related harm also places a substantial financial burden on healthcare systems globally, with associated costs estimated at around US $42 billion each year. Medication errors are unintended mistakes that can happen at any stage ofthe medication process, whether it’s during prescribing, storing, dispensing, preparation, or administration. When such errors occur repeatedly, follow a recognizable pattern, or lead to serious patient harm, it becomes critical to investigate the root causes and contributing factors. Understanding the clinical impact of these incidents, along with identifying practical solutions and preventive strategies, is key to ensuring they do notrecur.

Stages of Medication Use Process

1. 1. Storage
   2. Prescribing Stage
   3. Transcribing Stage
   4. Preparation Stage
   5. Dispensing Stage
   6. Administration Stage
   7. Monitoring Stage

What is a Medication Error?

A medication error is an unintended failure in the drug treatment process that leads to, or has the potential to lead to, harm to the patient.

Adverse Event:
GVP Annex I (Rev 3) defines an adverse event as any untoward medical occurrence in a patient or clinical trial subject administered a medicinal product and which does not necessarily have a causal relationship with this treatment. An adverse event can therefore be any unfavorable and unintended sign (including an abnormal laboratory finding, for example), symptom, or disease temporally associated with the use of a medicinal product, whether or not considered related to the medicinal product. Medication related adverse events should be distinguished from other adverse events (e.g. fall, surgery on wrong body site etc.).

Adverse Reaction:

An adverse reaction (ADR) is a response to a medicinal product which is noxious and unintended (Directive 2001/83/EC, Article 1 (11)). This includes adverse reactions which arise from:

• the use of a medicinal product within the terms of the marketing authorization.
• the use outside the terms of the marketing authorization, including overdose, off label use, misuse, abuse and medication errors;
• occupational exposure.

Patient Safety Incident

WHO’s Conceptual Framework for International Classification for Patient Safety (WHO ICPS) defines a patient safety incident as an event or circumstance that could have resulted, or did result, in unnecessary harm to a patient. The scope of patient safety incidents covers the entire health care process whereas the scope of (suspected) adverse reactions in pharmacovigilance is limited to the use of medicines by a consumer or healthcare professional. Patient safety incidents may occur in hospitals or other health care communities and may or may not involve a medicinal product.

Correlation Among Medication Errors, Preventable and Non-Preventable Adverse Reactions, and Intercepted Errors

The diagram is provided for illustrative purposes only to support understanding of medication errors in the context of patient safety, and is not intended to inform or replace pharmacovigilance reporting obligations.

 

Classification of Medication Errors Reports

To support effective recording, coding, reporting, and assessment, medication errors should be classified based on factual information specific to each case.
Itis important to clearly distinguish between:

• Medication errors associated with adverse reaction(s)
• Medication errors without harm
• Intercepted medication errors
• Potential medication errors

The classification depends on where the break occurs in the sequence of events leading to the error and the resulting consequences forthe patient, as illustrated below:

Ref: European Medicines Agency Good practice guide on recording, coding, reporting and assessment of medication errors. EMA/762563/2014

 

Intercepted Medication errors (Near Miss)
An intercepted error indicates that an intervention caused a break in the chain of events in the treatment process before reaching the patient which would have resulted in a ‘potential’ ADR. The intervention has prevented actual harm being caused to the patient. A near miss from a patient safety perspective is a random break in the chain of events leading up to a potential adverse event which has prevented injury, damage, illness or harm, but the potential for harm was nonetheless very near.
Example: A wrongly prepared medicine is intercepted by a nurse before being administered.

 

Potential medication errors
The recognition of circumstances that could lead to a medication error, and may or may not involve a patient. Refers to all possible mistakes in the prescribing, storing, dispensing, preparation for administration or administration of a medicinal product by all persons who are involved in the medication process and may lead to: a) medication error with harm, but without knowing the actual cause, b) medication error without harm and without knowing the actual cause, or c) medication error without harm, but with the awareness ofthe actual cause.
Example: Pharmacist noticed that the names of two medicines are similar and could clearly lead to product name confusion in practice, but no patient was actually involved or has taken the medicine.(Scenario c)

 

Medication Errors with Harm (Adverse Reactions)Medication errors that result in harm to the patient, specifically those associated with one or more adverse reactions. These cases also involve preventability.
Example: A patient receives an incorrect dose leading to hypotension and hospitalization.

 

Medication Errors without harm

NCC MERP Index for Categorizing Medication Errors

Ref:©2025 National Coordinating Council for Medication Error Reporting and Prevention. All Rights Reserved. *Permission is here by granted to reproduce information contained here in provided that such reproduction shall not modify the text and shall include the copyright notice appearing on the pages from which it was copied.
This copyright statement will change to the new year after the 1st of every year