Six Things I Keep Coming Back To in Clinical Signalling

The mechanics of signal detection are familiar enough. A disproportionality score crosses a threshold, a case series is reviewed,an action is taken. Underneath the mechanics sits the actual work: a
series of judgment calls that no dashboard captures. The following are a few themes I keep coming back to in this work.

1. Inputs are not signals

The following are not signals:

• A disproportionality alert crossing a numerical threshold
• A cluster of ICSRs reporting a similar event
• A literature hit on a class effect

These are inputs. A signal is the output of clinical evaluation applied to those inputs, weighted against biological plausibility, exposure context, background incidence, indication, comparator behaviour, and the totality of available evidence. Conflating inputs with signals produces either over-escalation (noise overwhelms the system) or under-escalation (weak but real patterns are dismissed on statistical grounds).

2. Standardised thresholds are operationally convenient, scientifically limited

Most signal management SOPs apply uniform numerical cut-offs across the portfolio. The same disproportionality threshold, the same case count minimum, the same statistical significance criterion. Identical numbers can carry very different clinical weight depending on the exposure context behind them.
Example. Two products generate three cases of the same serious event:

• Product A: paediatric biologic, recently launched, limited cumulative exposure. Three cases of severe neutropenia against a small denominator represent a non-trivial proportion of exposed patients. The safety database is immature, comparator data are sparse, and the event is clinically serious. The cluster warrants substantive evaluation regardless of where it falls on a statistical threshold.
• Product B: small molecule, 40 years post-launch, tens of millions of exposures, well-characterised safety profile. Three cases of the same event against that denominator are very likely noise, stimulated reporting, or media-driven case generation. Statistical signals on mature products with known profiles frequently re- surface long-established reactions rather than novel risks Threshold calibration by product maturity, exposure base, indication seriousness, and event severity is therefore not optional refinement. It is a precondition for the methods producing meaningful output.

3. Stratified analysis is the analysis

Primary screens on pooled data routinely mask clinically important subgroup effects. Relevant stratification variables include:

• Age
• Gender
• Indication
• Concomitant medications
• Dose and regimen
• Time on therapy
• Renal and hepatic function
• Formulation and route of administration
• Geography and reporting system

Example. A product is approved for two indications: mild hypertension (100,000 exposed patients) and a rare severe condition (1,000 exposed
patients). Acute kidney injury is the event of interest.

• Hypertension cohort: 20 AKI cases in 100,000 patients. Crude reporting rate of 0.02%. Consistent with expected background.
• Rare condition cohort: 15 AKI cases in 1,000 patients. Crude reporting rate of 1.5%, approximately 75 times higher than the hypertension cohort and substantially above plausible background.
• Pooled analysis: 35 cases in 101,000 patients. Crude reporting rate of 0.035%. Unremarkable on its face.

Without stratification, the second cohort signal is invisible. The label remains unchanged. The risk in the smaller, sicker population persists undetected. Stratified analysis is not a follow-up step after the primary screen. The primary screen, run on pooled data alone, is structurally incapable of surfacing this category of signal.

4. The case narrative remains the most diagnostic instrument

Quantitative methods receive most of the methodological attention, but the individual case narrative continues to carry decisive evidential weight. Relevant elements include:

• Temporal relationship between exposure and event
• Dechallenge information
• Rechallenge information where available
• Alternative aetiologies and confounders
• Reporter's clinical reasoning and differential

The components of signal evaluation that have not been automated are the components that resist automation. Narrative review is the principal example.

5. Confounding by indication is persistent

Patients receive medicines because they are unwell. The underlying condition can produce events that resemble adverse reactions. Common examples:

• Thromboembolic events in oncology populations
• Serious infections in immunology populations
• Cardiovascular events in diabetes populations
• Hepatic events in hepatology populations

The diagnostic question is what the expected event rate would be in the absence of any drug effect. Without an answer to that question, attribution is unreliable regardless of the statistical output.

6. Communication of graded confidence is underdeveloped

A signal is rarely binary. Strength of evidence accumulates, shifts, and occasionally reverses. Cross-functional decision-makers require a vocabulary richer than confirmed or refuted. Relevant dimensions include:

• Strength and consistency of evidence across data sources
• Biological plausibility
• Specificity of the association
• Dose-response relationship
• Public health relevance
• Reversibility on discontinuation

Collapsing these dimensions into a single binary output understates what the function is actually producing and weakens the basis for downstream decisions on labelling, risk minimisation, and benefit-risk reassessment.

Closing observation

Signal detection methods will continue to improve. What does most of the work in signal management is clinical judgment: deciding what threshold is meaningful for a specific product, how to stratify the data, what a case narrative is really telling you, whether the indication is driving the event rather than the drug, and how to communicate uncertainty without forcing it into a binary.