Interictal discharge detection on routine scalp EEG
A pipeline that reads a routine 10–20 scalp EEG and drafts the sentence a neurologist would write: whether interictal epileptiform discharges are present, where they project, and whether sleep brings them out.
ProblemWhy this question
A routine EEG is roughly twenty minutes of ongoing rhythm in which a handful of sub-second events may or may not appear. The clinically decisive ones are interictal epileptiform discharges (spikes, sharp waves, spike-and-wave, polyspikes) and whether they are there changes what happens to the patient. They are also easy to miss and, worse, easy to over-read: normal sharp transients look a great deal like the real thing.
So the tool has two jobs. Detection: does this recording contain discharges at all. Description: for the discharges found, their spatial projection, their morphology, and whether sleep activates them, assembled into a sentence such as “interictal epileptiform abnormalities present: sharp-and-slow-wave complexes with left temporal projection, activated during sleep.”
PipelineMethods
- Stage 1, detection. Gradient boosting (400 trees, depth 3) over 26 interpretable signal features, computed on one electrode at a time and aggregated by taking the maximum across the 19. Sharpness dominates the ranking, which is what a neurologist would predict: a spike is defined by how abruptly it rises, not by how big it is. Scoring per channel also names the electrode for free.
- Stage 1b, hard-negative discriminator. A second filter answering only the hard question: given something that already looks like a discharge, is it real? Trained on 32,433 expert-voted events with vote-weighted soft targets, plus 21,833 hard negatives mined from 4,412 control recordings: the windows the previous model itself scored highest, i.e. its own false positives.
- Stage 2, localization and state. Spatial class over five regions, plus wake/sleep.
- Stage 3, draft description. Template-based and traceable to the events that triggered it, so a reader can audit any sentence back to the epochs behind it.
- Higher-order interactions. Three of stage 1’s 26 features are couplings rather than shapes: bicoherence at two band pairs, and phase-amplitude coupling between the 2-6 Hz phase and the 20-70 Hz amplitude. They ask whether the rhythms of one electrode are locked to each other, which no measure of amplitude or sharpness can ask. Where they are computed turned out to matter more than whether: as extra columns on a whole-segment vector they were worth +0.007 ROC-AUC and were dropped, while the same estimators computed one electrode at a time carry PR-AUC +0.034, per-recording ROC-AUC +0.013 and channel localisation +0.025. They are used in stage 1 only. Stage 1b is a convolutional network on the raw signal and has no engineered features.
ResultsDetection, internal
5-fold subject-level cross-validation, out-of-fold over all 84 recordings. Per-recording, the question a clinician actually asks. Spreads are the standard deviation across the five folds, which is what moves when the patients change:
Per segment the detector reaches sensitivity 0.765 at a 10% false-alarm ceiling, with PR-AUC 0.707 pooled and 0.669 ± 0.183 across folds, a spread from 0.42 to 0.85 that is the honest measure of how much 84 recordings can settle. On the pediatric subset (32 patients, age ≤ 18) every measure improves: ROC-AUC 0.957 against 0.907 for the full cohort, sensitivity 0.911 against 0.765. That is the encouraging finding for a pediatric deployment, with the caveat that the subset holds only 10 discharge-free recordings, too few to calibrate a recording-level threshold on.
Confusion matrix, per recording
Out-of-fold over all 84 recordings: 52 with expert-annotated discharges, 32 without. Counts lead and the share of each row follows in brackets, so the top row reads as sensitivity and the bottom as specificity. Read the counts first: on a corpus this size one recording is worth 0.019 of sensitivity, and a percentage suggests a precision the corpus does not have.
The ten missed recordings are the number that matters for a tool meant to catch what a less experienced reader might overlook. The operating point behind this table maximises Youden’s J, which treats a missed recording and a false alarm as equally costly. Here they are not, and choosing that exchange rate is a clinical judgement still to be made.
The tradeoff that decides usability
A single operating point hides the shape of the curve, and the shape is what determines whether anyone would tolerate the alarms:
| Constraint | Result |
|---|---|
| FPR ≤ 0.05 | sensitivity 0.686 |
| FPR ≤ 0.10 | sensitivity 0.765 ← configured operating point |
| sensitivity ≥ 0.90 | costs FPR 0.324 |
Reaching 0.90 sensitivity costs a 32% false-positive rate. A reader rejecting a third of all flags stops trusting the tool, so the pipeline fixes the ceiling at FPR 0.10 and states the shortfall against the target rather than quietly relaxing it.
ResultsExternal validation, the number that matters
Trained on all of the internal corpus, tested on 100 segments from a different country, hospital and recording system, against a reference standard of video-EEG of habitual events. Every negative in that set is itself a sharp transient, a clinical lookalike, selected precisely because such waveforms cause EEG over-reading.
| ROC-AUC | Accuracy | |
|---|---|---|
| Stage 1 alone | 0.663 | 0.630 |
| Stage 1 + 1b, adjudicated events only | 0.790 | 0.740 |
| Stage 1 + 1b, with mined negatives | 0.888 | 0.800 |
| Human experts, same 100 cases | not published | 0.920 |
The model is still worse than a human here, and that gap is the honest headline. The hard-negative stage moved external ROC-AUC from 0.663 to 0.790 on adjudicated events alone, and to 0.888 once negatives mined from control recordings were added; expert scoring on the same cases is 0.920. The deficit narrowed by roughly a third. It did not close.
The internal and external numbers are not in conflict: they ask different questions. Internally most negatives are ordinary background, and separating a spike from resting rhythm is the easy problem. Externally every negative already looks like a discharge, and separating those is the hard one. The consequence is stated plainly in the project: the current model is not ready to pre-screen unselected recordings, and the priority is more hard negatives, not more architecture.
On the in-domain held-out set the same pattern shows up honestly: 0.937 ROC-AUC on clear-cut events, 0.709 on the borderline ones where the expert panel itself disagreed. Those cases are genuinely hard, not merely unlearned, and the mined negatives barely moved them.
ResultsDescription and localization
Draft descriptions are scored by structured-field agreement against expert annotation, not by text-overlap metrics: a sentence can score well on BLEU while naming the wrong lobe.
State is solid; localization is the weakest part of the pipeline. Spatial class is essentially a per-patient property (33 of 52 discharge-positive recordings carry exactly one class) so the effective sample size is patients, not epochs. The centro-parietal class rests on four patients. The honest conclusion is that this corpus is too small to learn five-way localization reliably, whatever the epoch counts suggest.
DemoSee it run
An exported, self-contained page: pick one of three held-out recordings, watch detection, localization and description run, and see where the pipeline looked. Every figure on it is real output from the committed code.
Open the interactive demoOpens as its own page: it is the pipeline’s own export, served unmodified rather than re-skinned.
NextWhere this goes
Phase 2 is the pediatric hospital setting the whole project is aimed at, and it does not start until the lookalike problem is under control: a tool meeting routine EEGs will meet sharp mimics constantly, because that is what routine EEG is full of.
One finding from phase 1 is worth carrying forward on its own: an amplitude-normalization bug once made external performance worse than baseline, because per-corpus scaling let outlier events shrink new recordings 28-fold. Acquisition systems differ in absolute microvolts, so any model that depends on them will not transfer. It is pinned by a test now.
Talk to us about this work