Somebody sends me a screenshot roughly once a fortnight. Same audio file, three different tools, three answers that cannot all be right. Usually the question attached is which one is correct.
Often none of them are correct in the way the question assumes, because the question assumes there is a number sitting inside the file waiting to be read out. There is not. Each tool is producing an estimate from a different model, trained on different data, calibrated differently, and reported through a different threshold. Disagreement is the expected outcome, not a malfunction.
That is not a comfortable thing for a detection vendor to write. It is still the accurate description, and understanding it will make you much better at reading these results.
What the number is
Start with what a detector actually computes.
The model produces a value that represents how strongly the audio resembles the synthetic examples it was trained on, relative to the genuine ones. That raw value is not a probability and does not naturally live on a 0 to 100 scale. It gets mapped onto one, and how it gets mapped is a design choice made by whoever built the tool.
So when you see 91 percent, that is not the probability that the clip is AI-generated in any strict sense. It is a model output that has been pushed through a transformation, and the transformation carries assumptions: about the mix of synthetic and genuine audio the tool expects to see, about how the vendor wants the scale to feel, about where the useful decision boundary sits.
Two tools can rank a set of clips in exactly the same order, agreeing perfectly about which files are more suspicious than which, and still print numbers that look wildly different. Ranking and calibration are separate properties, and most of the disagreement people notice is calibration disagreement wearing the costume of substantive disagreement.
Calibration is the part most tools get wrong
A well-calibrated detector is one where the numbers mean what they appear to mean. Of all the clips it scores at 80, roughly 80 percent should actually be synthetic.
This is harder than it sounds and a lot of tools do not attempt it. The easy path is to output whatever the model produces, scaled to look decisive, because a tool that says 94 percent feels more authoritative than one that says 61 percent. Users reward confidence. There is a commercial gravity pulling every vendor toward numbers that cluster near the extremes.
Our detector applies calibration and reports whether a given result was calibrated. I am not going to claim we have solved it, because calibration is only valid for the distribution you calibrated against, and real-world audio does not politely stay inside that distribution. But the difference between a tool that attempts calibration and one that does not is the difference between a number that means something and a number that is decoration.
A practical test: feed a tool a set of clips you know the answer to, including hard ones. If everything comes back above 90 or below 10, with almost nothing in the middle, the tool is not measuring confidence. It is performing it.
Different training data means different blind spots
Every detector was trained on a finite set of examples, and the composition of that set determines what it can see.
The public research lineage here comes from anti-spoofing work, particularly the ASVspoof challenge series, which produced shared datasets of genuine and attacked speech. Those datasets were built around the attacks that existed when they were assembled. A model that leaned heavily on one era of generation will be sharp against that era and progressively vaguer against what came after.
There is no way around this structurally. Detection is always chasing generation, and the lag varies by attack family in ways that are not visible from outside. Two tools trained a year apart, or on different mixes of attack types, will genuinely disagree about a clip from a family one of them saw plenty of and the other barely saw at all.
The same applies on the genuine side, and this is the part that gets less attention. If a model's human examples were mostly clean studio English, then accented speech, non-English speech, phone-quality recordings, and unusual voices are all out of distribution. Out of distribution does not produce a shrug from a model. It produces a confident answer based on nothing relevant, which is exactly what a false positive looks like from the inside.
The mistake we made, which is the clearest illustration I have
An earlier model of ours, internally textsight:voice-v1, scored ordinary human voice notes at 99 and 100 percent AI. Consistently.
The model was not malfunctioning. It had learned to separate speech from music rather than synthetic speech from genuine speech, because of what its training set actually contained. Compressed, close-miked phone audio fell on the wrong side of that boundary. It was answering a question with total confidence, and it was the wrong question.
We replaced it with the anti-spoofing architecture we run now. The reason I keep telling this story is that from the outside, that model looked fine. It returned decisive numbers. It had an interface. Nothing about the output revealed that the boundary it had learned was not the boundary anyone wanted.
Confidence is a property of a model's internal state. It is not a report on reality. Any detector can be confidently wrong, and none of them will tell you when they are.
The clip itself drives a lot of the variance
Beyond model differences, the material matters enormously, and it affects different tools differently.
Length. Under about ten seconds of continuous speech, there is simply less evidence, and tools differ in how they handle thin evidence. Some abstain. Some extrapolate. Two tools looking at eight seconds of audio can diverge purely on that policy choice.
Compression history. Detection evidence lives in fine spectral detail, and lossy codecs are engineered to discard detail that human hearing does not register. A clip that has been recorded, uploaded, re-encoded, downloaded and forwarded has had several passes of that. Models vary in how robust they are to it, and a chain of re-encodings can move a clip from clearly synthetic to genuinely ambiguous.
Post-processing. Noise reduction, normalisation and mastering all reshape the spectrum. Modern noise suppression in particular regenerates parts of the signal, which pushes genuine audio toward the artifact profile detectors look for.
Channel. Phone audio discards everything above roughly 3.4 kHz, where a lot of the evidence sits. Covered in more depth in detecting cloned voices on phone calls.
The general pattern: the harder the material, the more tools diverge. On a clean thirty-second uncompressed file most reasonable detectors agree. On a nine-second forwarded voice note they scatter, and the scattering is honest information about how little the file can support.
Base rates, and why a high score can still probably be wrong
This is the part that changes how you should read every result you get, and it has nothing to do with the model.
Suppose a detector is right 95 percent of the time in both directions. Good performance. Now run it across a thousand clips in a setting where ten of them are actually synthetic.
It correctly flags around nine or ten of the real ones. It also incorrectly flags about 50 of the 990 genuine ones. So of roughly 60 flagged clips, around 50 are wrong. A flag from a very good detector, in a low-prevalence setting, is more likely than not a false alarm.
Nothing about the tool is broken in that scenario. Rarity does the damage. This is the same arithmetic that makes rare-disease screening produce mostly false positives, and it is the single most important thing to internalise about detection results.
The practical consequence: a detection score means something quite different depending on what you already believed before you ran it. A high score on a clip that arrived anonymously, with stripped metadata, making an implausible claim, is meaningful confirmation. The same high score on a random clip from your own archive, with no other reason for suspicion, most likely is not.
Why we report uncertain as a verdict
Our detector returns an AI score from 0 to 100, a human score that is exactly its complement, and one of three verdicts: human, AI, or uncertain.
The uncertain verdict exists because there are two distinct situations that a two-way system collapses together. One is that the model looked and found the evidence genuinely mixed. The other is that the model could not properly look at all, because there was not enough usable speech in the clip.
We run a quality check before scoring. Too short, too quiet, no meaningful speech content, and the detector declines rather than guessing. That decline surfaces as uncertain, and it never gets rounded into human, which would be the dangerous default.
I would look for this in any tool you evaluate. Give it three seconds of a cough, a clip that is mostly silence, a heavily distorted fragment. If it produces a decisive percentage every time, it has no abstain path, which means some fraction of its confident answers are generated rather than measured, and you have no way to tell which ones.
How to read a result properly
Treat it as one input. Weigh it against provenance, corroboration, and context. The journalist verification workflow sets out a full order of operations, and detection sits fifth in it for good reason.
Read the direction more than the digits. The difference between 88 and 94 is mostly calibration noise. The difference between 88 and 22 is a finding.
Never treat a low score as proof of authenticity. It means no artifacts were found, and artifacts get destroyed by exactly the conditions most disputed audio has been through. It also says nothing about deceptive editing of genuine speech, which no detector catches because there is nothing synthetic to catch.
When tools disagree, look at the clip. Short, compressed, noisy, or narrowband? Then disagreement is the honest signal and the answer is that this file cannot bear the weight you want to put on it. Get better material or get a different kind of evidence.
Do not average the scores. Three numbers produced by three different calibrations do not combine into a meaningful fourth number. Agreement across tools is a signal. Their mean is not a measurement.
What I would want a vendor to tell you
If I were buying detection rather than building it, I would ask four things.
Does the tool ever abstain, and under what conditions? Is the score calibrated, and against what distribution? What does its false-positive rate look like on genuine human audio in the conditions I actually care about, with a published protocol rather than a claim? And does it attribute clips to named products, because if it does, I want to hear how it separates a product from every other one running similar architecture.
We answer those, including where the answer is unflattering. You can try our voice detector free without an account, on MP3, WAV, M4A, OGG or FLAC files up to 25 MB. Feed it something hard and see whether it admits the difficulty. That will tell you more than a marketing page will.
Try it on your own writing