Home › Accuracy & Methodology

How we test our AI detector, honestly.

We don't publish a single headline accuracy number. Here's how we actually evaluate the detector, the accuracy bands we see, and the limits we're upfront about.

English-optimized · We prioritize a low false-positive rate · Verdicts are reported as confidence bands, not a single false-precision score.

Scope English long-form Priority Low false positives Output Confidence bands

Our position

We deliberately do not publish a single headline "accuracy" percentage. One number across every content length, genre, and language is misleading, and it's exactly the move overclaiming detectors make. Accuracy on a 1,000-word essay and accuracy on a two-sentence comment are not the same thing, so we won't pretend one figure covers both.

Plain English: AI detection is probabilistic. Treat any single verdict as one signal, not proof. No detector is 100% accurate, and anyone claiming otherwise is selling you a number, not a result.

What we test on

We evaluate against an internal, labelled test set that mixes three kinds of text:

The set rotates each release so the detector can't overfit to fixed examples, and we deliberately include second-language and non-native English academic prose (see False positives & ESL). The detector is English-only / English-optimized; scores on other languages aren't reliable. We have published our own false-positive measurement with the full dataset: see the benchmark. It is our own testing, not an independent evaluation, and we say so plainly there. An independent third-party benchmark remains something we want and do not yet have.

What the metrics mean

These are the concepts we track internally. We report them in-product as confidence bands rather than as a single false-precision number.

Accuracy

Correct verdicts (AI / human) divided by total documents. A "verdict" is the Authenticity Score thresholded at 50 (≥50 = human, <50 = AI).

False-positive rate (FPR)

Human docs wrongly flagged as AI, divided by total human docs. We optimize for FPR first, recall second. A wrong "AI" flag on real human writing damages trust badly, so we tune the model conservatively.

False-negative rate (FNR)

AI docs flagged as human, divided by total AI docs. We accept a higher tolerance here, because the cost of missing some AI is lower than the cost of a false accusation.

Precision

Of everything we flag as AI, the share that really is AI: true AI ÷ (true AI + false AI). High precision is what keeps a verdict trustworthy.

What our benchmark shows

Rather than a single number, we report accuracy in bands tied to content length, because that's where the real variation is. These are the same figures shown on our AI detector page:

We don't publish head-to-head tables of competitor scores. Numbers like that depend heavily on which test set, which model versions, and which day you run them, and presenting them as fixed facts would be exactly the kind of overclaiming this page exists to avoid. If you want to compare, run the same text through several detectors yourself.

These are TextSight's internal results, not a peer-reviewed external evaluation. Published academic work on detectors generally reports accuracy in the 80–95% range on curated sets, with sharper drops on adversarial or paraphrased content; our internal results sit within that broader band.

How the detector works

The Authenticity Score is a calibrated combination of several independent signals rather than a single model's guess. Broadly, it blends:

Combining signals, and requiring agreement between them, is what lets us keep the false-positive rate low instead of firing on any single suspicious cue.

False positives & second-language writing

The single worst thing an AI detector can do is wrongly accuse a human of using AI. We treat a low false-positive rate as our first objective, ahead of raw accuracy.

Second-language / ESL writing. Non-native English prose carries a documented false-positive risk across the entire AI-detection industry, not just here. A Stanford study (Liang et al., 2023) found that GPT detectors flagged 61.3% of non-native TOEFL essays as AI-written, while barely flagging native writing. We deliberately include second-language academic prose in training to reduce this bias, but no detector eliminates it. If you write in English as a second language, weight any single verdict accordingly.

Reference: Liang et al., "GPT detectors are biased against non-native English writers" (2023). arxiv.org/abs/2304.02819.

What we can't do (yet)

We stay honest about limits. No detector is infallible, and any single verdict is one signal, not proof. Here's what we know we don't do well.

How the model changes over time

We retrain on a rolling cadence as new generator models reach scale. A retrain is triggered either by a major upstream model release or by accumulated user-reported false positives crossing an internal threshold, and every retrain has to clear our pre-release benchmark before it ships.

Accuracy improves on average, but not monotonically: each retrain typically improves results on existing model families and can briefly regress on a brand-new generator until it's added to training. We treat AI detection as an ongoing arms race, not a solved problem.

Questions about our methodology? Or think we got a verdict wrong? research@textsight.ai · we read every email, and reported false positives become test-set additions.

Further reading

Run the test yourself.

Paste any text. We'll show you exactly what every classifier said.

Try the detector See sample report