AI-Generated Text Detection Implementation
AI text detection—arms race. Detection models trained on specific LLM texts, LLMs constantly evolve. No detector achieves 100% accuracy—fundamental task limitation.
How Detectors Work
Statistical methods (Perplexity, Burstiness):
- AI texts have low perplexity (predictable words)
- Low burstiness (uniform sentence length without "bursts")
- Implementation:
openai/detect-gptalgorithm, GPTZero method
Watermarking:
- At generation level, LLM embeds statistical pattern into token selection
- Detectable without source text
- Implementation:
extended_watermark_processor(John Kirchenbauer et al.) - Limitation: works only if generating LLM supports watermarking
Fine-tuned detectors:
-
roberta-base-openai-detector(OpenAI, trained on GPT-2) -
Hello-SimpleAI/chatgpt-detector-roberta(ChatGPT) - Problem: high false positive for neutral academic texts
Commercial APIs
- Originality.ai: specializes in SEO content, 97%+ accuracy claimed
- GPTZero API: widespread in education, supports Russian
- Sapling AI: corporate version
Limitations and Honesty
False positive rate for best detectors: 5–15% on human texts. Academic texts with formal style wrongly marked as AI-generated. Paraphrasing through another LLM bypasses most detectors. Use detection as one signal, not final judgment.