Summary of AI photo detector accuracy
This report covers the detection model only, the part of the AI photo detector that judges pixels when an image has no Content Credentials or generator metadata. When those labels are present they decide the verdict, and they are close to certain. Here we test the hard case: a bare image with nothing but pixels.
Every image lands in one of the three verdict bands the site uses: AI (AI score 80 or more), Not sure (between 25 and 80) or Real (25 or less). The AI score is a rescaled model output; on the raw output these bands sit at 0.3 and 0.02. We chose them so that only about 1 in 100 real photos falls in the AI band.
| Images | Condition | Called AI | Not sure | Called real |
|---|---|---|---|---|
| Real photos (400) | Original file | 1% | 3% | 96% |
| Social-media copy | 2% | 6% | 93% | |
| AI images (615) | Original file | 43% | 20% | 37% |
| Social-media copy | 32% | 25% | 43% |
In plain words: the model rarely calls a real photo AI, but it misses many images from current commercial generators. A "Likely a real photo" result from the model alone is therefore weak evidence. That is why the site shows the evidence lines separately and why a "real" verdict always comes with a warning.
Results by generator
Share of each generator's images the model called AI and called real (the rest were "Not sure"). "Original" is the file as the generator produced it, with metadata removed so the model has to decide alone. "Social copy" simulates a repost: resized to 1080 pixels on the long side and saved as JPEG at quality 75.
| Generator | Images | Original | Social copy | ||
|---|---|---|---|---|---|
| AI | Real | AI | Real | ||
| Grok AuroraxAI | 22 | 95% | 5% | 82% | 5% |
| Ideogram 2.0Ideogram | 19 | 84% | 5% | 26% | 32% |
| Illustrious XL (anime SDXL)open | 50 | 82% | 0% | 86% | 2% |
| Lumina Image 2open | 32 | 81% | 6% | 56% | 16% |
| Recraft V2Recraft | 20 | 75% | 20% | 60% | 20% |
| Halfmoon (Reve)Reve | 14 | 71% | 14% | 43% | 29% |
| Midjourney v7Midjourney | 50 | 62% | 16% | 22% | 42% |
| Runway FramesRunway | 22 | 55% | 32% | 32% | 32% |
| ERNIE ImageBaidu | 31 | 48% | 32% | 29% | 29% |
| Recraft V3Recraft | 50 | 44% | 22% | 32% | 36% |
| ERNIE Image TurboBaidu | 50 | 28% | 54% | 30% | 52% |
| Z-Image TurboAlibaba Tongyi, open | 50 | 26% | 48% | 10% | 58% |
| GPT Image 1.5OpenAI | 50 | 26% | 44% | 24% | 54% |
| Seedream 5.0ByteDance | 33 | 18% | 61% | 27% | 36% |
| Sora 2OpenAI | 29 | 17% | 52% | 17% | 59% |
| GPT Image 2OpenAI | 41 | 10% | 68% | 7% | 73% |
| FLUX.2 [klein] 9BBlack Forest Labs, open | 50 | 2% | 92% | 2% | 94% |
| Nano Banana Pro (Gemini 3 Pro Image)Google | 2 | 0% | 0% | 0% | 0% |
Generators with fewer than 20 test images are shown for completeness, but their numbers can swing a lot with a few images.
Real photo sets
| Set | Images | Original: called real | Original: called AI | Social copy: called real | Social copy: called AI |
|---|---|---|---|---|---|
| ImageNet photos | 200 | 96% | 1% | 94% | 2% |
| DOCCI photos | 200 | 96% | 1% | 91% | 1% |
How we test
- Test set: OpenFake test split (ComplexDataLab, 2026 release), first shard, still images only. It contains recent commercial and open generators released between late 2024 and 2026, and real photos from ImageNet and DOCCI. None of these images were used to train the model.
- Same pipeline as the site: shortest side resized to 440 pixels, centre crop of 384×384, the same compressed model file the browser downloads.
- Two conditions: the original file, and a simulated social-media copy (downscale plus JPEG compression), because most images people check have been reposted.
- Fixed set: we rerun the same images for every model update, so results stay comparable over time.
What these numbers mean for you
- If the verdict is based on Content Credentials or metadata, it is far more reliable than anything in the tables above.
- If the model says AI, take it seriously: false alarms on real photos are rare in our tests.
- If the model says real or not sure, the image may still be AI, especially from newer commercial tools. Use the manual checks and a reverse image search.
Model and update log
2026-11-09: first public model, commfor-vits384-int8-2026.11. A ViT-Small detector from the Community Forensics project (CVPR 2025, MIT licence), quantised to 8-bit for the browser. Known gap: recent commercial generators. Next step: fine-tuning on images from current generators, with results posted here.