AI writing took a major leap in late 2022 thanks to the release of the free ChatGPT language model.
ChatGPT is capable of producing content that looks as if it was written by a professional writer who is also an expert in the niche. ChatGPT writes emails, blog posts, code, and much more.
But this raises some questions about the originality and quality of the content.
To get an idea of how human-written your content looks like, you can use CrossPlag’s AI content detector.
Notice that this is not the same algorithm that Google uses to detect AI-written low-value content. So I wouldn’t necessarily use a tool like this to “validate” AI-written blog posts.
This is a comprehensive review of the CrossPlag AI detector. I’ve run some tests with both AI-written (ChatGPT) and human-written content. Besides, I try some simple tricks to try to fool CrossPlag’s AI detector.
đź’ˇ Make sure to read my comprehensive guide to the Best AI Content Detectors.
What Is the CrossPlag AI Detector?
CrossPlag offers an AI-detecting service that is still in the testing phase. Notice that this tool only works for English content!
The tool works by following these simple steps:
- Copy-paste text into the CrossPlag AI content detector.
- Wait for 1-2 seconds.
- See a score between 0-100%.
The reason why such an AI detector exists is to help content creators get an idea of how bot-like the content is. Once these tools become more accurate, they can be applied in detecting AI use in homework, and similar.
Performance
Let’s run some tests to see how the CrossPlag AI detector performs.
The idea of the following test is simple. I will:
- Input 10 human-written text samples to CrossPlag’s AI detector.
- Input 10 AI-written text samples to CrossPlag’s AI detector.
Based on the outcomes, I’ll calculate the accuracy and analyze the results.
Lastly, I’m going to perform three easy tricks with which I try to fool the AI detector.
Let’s get started!
1. Human-Generated Content
First, let’s input 10 human-written text samples into CrossPlag’s AI detector.
In an ideal world, CrossPlag should show a 0% AI Content Index for each of these samples.
Example 1
99% AI written—mission failed.
Example 2
99% AI written—mission failed.
Example 3
6% AI written—mission succeeded.
Example 4
11% AI written—mission succeeded.
Example 5
8% AI written—mission succeeded.
Example 6
84% AI written—mission failed.
Example 7
92% AI written—mission failed.
Example 8
100% AI written—mission failed.
Example 9
95% AI written—mission failed.
Example 10
72% AI written—mission failed.
As a result, CrossPlag only recognized 3/10 of the human-written articles as human-written. It falsely claims that 60% of my text is produced by AI.
But because we’re talking about an AI content detector, perhaps we shouldn’t use it to detect human-written content.
2. AI-Written Content
Let’s see how CrossPlag recognizes AI-written content from the 10 AI-written samples I give it. I’ve generated these text samples using ChatGPT.
Example 1
10% AI written—mission failed.
Example 2
100% AI written—mission succeeded.
Example 3
100% AI written—mission succeeded.
Example 4
88% AI written—mission succeeded.
Example 5
100% AI written—mission succeeded.
Example 6
100% AI written—mission succeeded.
Example 7
69% AI written—mission succeeded.
Example 8
100% AI written—mission succeeded.
Example 9
93% AI written—mission succeeded.
Example 10
98% AI written—mission succeeded.
In total, CrossPlag recognized 9/10 of the AI-written samples as AI-written. This is quite a good score, to be honest!
The tool clearly performs well when it needs to detect AI-written content. But it doesn’t really work when you’re using it to recognize human-written content.
If you know you’ve written the content with AI, this is not a big problem. But if you’re using CrossPlag to check the content that you don’t know in advance, you can’t really trust it.
Can You Trick the AI Content Detector?
Let’s see if we can trick the CrossPlag AI content detector to pass the AI content detection.
Test 1: Remove a Comma
➡️ TLDR; Introducing a small grammatical error didn’t affect the outcome of the AI detector much.
Let’s make an ever-so-slight change to the text by introducing a grammar mistake in it and see what happens.
I will remove the comma after the word “Additionally”:
Awesome! The tool didn’t worry about this change too much. It still believes the content is mostly written by AI—which is indeed the case.
The reason I wanted to test this is that some other AI content detectors went from 0% human written to 98% human-written by removing a single comma from the content.
Test 2: Make a Typo
➡️ TLDR; Removing one character from the text dropped the AI Content Index from 100% to 58%.
Removing a comma didn’t affect the AI detector’s outcome by much. But for good measure, let’s run another similar test.
This time, let’s introduce an intentional typo in the content.
For example, let’s remove the letter “i” from “their”:
This time the AI index changed quite drastically.
This change kind of makes sense—AI writers these days don’t make these types of mistakes. But it emphasizes this kind of typo too much, in my opinion. I would think the score would go from 100% to 95% at most. But not all the way to 58%.
Based on this simple test, it’s quite easy to trick the CrossPlag AI detector.
On the other hand, now the text has a typo. In other words, this is not the most practical way for passing an AI content detector test.
Test 3: Use an AI Paraphraser
➡️ TLDR; Paraphrasing the AI-written text did not change CrossPlag AI content detectors mind.
Let’s run one more test to try to fool the AI checker.
These days there are quite powerful paraphrasing tools that are also powered by AI. An AI paraphrase takes your text as input and spits out a re-worded version of the text.
One example of such a paraphrasing tool is QuillBot.
I’ve written an entire review of QuillBot. Make sure to check it out in case this sounds interesting.
Anyways, when testing other AI content detectors, I’ve been able to fool them with this trick. Let’s see what happens with CrossPlag’s AI detector.
First, here’s a paraphrased version of one of the AI-written samples.
If the detector works, this should be marked as AI-written content.
Great, CrossPlag still flags it as AI-written content!
Based on these simple tests, it seems to be more difficult to trick the CrossPlag AI-checker than e.g. writer.com’s AI detector.
Although CrossPlag also was confused by removing a single letter from the AI-written text and reduced the AI-generated score quite a bit.
Although more rigorous testing is needed to get a better idea of the performance.
Pros
- Free to use. CrossPlag AI content detector is free to use. After 2 detections you need to sign up for the service, but you can continue using it for free.
- Accurate in detecting AI-written content. Based on the 10 AI-written samples, CrossPlag did a stellar job as it detected 90% of the content to be written by AI.
Cons
- Makes mistakes. CrossPlag cannot tell if human-written content is written by humans or AI. This is problematic if you don’t know who has written the content in advance.
- Doesn’t reflect Google’s AI detector. This is important for bloggers. If Google uses an AI detector, it probably works entirely differently than free online tools. So don’t use AI detectors and think “Now Google won’t detect my content as AI-written”.
Final Verdict
I think CrossPlag’s AI content detector is a great showcase of how seemingly well-written AI-generated content can easily be detected as AI-written.
You can use this type of tool to get an idea of the content. But CrossPlag is by no means accurate. If you don’t know whether the content is written by human or AI in advance, you can’t really trust the tool’s score.
For bloggers that generate blog posts with AI, this is a good reminder. Google will probably be able to point AI-written thin posts out quite easily. Although Google probably uses an entirely different method.