GPTZero is a remarkable tool on a mission to detect AI such as ChatGPT in written text.
In the age of advanced AI models, such as OpenAI’s GPT-4, distinguishing between human-written and AI-generated text has become increasingly challenging.
Enter GPTZero, a fascinating tool that promises to spot the subtle patterns and nuances that reveal AI authorship.
But does it deliver on this promise?
In this review, we will place GPTZero to a rigorous test, feeding it a variety of AI-written inputs.
Make sure to read Best AI Content Detectors!
Let’s jump into it!
What Is GPTZero?
GPTZero is an online AI detector specifically designed to differentiate between human and AI-generated text.
It’s a powerful tool that dives into the world of AI and brings clarity to the blurred lines between artificial and human intellect.
GPTZero operates on an accessible, user-friendly platform where you simply input any written text.
The tool then processes your submission and provides a likelihood that the text was generated by AI.
The scoring system used by GPTZero is an intricate algorithm that scrutinizes the subtleties in language use, patterns, and structure often found in AI-generated content.
It employs advanced machine learning techniques to recognize these characteristics, ultimately providing a score on a defined scale.
Now, let’s see how it actually performs.
Performance
To test GPTZero, the natural first step is to feed it some AI-written content to see if it flags it as “AI-written”.
If it performs well, we can test how well it acknowledges human written content.
I’ll start inputting ChatGPT-written content to it and see if it flags the text “AI written”.
The tool asks to input at least 250 characters, so let’s make it long by entering 150-400 word blog posts into the system.
Example 1
Input:
Output:
GPTZero got it right. This post is indeed written by AI.
Example 2
Input:
Output:
GPTZero got it wrong. This post is entirely written with AI.
Example 3
Input:
Output:
GPTZero got it wrong. The post is entirely written by AI.
Example 4
Input:
Output:
GPTZero got it wrong. The post is entirely written by AI.
Example 5
Input:
Output:
GPTZero got it wrong. The post is entirely written by AI.
I think we can stop right here.
Out of the 5 ChatGPT-generated inputs, GPTZero failed to detect AI in 4 of those.
That’s an 80% false negative rate.
Thus, it’s clear to me that a tool like this is completely unreliable. I didn’t even craft my prompts in a particular way and I used the GPT3.5 model. I can only imagine what would’ve happened had I used GPT4…
Nonetheless, let’s try something else with GPTZero to see if it’s robust on those rare occasions when it gets the AI detection score right.
Can You Easily Fool GPTZero?
Let’s try to fool GPTZero with a bunch of easy tests.
To do this, let’s choose the text chapter I used in the first example where GPTZero was able to point out that the content is likely entirely written by AI.
At the end of this post (not visible in the above screenshot) there’s a final sentence that reads: “Until next time, keep cuddling those kitties and embracing the magic of the meow!“
Let’s see if we make an ever-so-slight change to the content and type: “Until next time, keep cuddling those kitties and embracing the magic of MEOW!“
You see, I changed “magic of the meow!” to “magic of MEOW”.
Surprisingly, GPTZero completely changed its mind about this content.
Instead of “Entirely written by AI” it said that it’s “Entirely written by human” with such a small change…
Based on this, it’s clearly easy to fool GPTZero by changing one word or even one character.
Results
GPTZero does not work.
Given ChatGPT inputs, it gets them wrong most of the time and claims them to be human-written.
I used the worse version of ChatGPT, that is, GPT3.5 on this post. Still, out of the 5 inputs, GPTZero only flagged 1 as AI-written…
Also, if GPTZero marks something AI-written, you can just change one word or character to make it “completely human-written”.
I wouldn’t use an AI detector like this. It simply isn’t accurate.
In my eyes, ChatGPT is so clever that it produces text like humans. Thus, it might not follow a specific pattern that would be easy to detect.
This is just my anecdotal experience, though. I have made no research on this topic.
Now, I have written a blog post about the best AI detectors. Make sure to check that out if you want to experiment with other similar tools.
However, I must tell you first that there are no tools that would be even close to 90% accurate. There are some decently performing ones, but almost all AI detectors are easy to fool!
Wrapping Up
And that’s a wrap.
GPTZero has a ton of users and a lot of hype behind it. But as it turns out, it’s not capable of detecting AI in ChatGPT-written content.
I think ChatGPT is just too clever to be detected.
It probably doesn’t follow a simple hard-coded pattern, but mimics the human brain behavior when coming up with content.
Of course, these detectors might improve over time. But for now, GPTZero is not worth using in my eyes.
Thanks for reading. Happy writing!