I found a great way to fool these detectors: piping output through multiple generative models.
1. Generate text by promoting ChatGPT.
2. Rewrite / copyedit with Wordtune [1], InstaText [2] or Jasper.
This fools GPTZero [4] consistently.
Of course soon these emotive, genre or communication style specialisations will be promptable too by a single model too. Detectors will be integrated as adversarial agents in training. There is no stopping generative text tooling, better adopt and integrated it fully into education and work. Resistance is futile.
This approach seems to require knowledge of which LLM was used to generate the given text. I wonder if e.g. model fine-tuning - as already provided by Open AI [0] - could evade this detection approach.
> For example, students may use LLMs to complete written assignments, leaving instructors unable to accurately assess student learning.
I'm one student using GPT to write my written assignments, and I cannot understand what is the issue with that. Even though it enhances the quality of my text, I cannot just copy, paste, submit and profit. I still have to understand what is being written. My workflow is normally the following:
- I ask for an outline to the essay. I use this outline as guide, remove some suggestion, add others that I think are important, rephrase some. I could as well google for similar written essays and do the same.
- Then I ask for the topics, please describe X, Y.. I read it, as a regular source. I understand it, and rewrite it. I add stuff, remove stuff, add references. Normally I endup rewriting everything
- Push the text through Grammarly, rephrase some stuff, check for plagiarism and submit it. I have to read and re-read it, many times.
I have the impression that I've got much more effective avoiding the mechanical part of the task of writing. In another hand my course has some stupid enforcement like 15k words/essay. So they are almost asking me to do some bullshit around the topic instead of going straight to the point. I simply automate the burden. As any software developer would do. I however study philosophy.
So i guess, it will help the universities and education institutions to rethink their traditional way of evaluate written assignments, which probably will be positive. It helps me definitely to write better.
The problem is that people aren't doing what you're doing, and ARE just copy/pasting whatever gets spit out. Using it to enhance your wording + thoughts is absolutely the way to make use of these tools.
Ok, but it's worse than copy and paste from internet, because not every thing that chatgpt writes make sense. Longer the output, higher the chances that it doesn't make sense.
Honestly this doesn't sound like any less work than just reading the sources and writing it myself in the first place. You seem to be doing just as much revising and writing, with a LOT more "read and re-read."
Yes, maybe because I'm studying because I want, as hobby,but in another hand,in my university the last examination, each semester, is oral, so if you don't know, you fail. So I have to learn.
As would be with code too. You have to validate the code, integrate it to your codebase and understand it,since soon or later you will have to debug it in production
The approach to feed text through a language model and then obtain some kind of vector/hash from it will be leveraged to fingerprint writing and track people.
I had a go at using ChatGPT to write some (not very good) fiction. I give it my story idea, and it gives me a few paragraphs. I edit its output, keeping the stuff I like, deleting things I don't, rewording some things, adding a few sentences of my own. Then I say "Now write a new paragraph about how <new story element>". I read what it says and then I say "Change what you just said so that...". Then I paste that new paragraph into my document at the appropriate point. So the text I'm constructing is neither purely ChatGPT nor purely me, it is a mix of my ideas and its ideas, my words and its words. I wonder if this tool would detect such a text as ChatGPT-generated or not?
Back in the day I made my money cleaning up software projects where outsourcing to India, Romania and Brazil had gone horribly wrong and burned through piles of cash with very poor results to show for it.
I suspect the incoming software development youth will be able to make their early money cleaning up the mess of AI application to software development.
Nobody is ever going to ship unreviewed code. It doesn't matter if a human or a robot wrote it.
Someone also has to write acceptance criteria beyond a mere prompt for a chat bot.
Someone has to approve it all, and it's not going to be a non-technical person. Someone also has to test it. Someone also has to integrate it with the rest of the solution. Code has dependencies and you can't manage all of them with AI alone.
Since the developer does all this already where the literal code is such a small part of the role, why even bother using generated code?
I can't judge the capabilities of this tool, but I can't help but be skeptical on the mid-long term prospects. The fakes (text, audio, video) will improve to the point of being virtually unrecognizable from real material, no matter how good your fake detection is.
Can some chess players detect when an opponent is using stockfish? Yes, if they’re Magnus Carlsen and memorized a particular line of play. This is a bit like that. But yes, I suppose when these tools become so fluent that they basically never need to use the same pattern twice, we’re toast.
AFAIK top chess players can recognize they're playing against an engine relatively easily. The engines are so far ahead that they often play moves which are "surprising" even for Carlsen, i.e. lines which may seem suboptimal or just unknown - this would normally indicate a weak player, but engines then crush even Carlsen with ease.
But these engines are optimized to win, not to mimic humans, so it's a different story from ChatGPT.
It’s typically not the moves alone that indicate engine use to the other player. The timing is also incredibly important. If it takes you 5 seconds to recapture a piece, where that’s the only viable move, but it also takes you 5 seconds to come up with an insane brilliancy that’s the result of a deeply calculated line, cheating is a likely explanation.
Not really comparable IMO. Stockfish is not built to resemble human play and the move times give away a ton of information. A better comparison would be if a top player could tell when they are looking at games played by Maia.
From a technical standpoint, this is interesting. Like all other methods I’ve seen, it requires access to the original model logits, so it can’t be used in closed models like GPT-3 except by OpenAI themselves.
But I would classify this into the category of dangerous research. There is a conflict of interest: overstating the effectiveness leads less technical people to give undue trust to what is ultimately a statistical decision. And it shouldn’t require repeating how serious the consequences of this decision are.
The most likely outcome is that methods like this are used to diffuse responsibility. The next step is for institutions to buy into one of these classifiers and then to create a punitive “policy”.
> This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g, T5).
Very interesting, though the passage above makes me wonder how robust it is to different models or even finetuned variations on models, as even GPT-3(.5) has evolved quite a bit over recent releases since its initial introduction, and there is likely to only be a greater and greater proliferation of models over time.
Awesome thread by one of the authors of paper, concise and insightful, thanks for sharing !
Research was aiming to bring a tool/approach on distinguishing text from LLM and other sources but in the end of the day it will only benefit those with non-open-source LLMs adjust to such technique and "fool" better everyone else (cause we need log-probs out of a model on each of the sample text).
It seems kinda ironic for me, maybe i missed some crucial point here.
“Unlike human-written text, model-generated text tends to lie in areas where the log probability function has negative curvature (e.g., local maxima of the log probability).”
Is there really something of value to be had by fighting a war against the coming tide of AI that can spit out whatever you ask for?
To me it looks like old world professors calling the council because the abacus is under threat by the calculator, and we need to ensure the students are absolutely not using the calculator for their abacus studies. This all being despite the fact that society at large is moving as fast as it can to leave the abacus for calculators.
I can't help that but feel that people have lost the forest for the trees. So transfixed on the small steps that they have completely lost track of why we take all those small steps, and the utility that they ultimately provide.
If you want to study whatever subject and become an expert, history or chemical engineering, cool, go ahead. I'm still gonna chose my AI consultant over you in 5-10 years.
It is valuable for SPAM filters, and other fraudulent activity. If you get an e-mail from someone you know and it says "I need itunes gift cards right away!" most people know that is a scam. But if you get an e-mail that appears to be following up on a previous conversation, you are much more likely to click on a link in that e-mail.
I doubt spammers will be the only ones leveraging AI generated text. I'm sure legit marketing people and even administrators and support staff will benefit from using AI written responses. So you're going to filter out valuable emails as well.
> I'm still gonna chose my AI consultant over you in 5-10 years.
AI will never replace experts.
Someone still has to curate the firehose of information returned. What's the point then?
AI can only interpolate a response from internet sources, and claims of plagiarism are already coming hard. The internet was never a good source of expertise.
If an AI can be trained on the internet, it can be trained on a textbook or lecture notes. I think it’s shortsighted to insist that AI won’t replace experts based on the training sets that AIs are currently being trained on.
Who writes the books and lectures? This is literally chicken and egg. Can't have eggs without chickens. Can't have chickens without eggs. This is not an oversimplification.
AI can detect pancreatic cancer better than the best radiologists. The belief that machine learning cannot surpass human intelligence is contrary to the evidence.
That’s what Wall street said about expert traders. And then replaced them all with bots and gave the capital to the bots to be traded.
I remember when Garry Kasparov insisted that centaurs -- computer+human — do better than just a computer or just a human. He held out longer than most in this belief. But seriously… can you name a field where computers and bots can NEVER beat humans?
Any topic where being an expert is not mere memorization or imitation without comprehension, and discovering new facts is not just relentlessly churning through a rule set and producing what was already self-evident.
I don't disagree whether computers are good at computation. That's what they're for. I disagree that AI can replace experts because expertise is a moving target.
Where AI approximates a response given at some point in time or within a range of time written by a crowd of people who are not necessarily experts, experts must inevitably contradict themselves on something and update their theories. AI as currently implemented will always lag behind human experts.
In other words, do you seriously believe that an AI trained on scientific publications from the middle ages would eventually rediscover what we know now and beyond? I think it would be stuck forever hallucinating recipes to turn lead into gold.
What does 'never' mean here. Do you literally mean that it can't happen even if human society progresses for another millenia? Or just that it won't happen in your career. The latter is somewhat plausible to me, but the former is only happening if there is a civilization ending catastrophe.
For this to remain true for the indefinite future we must assume that strong AI is strictly impossible. Our scientific knowledge today does not suggest that this is the case.
Experts used to evaluate your credit application. Experts would adjust depth of field in portrait photography, and bracketed exposure in HDR photography. Experts would beat you at chess.
AI now does the majority of instances of these things and many more. Not every instance, so you can say that experts have not been completely replaced, but they have been substantially replaced.
As for ‘can only interpolate a response from internet sources’, first, there’s no requirement that training data comes from the internet, and second, if the AI has learned the correct function, ‘interpolation’ is not a criticism.
Typically, outside experts are useful for out of distribution inference about a particular situation. Fine-tuned generative AI is self evidently not good at generalizing outside the scope of its training data, and becoming reliant on it for this task is foolish.
Signed, AI researcher.
I mean, is the very situation you stated not a situation you are trying to solve? Would you stumble upon such a solution and bury it because AI isn't supposed to be more informed than experts?
I found a great way to fool these detectors: piping output through multiple generative models.
1. Generate text by promoting ChatGPT.
2. Rewrite / copyedit with Wordtune [1], InstaText [2] or Jasper.
This fools GPTZero [4] consistently.
Of course soon these emotive, genre or communication style specialisations will be promptable too by a single model too. Detectors will be integrated as adversarial agents in training. There is no stopping generative text tooling, better adopt and integrated it fully into education and work. Resistance is futile.
1. https://www.wordtune.com/
2. https://instatext.io/
3. https://www.jasper.ai/
4. https://gptzero.me/
This approach seems to require knowledge of which LLM was used to generate the given text. I wonder if e.g. model fine-tuning - as already provided by Open AI [0] - could evade this detection approach.
[0] https://beta.openai.com/docs/guides/fine-tuning
> For example, students may use LLMs to complete written assignments, leaving instructors unable to accurately assess student learning.
I'm one student using GPT to write my written assignments, and I cannot understand what is the issue with that. Even though it enhances the quality of my text, I cannot just copy, paste, submit and profit. I still have to understand what is being written. My workflow is normally the following:
- I ask for an outline to the essay. I use this outline as guide, remove some suggestion, add others that I think are important, rephrase some. I could as well google for similar written essays and do the same.
- Then I ask for the topics, please describe X, Y.. I read it, as a regular source. I understand it, and rewrite it. I add stuff, remove stuff, add references. Normally I endup rewriting everything
- Push the text through Grammarly, rephrase some stuff, check for plagiarism and submit it. I have to read and re-read it, many times.
I have the impression that I've got much more effective avoiding the mechanical part of the task of writing. In another hand my course has some stupid enforcement like 15k words/essay. So they are almost asking me to do some bullshit around the topic instead of going straight to the point. I simply automate the burden. As any software developer would do. I however study philosophy.
So i guess, it will help the universities and education institutions to rethink their traditional way of evaluate written assignments, which probably will be positive. It helps me definitely to write better.
The problem is that people aren't doing what you're doing, and ARE just copy/pasting whatever gets spit out. Using it to enhance your wording + thoughts is absolutely the way to make use of these tools.
Ok, but it's worse than copy and paste from internet, because not every thing that chatgpt writes make sense. Longer the output, higher the chances that it doesn't make sense.
Honestly this doesn't sound like any less work than just reading the sources and writing it myself in the first place. You seem to be doing just as much revising and writing, with a LOT more "read and re-read."
Yes, maybe because I'm studying because I want, as hobby,but in another hand,in my university the last examination, each semester, is oral, so if you don't know, you fail. So I have to learn.
See the more recent comment here by @skissane about using chatGPT to write fiction. Process sounds quite similar.
As would be with code too. You have to validate the code, integrate it to your codebase and understand it,since soon or later you will have to debug it in production
The approach to feed text through a language model and then obtain some kind of vector/hash from it will be leveraged to fingerprint writing and track people.
In addition, this can feed a sort of “recaptcha” loop that trains AI to notice AI and create new text less likely to be noticed…
I had a go at using ChatGPT to write some (not very good) fiction. I give it my story idea, and it gives me a few paragraphs. I edit its output, keeping the stuff I like, deleting things I don't, rewording some things, adding a few sentences of my own. Then I say "Now write a new paragraph about how <new story element>". I read what it says and then I say "Change what you just said so that...". Then I paste that new paragraph into my document at the appropriate point. So the text I'm constructing is neither purely ChatGPT nor purely me, it is a mix of my ideas and its ideas, my words and its words. I wonder if this tool would detect such a text as ChatGPT-generated or not?
I might not be understanding what AUROC really means, but how is this performing in terms of false positives/negatives in comparison to human readers?
Here’s a post on understanding AUC (Area Under The Curve) and ROC (Receiver Operating Characteristics):
https://towardsdatascience.com/understanding-auc-roc-curve-6...
Back in the day I made my money cleaning up software projects where outsourcing to India, Romania and Brazil had gone horribly wrong and burned through piles of cash with very poor results to show for it.
I suspect the incoming software development youth will be able to make their early money cleaning up the mess of AI application to software development.
Nobody is ever going to ship unreviewed code. It doesn't matter if a human or a robot wrote it.
Someone also has to write acceptance criteria beyond a mere prompt for a chat bot.
Someone has to approve it all, and it's not going to be a non-technical person. Someone also has to test it. Someone also has to integrate it with the rest of the solution. Code has dependencies and you can't manage all of them with AI alone.
Since the developer does all this already where the literal code is such a small part of the role, why even bother using generated code?
I think you overestimate how a lot of software is and, in particular, will be created.
Good then their code will be even shittier than it already is.
How does this compare to GPT2 output detector?
I created AIwritingcheck.org to provide teachers with a user friendly interface for this model.
Consider trying out GPTKit https://gptkit.ai it has higher accuracy than GPTZero and uses 6 different methods
This recommendation would come across as a little more credible if you had been upfront that you're the creator behind it.
Wouldn't GPT-2 be detectable by virtue of being bad? Or is it bad in a way that is more or less indistinguishable from student writing?
It's been a while since I used it but I very rarely got plausible output from it.
Stop doing this. Have teachers figure out how to move forward with this and students, but the false-positives are not worth it
I can't judge the capabilities of this tool, but I can't help but be skeptical on the mid-long term prospects. The fakes (text, audio, video) will improve to the point of being virtually unrecognizable from real material, no matter how good your fake detection is.
Can some chess players detect when an opponent is using stockfish? Yes, if they’re Magnus Carlsen and memorized a particular line of play. This is a bit like that. But yes, I suppose when these tools become so fluent that they basically never need to use the same pattern twice, we’re toast.
AFAIK top chess players can recognize they're playing against an engine relatively easily. The engines are so far ahead that they often play moves which are "surprising" even for Carlsen, i.e. lines which may seem suboptimal or just unknown - this would normally indicate a weak player, but engines then crush even Carlsen with ease.
But these engines are optimized to win, not to mimic humans, so it's a different story from ChatGPT.
It’s typically not the moves alone that indicate engine use to the other player. The timing is also incredibly important. If it takes you 5 seconds to recapture a piece, where that’s the only viable move, but it also takes you 5 seconds to come up with an insane brilliancy that’s the result of a deeply calculated line, cheating is a likely explanation.
Not really comparable IMO. Stockfish is not built to resemble human play and the move times give away a ton of information. A better comparison would be if a top player could tell when they are looking at games played by Maia.
https://maiachess.com/
From a technical standpoint, this is interesting. Like all other methods I’ve seen, it requires access to the original model logits, so it can’t be used in closed models like GPT-3 except by OpenAI themselves.
But I would classify this into the category of dangerous research. There is a conflict of interest: overstating the effectiveness leads less technical people to give undue trust to what is ultimately a statistical decision. And it shouldn’t require repeating how serious the consequences of this decision are.
The most likely outcome is that methods like this are used to diffuse responsibility. The next step is for institutions to buy into one of these classifiers and then to create a punitive “policy”.
You can get all the necessary log probabilities from the GPT-3 API.
Seems to me you might be able to guess without the weights by manually changing a word or two and seeing how much gpt wants to change.
Hmm seems you may be right. You can get the top 5 out of the box but you’ll need to file for an exception to get any more than that.
> This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g, T5).
Very interesting, though the passage above makes me wonder how robust it is to different models or even finetuned variations on models, as even GPT-3(.5) has evolved quite a bit over recent releases since its initial introduction, and there is likely to only be a greater and greater proliferation of models over time.
Research paper:
https://arxiv.org/abs/2301.11305
Additional explanation:
https://twitter.com/_eric_mitchell_/status/16188203614199152...
Awesome thread by one of the authors of paper, concise and insightful, thanks for sharing !
Research was aiming to bring a tool/approach on distinguishing text from LLM and other sources but in the end of the day it will only benefit those with non-open-source LLMs adjust to such technique and "fool" better everyone else (cause we need log-probs out of a model on each of the sample text).
It seems kinda ironic for me, maybe i missed some crucial point here.
Highly unlikely any popular model would not provide log-probs, since they’re potentially useful. For example:
https://aidungeon.medium.com/controlling-gpt-3-with-logit-bi...
“Unlike human-written text, model-generated text tends to lie in areas where the log probability function has negative curvature (e.g., local maxima of the log probability).”
Is there really something of value to be had by fighting a war against the coming tide of AI that can spit out whatever you ask for?
To me it looks like old world professors calling the council because the abacus is under threat by the calculator, and we need to ensure the students are absolutely not using the calculator for their abacus studies. This all being despite the fact that society at large is moving as fast as it can to leave the abacus for calculators.
I can't help that but feel that people have lost the forest for the trees. So transfixed on the small steps that they have completely lost track of why we take all those small steps, and the utility that they ultimately provide.
If you want to study whatever subject and become an expert, history or chemical engineering, cool, go ahead. I'm still gonna chose my AI consultant over you in 5-10 years.
It is valuable for SPAM filters, and other fraudulent activity. If you get an e-mail from someone you know and it says "I need itunes gift cards right away!" most people know that is a scam. But if you get an e-mail that appears to be following up on a previous conversation, you are much more likely to click on a link in that e-mail.
I doubt spammers will be the only ones leveraging AI generated text. I'm sure legit marketing people and even administrators and support staff will benefit from using AI written responses. So you're going to filter out valuable emails as well.
There are no valuable emails from 'legit marketing people', especially so if they're using AI-generated messages to hit you with a firehose.
Maybe not for you, I subscribe to brands I like.
> I'm still gonna chose my AI consultant over you in 5-10 years.
AI will never replace experts.
Someone still has to curate the firehose of information returned. What's the point then?
AI can only interpolate a response from internet sources, and claims of plagiarism are already coming hard. The internet was never a good source of expertise.
If an AI can be trained on the internet, it can be trained on a textbook or lecture notes. I think it’s shortsighted to insist that AI won’t replace experts based on the training sets that AIs are currently being trained on.
Who writes the books and lectures? This is literally chicken and egg. Can't have eggs without chickens. Can't have chickens without eggs. This is not an oversimplification.
Yes, now you're getting it.
What happens when the AI can write itself? What happens when it can design its own hardware? And assemble it?
What have we created when it is fully capable of self replication?
Stagnant crap nobody wants?
AI can detect pancreatic cancer better than the best radiologists. The belief that machine learning cannot surpass human intelligence is contrary to the evidence.
Citation for your pancreatic cancer claim? I am skeptical, or perhaps we have different definitions of better.
That’s what Wall street said about expert traders. And then replaced them all with bots and gave the capital to the bots to be traded.
I remember when Garry Kasparov insisted that centaurs -- computer+human — do better than just a computer or just a human. He held out longer than most in this belief. But seriously… can you name a field where computers and bots can NEVER beat humans?
Any topic where being an expert is not mere memorization or imitation without comprehension, and discovering new facts is not just relentlessly churning through a rule set and producing what was already self-evident.
I don't disagree whether computers are good at computation. That's what they're for. I disagree that AI can replace experts because expertise is a moving target.
Where AI approximates a response given at some point in time or within a range of time written by a crowd of people who are not necessarily experts, experts must inevitably contradict themselves on something and update their theories. AI as currently implemented will always lag behind human experts.
In other words, do you seriously believe that an AI trained on scientific publications from the middle ages would eventually rediscover what we know now and beyond? I think it would be stuck forever hallucinating recipes to turn lead into gold.
>AI will never replace experts
Just like it will never beat a top ranked GO player.
Go has static rules that can be mastered and a finite number of games possible. Expertise is always evolving.
> a finite number of games possible.
Effectively infinite with modern hardware. AI did not solve Go by making an effort to explore the state space. That's what's amazing about it.
There are vastly more possible games than atoms in the observable universe. The fact that it is finite is irrelevant.
> The internet was never a good source of expertise.
Agree, but world listen internet "experts". A lot.
What does 'never' mean here. Do you literally mean that it can't happen even if human society progresses for another millenia? Or just that it won't happen in your career. The latter is somewhat plausible to me, but the former is only happening if there is a civilization ending catastrophe.
For this to remain true for the indefinite future we must assume that strong AI is strictly impossible. Our scientific knowledge today does not suggest that this is the case.
It already has in several useful, practical ways.
Experts used to evaluate your credit application. Experts would adjust depth of field in portrait photography, and bracketed exposure in HDR photography. Experts would beat you at chess.
AI now does the majority of instances of these things and many more. Not every instance, so you can say that experts have not been completely replaced, but they have been substantially replaced.
As for ‘can only interpolate a response from internet sources’, first, there’s no requirement that training data comes from the internet, and second, if the AI has learned the correct function, ‘interpolation’ is not a criticism.
Typically, outside experts are useful for out of distribution inference about a particular situation. Fine-tuned generative AI is self evidently not good at generalizing outside the scope of its training data, and becoming reliant on it for this task is foolish. Signed, AI researcher.
I mean, is the very situation you stated not a situation you are trying to solve? Would you stumble upon such a solution and bury it because AI isn't supposed to be more informed than experts?
these kind of comments “I will hire an AI instead of a human” are all quite ironic.
the only things you will find out from asking an AI consultant is what occurred to _you_ to ask it.
Interacting with a human will always give you more ideas, opinions and .. more questions.
And you both can use the AI consultant, and get that important thing done.
This generation of AI? Sure.
Five or ten generations of AI from now? I'd place my full bet firmly on "Humans aren't nearly as special as they tell themselves they are"
I value being able to spot a T-1000 in disguise!
I feel you — but there's another possible reason, which would be to (optionally) separate out the AI generated content from future model training.
How long before someone trains a troll bot on 4chan content?
Remember Tay?
https://en.wikipedia.org/wiki/Tay_(bot)