Yesterday, OpenAI, the company behind ChatGPT and DALL-E 2, launched a new “AI classifier” tool. This tool can distinguish whether a person has authored text or used artificial intelligence to generate textual content.
Since ChatGPT captured global attention for its text-generative AI abilities, many have been quite critical of its misuse. Sure, ChatGPT can be used to write codes, compose essays, suggest jokes for Twitter, or even help you improve your flirting skills on chatting, but the debate has been around the use of this tool in academia.
The chatbot has aced tests at the graduate level in various disciplines, including those for four law courses at the University of Minnesota and the MBA program final exam at the University of Pennsylvania. It also passed the American US Medical Licensing Examination comfortably within the passing range.
However, the accessibility and capability of ChatGPT have faced heat from educators worldwide. There have been multiple instances where teachers have expressed concerns about students employing ChatGPT to complete their assignments or write their papers for them. In January, the New York City Education Department banned the use of ChatGPT in schools, highlighting concerns about the reliability and accuracy of the content generated.
A few weeks ago, the International Conference on Machine Learning (ICML) published a guideline stating that papers using large-scale language models (LLMs), such as ChatGPT, are only allowed if the generated text is included as part of the paper’s experimental analysis. Additionally, Stack Overflow and other websites have prohibited users from publishing ChatGPT-generated content because they believe that the AI makes it too simple for users to oversaturate discussion threads with inappropriate responses. Recently, Yann LeCun, Meta’s chief AI scientist, claimed that he does not find ChatGPT to be particularly innovative.
But not every educator feels the same about ChatGPT. For example, in reaction to ChatGPT and concerns that students would use it to write essays and cheat on examinations, the Group of Eight (Go8) consortium of prestigious Australian institutions announced last month that its members would set more pen-and-paper assessments. However, soon they found themselves discussing the benefits of tools like ChatGPT for disabled students.
Read Also: Interesting ways users are testing ChatGPT, a new conversational language model
According to OpenAI’s press statement, the AI classifier is a language model refined using a dataset of pairs of texts authored by humans and AI on the same topic. This dataset was compiled from a number of sources, including pretraining data and human demonstrations on InstructGPT prompts.
OpenAI believes the new tool will help stop the exploitation of AI text-generation tools for unethical purposes, including automated misinformation campaigns, plagiarism, academic dishonesty, and posing an AI chatbot as a human.
Despite the fact that this could be a game-changer in addressing worries regarding AI-generated content, OpenAI clarified that there are limitations. According to OpenAI, “the AI classifier is not fully reliable.”
With a 26% accuracy rate in detecting non-human generated text, the classifier still has room for improvement. During the tests, OpenAI also found that the classifier can mistakenly classify human-written material as AI-written text 9% of the time. There are other caveats too. For instance, the classifier performs relatively worse when the text is fewer than 1,000 characters in length or about 150 to 250 words. Additionally, the classifier may not work as well with text in other languages because it is designed for English-language text.
We used the OpenAI classifier to check its effectiveness and found that while it correctly labeled some Analytics Drift articles, it did mislabel a few. On Twitter, the Founder of AlphaSignal.ai shared how they were successful in tricking the classifier by using GPT3 to rewrite the text. So, the claims are true!!
You can access the AI Classifer with just a few clicks! Visit the page, log in, copy your text, and then wait for the classifier to do its thing. It won’t take long for it to rank your text on a scale ranging from very unlikely to likely. It’s that easy!
Edward Tian, a student at Princeton University, released GPTZero earlier last month and noted on the tool’s website that it was designed for educators. So far, more than 5000 educators from top universities, including Yale and Harvard, have signed up to use GPTZero. Copyleaks, a plagiarism detection program, introduced its own AI Content Detector for educational institutions and publishing last month. The Giant Learning Model Test Room, a partnership between the MIT-IBM Watson AI Lab and the Harvard Natural Language Processing Group, can also recognize AI-generated work using predictive text.
