GPTZero: An App to Detect Whether Text is written by ChatGPT

Student-made app GPTZero determines whether the text was produced by AI or not.

A 22-year-old Princeton University senior named Edward Tian has developed an app to evaluate whether the text has been generated by ChatGPT, the popular chatbot that has raised concerns about its potential for unethical usage in academia. A part of Tian's winter break was spent developing GPTZero, which he claims can "quickly and efficiently" determine whether a person or ChatGPT wrote an essay. Tian is a computer science major with a minor in journalism. His desire to combat what he perceives as a rise in AI plagiarism led him to develop the bot. "There is a lot of excitement surrounding ChatGPT. Has AI written this and that? We deserve to know as humans!" In a tweet promoting GPTZero, Tian wrote. There have been allegations of students utilizing the ground-breaking language model to pass off AI-written assignments as their own since ChatGPT's introduction in late November. Teachers now have a new option at their disposal if they are concerned about pupils turning in essays created by a well-known chatbot. After Tian put his bot online on January 2, other teachers contacted him to tell him about the success they had testing it.

"Perplexity" and "burstiness" are two signs that GPTZero utilizes to assess whether an excerpt was written by a bot. Perplexity is a metric for text complexity; if GPTZero finds the text confusing, it has a high level of complexity and was probably created by a human. The text will have minimal complexity and so be more likely to be generated by AI if it is more familiar to the bot because it has been trained on such material. Burstiness contrasts the various sentence versions separately. Humans prefer to write more quickly, for instance, mixing longer, more complicated words with shorter ones. AI sentences are typically more consistent.

Tian compared the app's analysis of a New Yorker article and a ChatGPT post on LinkedIn in a demonstration video. It effectively discriminated between text produced by humans and AI. Within a week of its debut, GPTZero had been used by more than 30,000 users. Due to its extreme popularity, the app collapsed. Since then, Streamlit, the open source software that runs GPTZero, has intervened to support Tian by providing more memory and resources to deal with the website traffic.

In the struggle to stop AI plagiarism and fabrication, the college senior is not alone. The creator of ChatGPT, OpenAI, has expressed a dedication to halting AI plagiarism and other malicious applications. The organization has been working on a method to "watermark" GPT-generated text with an "unnoticeable secret signal" to identify its source, according to Scott Aaronson, an OpenAI researcher whose current area of expertise is AI safety. A tool to determine whether the text was produced by GPT-2, an earlier iteration of the AI model used to develop ChatGPT, has been released by the open-source AI community Hugging Face. A South Carolina philosophy professor who just so happened to be aware of the technique claimed to have used it to catch a student turning in work that had been created by an AI.

Tian is not against the usage of ChatGPT and other AI techniques. According to him, GPTZero is "not meant to be a tool to prevent the deployment of these technologies." However, like with any new technology, we must be able to responsibly adopt it and have protections. He remarked, "AI has been a black box for so long, and we truly don't know what's going on inside. And I wanted to start fighting back against it with GPTZero. Tian recognized that, despite what some users who have put his bot to the test have said, it is not infallible. He asserted that he is still striving to improve the model's precision. However, by creating an app that provides some insight into what distinguishes humans from AI, the tool advances Tian's primary goal of introducing transparency to AI.