Nearly two years since ChatGPT leapt onto the scene, upending the use of artificial intelligence in schools and beyond, companies have scrambled to provide tools that would help educators detect whether their students are passing off AI-generated work as their own.
But some researchers and civil rights advocates have expressed doubt that these types of AI detection systems are truly accurate and therefore helpful for schools as they navigate academic integrity policies.
The scope of these tools has now expanded to include at least one that can detect when students have used AI paraphrasing tools — meaning tools that adjust AI-generated text to cover up their use of generative AI. Turnitin, a plagiarism detection service, released the new feature this summer as an update to its previous AI detection tool.
As schools enter a new academic year, one expert from the Center for Democracy & Technology, a civil rights nonprofit, offers advice for how teachers and school leaders can best approach this latest development.
Don’t get caught up in the AI ‘arms race’
AI detection software is trained to spot text from AI generators and will eventually “get quite good” at identifying AI, said Hannah Quay-de la Vallee, senior technologist at CDT.
But then the generators will also change, adapt and improve, Quay-de la Vallee said.
How that “arms race” between AI detection software and AI generators like ChatGPT could influence the accuracy of detection software is worrisome, she said.
“Both of them are getting better over time, and what that means is that it’s hard to measure the efficacy of the … detectors in a long-term way,” Quay-de la Vallee said. What’s particularly concerning about these difficult-to-measure detection rates, she said, is that teachers may “think that they’re using a very accurate tool, and in reality, that accuracy fluctuates pretty consistently.”
Start conversations rather than rushing to discipline
Detection tools that target AI paraphrasing only further complicate the AI detector and generator arms race, Quay-de la Vallee said, because it’s one more thing for them to spar over. The paraphrasing detector just adds a new element into the mix that assumes humans are actively trying to deceive the detectors, she said.
Educators’ reliance on AI content detection tools rose 30 percentage points to 68% during the 2023-24 school year when compared to the prior year, according to CDT. The center also found that AI-related plagiarism suspicions drove an increase in student discipline due to AI use, from 48% to 64% between the 2022-23 and 2023-24 school years.
Quay-de la Vallee said she still fears schools will continue to over-rely on AI detectors to surface incidents of plagiarism that lead to disciplining students.
These detectors, however, can be used as a jumping off point for conversations with students, she said. “If you’re using the detector, that’s step one, and then you need to be talking to students and trying to figure out, like, what actually happened here.”
Districts should consider conducting a long-term analysis of AI detectors’ impact at individual schools, Quay-de la Vallee said. Some things to look out for could include how the detectors are being used, what these tools are finding, and how schools are reacting to or handling them.
It’s also important to also keep in mind that AI detectors could potentially cause schools to disproportionately discipline certain student populations based on the technology’s biases, she said.
