Bright Learning.

Guiding education through the AI revolution, for the people who run, teach, and care about schools.

Opinion

Stop Disciplining Students on a Score the Vendor Will Not Explain

AI writing detectors should be barred from academic-integrity cases because an accusation no one can examine is not evidence.

Turnitin's AI detector hands a teacher a single number, a percentage saying how much of an essay it judges to be written by AI. It does not show the teacher how it reached that figure, and it cannot show the student who now has to answer for it. By Vanderbilt University's own account of why it disabled the detector in 2023, the company says only that its tool looks for patterns common in AI writing, without defining those patterns. That opacity, more than any accuracy gap, is why districts should stop using AI writing detectors to decide whether a student cheated.

Let me be plain about where I stand and why. I write this for Bright Learning, which builds learning tools and has a stake in how artificial intelligence enters classrooms. This is an argument, not a neutral survey, and it is narrow on purpose. I am not saying schools should ignore AI or stop teaching students to use it well. I am saying that one specific practice, disciplining a student on a detector's score, should end.

Start with the arithmetic that vendors would rather you round away. Turnitin claimed its detector carried a one percent false-positive rate. One percent sounds like a rounding error until you scale it. Vanderbilt submitted roughly seventy-five thousand papers in 2022, which means about seven hundred and fifty of them could have been wrongly tagged as containing AI writing. Those are not abstractions. Each one is a student called in to defend work they actually did, and a one percent error at that scale is a standing machine for producing that scene hundreds of times over.

The error is not even randomly distributed. A Stanford study of widely used GPT detectors found that they consistently misclassified writing by non-native English speakers as AI-generated, while identifying native speakers' work correctly. The students most likely to be falsely accused are therefore the ones with the least standing to argue back, and the ones whose plain, carefully chosen prose already reads as unfamiliar to an American grader. A screening tool that fails hardest on the most vulnerable writers is not a neutral instrument.

Here is the argument that should settle it, and it has nothing to do with the detector's accuracy rate. A disciplinary process is supposed to let the accused see the evidence and contest it. A Turnitin score cannot be contested, because the method behind it is undisclosed. The student cannot point to the sentence that fooled it, cannot reproduce the result, and cannot cross-examine a percentage. An accusation that cannot be examined is not evidence. It is a verdict wearing the costume of evidence, and no school would accept that standard from a human proctor who refused to explain what he saw.

The strongest case for keeping the detectors is real, and worth stating fully. Teachers are facing genuine, widespread AI-assisted cheating, they are outnumbered, and they have no other tool that scales to the volume of essays they grade. In that bind, a detector is not supposed to be a verdict but a triage flag, a way to decide which papers deserve a closer human look. Take the tool away, the argument goes, and you appear to tell students that cheating is now free.

But triage only works if the flag points somewhere a human can verify, and this one does not. A teacher who reopens the flagged paper finds the same undisclosed score, not a reviewable trail, so the detector does not narrow the inquiry. It launders a guess into a suspicion. The honest alternative is harder and better: move the graded writing to where AI cannot quietly do it, through in-class drafts, visible revision history, and brief oral defenses of the work. That shifts the labor onto schools rather than onto falsely accused students, and it rests on evidence a student can actually face. Districts do not need a better detector. They need to stop outsourcing an accusation to a tool that will not say what it saw.

Take it further