Bright Learning.

Guiding education through the AI revolution, for the people who run, teach, and care about schools.

Opinion

Districts Should Switch Off AI Writing Detectors as Tools of Discipline

AI writing detectors fail where it matters most for schools, flagging innocent students, concentrating errors on non-native English speakers, and hiding their reasoning, so districts should strip them of disciplinary authority.

Let me be clear about what this is: an argument, not a buyer's guide. I think school districts should switch off AI-writing detectors as instruments of discipline, and I am going to try to convince you. Bright Learning sells no detection software and holds no financial stake in any vendor these tools compete with, so read this as a position, not a pitch.

Start with the strongest case for keeping the detectors, because it is a real one. Generative AI has made it trivial for a student to hand in an essay they never wrote, and teachers who already carry full rosters cannot read every paragraph closely enough to catch it by instinct. A detector, even an imperfect one, promises an overworked teacher a place to start looking. Honest students lose something when cheating goes unanswered, and doing nothing is not a neutral policy. If a detector reliably flagged machine writing and nothing else, the case for using it would be close to settled.

It does not, and the failure is not random. When a Stanford-led team ran seven widely used detectors against 91 English essays written by non-native speakers, the tools misclassified more than half of that human writing as AI-generated, an average false-positive rate of about 61 percent, and one detector flagged nearly 98 percent of the essays. The same detectors correctly classified more than 90 percent of essays by U.S. eighth-graders as human. The reason is mechanical rather than mysterious: the tools score writing by perplexity and read the common, lower-variety word choice of a second-language writer as the signature of a machine. A tool with that property is not merely inaccurate. It is a disciplinary instrument that aims its errors at the students who are already most likely to be doubted, and least equipped to contest the charge.

Scale makes the problem worse, not better. Turnitin, the dominant vendor, has advertised a false-positive rate of about one percent, which sounds tolerable until you multiply it. Vanderbilt University submitted 75,000 papers to Turnitin in 2022, and at that rate roughly 750 papers could have been wrongly flagged as having used AI. Vanderbilt disabled the detector, and its reasons should trouble any district considering one. The feature arrived with less than 24 hours of notice, no option to turn it off, and no detailed explanation of how it reaches its verdicts. You cannot run a fair disciplinary process on a black box, because a student cannot rebut evidence the institution itself cannot see.

Then there is the fact that should end the conversation. OpenAI, the company that builds the models these detectors claim to catch, released its own AI text classifier and then shut it down on July 20, 2023, citing a low rate of accuracy. Its tool correctly identified only 26 percent of AI-written text while flagging 9 percent of human writing as machine-made. If the organization with the deepest knowledge of how this writing is generated could not build a detector it was willing to stand behind, a district has little reason to trust a third-party product making the same promise. Cambridge and other Russell Group universities, along with the University of Texas at Austin, have already opted out of Turnitin's detector on exactly these grounds.

None of this means the underlying worry is imaginary, and here I want to be careful not to overclaim. Students will use AI, and schools do need an answer. The point is narrower and firmer: the answer cannot be a tool that manufactures false accusations, concentrates them on the most vulnerable students, and offers no way to audit its reasoning. The defensible move is to strip these detectors of any disciplinary authority and redesign assessment around work teachers can actually watch being made, in-class writing, drafts, revision, and conversation about the ideas. Switch the detector off as a weapon. Keeping it is not a defense of academic integrity; it is a decision to purchase false certainty and bill the cost to the students who can least afford it.

Take it further