Schools are using broken lie detectors on students

AI writing detectors like those built into Turnitin don't feel like powerful classification systems. They feel like helpful teaching tools. A professor uploads student essays, the system scans them, and suspicious ones get flagged for review. Simple, right? But that simplicity masks something more complex happening underneath. These tools are becoming what researchers call 'classification infrastructure'—systems so embedded in institutional workflows that they become invisible and nearly impossible to remove. Just like how electrical grids or water systems fade into the background of daily life, AI detectors are becoming the hidden backbone of academic integrity policies. The problem is that unlike water pipes, these systems are making consequential decisions about students' futures based on flawed assumptions about how writing works.

The data on false positives is stark: AI detectors wrongly flag authentic student writing 61% of the time for non-native English speakers. Black students face false accusations at nearly three times the rate of white students. Students with ADHD and other neurodivergent conditions get flagged disproportionately because their writing patterns don't match the AI's narrow definition of 'human' prose. These aren't random errors—they're systematic biases that consistently harm the same student populations. The tools are essentially digital sorting hats that channel certain students toward suspicion and punishment while others pass through unscathed. What makes this particularly insidious is that these biases compound over time. Each false positive goes into a student's permanent record, affecting their academic standing and future opportunities.

The fundamental problem runs deeper than biased algorithms. These detection tools assume a world that no longer exists: one where writing is either purely human or purely machine-generated. But that binary collapsed the moment AI writing assistants became widely available. Today's students exist along a spectrum of AI collaboration. Some use ChatGPT to brainstorm ideas. Others use Grammarly for editing suggestions. Still others use AI to help translate thoughts from their native language into English. The detection tools can't distinguish between these nuanced forms of collaboration and wholesale cheating, so they flag everything that doesn't match their narrow model of solo human composition. This creates a perverse incentive structure where students learn to write worse—less clearly, less precisely—to avoid triggering the AI detectors.

Perhaps most troubling is how these systems respond to their own failures. Rather than questioning whether AI detection is fundamentally flawed, institutions double down by building secondary systems to manage the primary system's mistakes. They create appeals processes, human review boards, and elaborate policies for handling false positives. This infrastructure of infrastructure makes the original tools seem more legitimate and harder to remove. Companies like Turnitin acknowledge accuracy limitations in their fine print while marketing their tools as reliable solutions. The result is an institutional investment in a technology that everyone involved knows is unreliable, but which has become too embedded to easily abandon. Students caught in this system find themselves fighting not just a false accusation, but an entire institutional apparatus built around maintaining the fiction that these tools work.

The deeper issue isn't technical—it's conceptual. We're trying to solve 21st-century problems with 20th-century frameworks. Academic integrity policies designed for an era of solo typing and library research don't make sense when students collaborate with AI in legitimate, creative ways. Rather than building better lie detectors, we need to develop new approaches to academic integrity that acknowledge the reality of AI-assisted writing. This means moving from blanket prohibition to thoughtful integration, from detection-based enforcement to pedagogy-based guidance. The goal should be helping students develop their own voices and ideas, not catching them in an algorithmic trap that mistakes collaboration for cheating.