Kids and AI: What Convenience Takes Away

Two numbers, one product

100% of homework completed for the student. 29% of explicit suicide references missed. Both numbers come from the same study, on the same product, tested the same week.

We usually file these under two separate headings: academic cheating on one side, child safety on the other. This piece makes the case that it is one mechanism, viewed from two opposite ends.

What the study actually found

Common Sense Media, the US nonprofit, stress-tested the AI features built into Google Search, AI Overview and AI Mode, across more than 2,600 interactions. The verdict: "unacceptable risk" for children. Both tools violated seven of the organization's eight safety principles, and crossed all five of its red lines.

AI Mode completed 100% of the homework a student should have done alone: 180 math problems and essays, finished without hesitation. AI Overview missed 29% of explicit references to suicide, and half of the indirect or coded ones. Both tools pointed test users toward an eating-disorder helpline that has been disconnected since 2023.

One point worth being precise about, since precision is this outlet's whole job. That 29% figure belongs to AI Overview, not to "AI" as a category: AI Mode actually catches more in several categories. On disclosures of substance abuse, for instance, AI Mode surfaced a helpline or medical guidance 77% of the time, versus 63% for AI Overview. The gap shows these safeguards exist, but they are patchy and inconsistent.

Google, for its part, disputes the methodology, calling it built on "ambiguous and artificial" prompts. That objection matters. It doesn't change the mechanism we're about to lay out.

These features are built into Google Search, impossible to turn off, and they are landing in France this summer. Common Sense goes as far as recommending elementary schools ban the search engine outright.

The friction was the point

Here's the pivot. When an adult asks AI to summarize a report or draft an email, it removes a friction, time, effort, tedium. That's a clean win. The adult already knows how to write an email; the machine just spares them a move they've already mastered.

For a child, that same removal touches something else entirely. The friction being taken away wasn't a chore sitting next to the learning, it was the learning. Wrestling with a math problem, hunting for the right words, getting stuck, starting over: that's where the wiring forms. Stripping effort out of a brain that's still under construction doesn't do it a favor. It skips the lesson.

One end: homework

We've covered the Brown study here before: students scoring 96% with AI assistance and 48% without. The number went viral as proof that AI makes people dumber. The real story is more precise, and more uncomfortable.

The homework was never the point. Nobody, in real life, needs the answer to exercise 4 on page 32. The assignment is a pretext, a device that forces the brain to make the trip. The grade measures the trip, not the destination.

When AI delivers the destination without the trip, the grade stays high and the learning vanishes. You end up with students turning in flawless work having learned nothing. It's the same trap as a GPS that gets you there without you ever remembering the route: fine, as long as the device works. The day it fails, you find out you never actually learned to drive.

The other end: distress

The second story seems to have nothing to do with the first. It has the exact same skeleton.

Before AI, a struggling teen who wanted to talk had to get past a wall of awkwardness. Approach a teacher, work up the nerve to tell a parent, dial a helpline. That wall was unpleasant. But the friction did invisible work: it routed the teen toward someone responsible, an adult who could notice, flag it, act.

A teacher spots homework that's gone off the rails. A parent hears an unusual silence at dinner. A trained hotline operator knows how to ask the question that opens a door. None of that lives inside a search box. The wall of awkwardness was also a signpost, and nobody noticed until it was gone.

The AI bar removes the wall. The question gets typed into a search box, the answer comes back in a second, with no witness. The friction disappears, and with it the filter that led to a human being. When the answer is right, time is saved. When it misses nearly a third of explicit signals, or points to a dead number, the child is left alone with a machine that never saw them.

The difficulty wasn't a bug

The adult instinct, faced with friction, is to erase it. That's practically the definition of good product design: fewer clicks, less waiting, less effort. An entire industry was built on that instinct, and it's often right.

But childhood isn't a product to optimize. The struggle of learning, the discomfort of asking for help, the slowness of understanding: these frictions look like flaws. They were the function. They're how a child builds a mind and finds the way to an adult.

AI is entering kids' lives like a hand smoothing everything in its path. The real question isn't whether to pick a side between danger and genius. It's deciding, deliberately, what to leave rough for the people who are still being built.