A new kind of risk is creeping into knowledge work. It isn’t the traditional form factors like fraud or laziness (remember “quiet quitting”?). It’s subtler than that, and in some ways more dangerous: people producing work that looks like competence without having any actual competence underneath.
You’ve probably seen the shape of it. A memo that reads clean but falls apart under basic questions. A “deep dive” that’s mostly paraphrased secondhand writing. Code that demos well yet breaks the moment it hits production.
Before we get ahead of ourselves I think it’s worth calling out upfront that AI didn’t invent any of this. What it did was make it cheaper, more accessible and harder to spot.
I’ve started calling this the Fog of False Fluency. False fluency is the feeling or appearance of mastery created by fast, coherent output. The fog is what makes the gap hard to see. Hard for managers. Hard for teams. Hard for customers. Ironically, it’s often hard even for the person producing the work.
Here’s the core hypothesis:
As AI makes it easier to generate high-quality looking output, more people will do fewer of the hard reps that build real understanding. Over time, critical thinking atrophies like any other unused muscle, and we end up with more people in important roles who sound qualified on paper but have not mastered the nuance.
When output becomes easier, plenty of people stop investing in the slower, less visible work of building deep mental models, especially if nobody is rewarding that effort. And most organizations are not great at distinguishing polish from grounding.
The scary part is that this can go unnoticed for a long time. A lot of roles allow you to coast as long as you can produce something plausible. The failures show up later, usually when the problem gets weird.
Why AI changes the shape of the problem
We’ve been outsourcing cognition forever. Books externalized memory. Calculators reduced mental arithmetic. Search reduced recall requirements. All of that shifted what people learned and how they worked. These aren’t inherently negative developments, and most have actually enhanced what we are capable of by reducing the amount of menial effort required. They helped us source the desired “ingredients” faster and cheaper as we developed our recipe.
AI, however, pushes into new frontier because it doesn’t just retrieve information. It makes the entire meal for us. It gives you an output that looks complete, coherent, and confident, which is exactly the kind of output humans tend to trust by default.
There’s also a learning dynamic here that matters. In many fields, competence is built through the very friction AI helps us avoid. You wrestle with the problem. You get stuck. You make a mistake and have to unwind it. You absorb constraints the hard way. That process is annoying, but it’s also the very thing that makes you good. AI can let people skip straight to something that looks acceptable, which boosts short-term productivity but also shrinks the “apprenticeship” time that used to force understanding to develop.
Used well, AI is leverage. Used carelessly, it becomes a shortcut that hollows out hard-earned skill.
How false fluency forms
I think the fog comes from three forces that reinforce each other.
First, cognitive offloading. There’s long-standing research showing that when people expect information to be accessible later, they store less of it internally and remember where to find it instead (look up the “Google Effect” or digital amnesia). AI expands that pattern from facts into synthesis. You’re not only outsourcing “what is the answer,” you’re also outsourcing “how do we think about it.”
Second, automation bias. Humans overweight machine outputs and monitor them less, especially when the output sounds confident and time is scarce. That’s a known risk in a lot of automation-heavy domains, and it’s already being discussed in the context of AI decision support in medicine and other high-stakes settings.
Third, illusions of understanding. Most people feel like they understand complicated things until they’re forced to explain them step by step. That moment of explanation is where the illusion breaks, and where real learning starts. AI makes it easier to avoid that moment because it can always produce a plausible explanation on demand, even if you personally can’t reconstruct it.
When you combine those three forces with modern incentives, a predictable pattern emerges. Output gets faster and cleaner. Fluency gets rewarded. Verification gets treated as overhead. People do fewer reps. Understanding thins out. The org looks productive until it hits an edge case where judgment matters.
An early warning sign we’ve already seen
We’ve already watched fluent output fail in the real world. A clean example is what happened when lawyers filed documents containing citations that didn’t exist because they relied on AI-generated content without verifying. The text of course looked professional. But the failure was underneath the surface, in the grounding.
That example is useful because law is supposed to have strong verification norms. If it can slip there, it will slip anywhere the culture around verification is weaker, which includes most environments where speed is prioritized and accountability is a secondary consideration.
Where the fog gets dangerous
False fluency can be irritating in low-stakes work; however, it gets serious fast when the work is high-stakes, the environment is uncertain, or the system is complex, such as in:
Medicine. AI can reduce burden and improve access to information, which is a legitimate win. The concern is what happens over years if clinicians gradually rely on AI for diagnosis framing, triage thinking, or justification language. Most cases are routine and the tool will look “right.” The real danger shows up with atypical presentations, incomplete histories, interacting conditions, and edge cases where you need a strong internal model to notice that something is off. If independent reasoning has atrophied, subtle mistakes are easier to miss, and the human becomes less of a reliable backstop.
Engineering. Software is a perfect habitat for false fluency because output is easy to produce and hard to fully validate. AI can generate code that works in the demo. It can even generate decent tests. But complex systems fail in the corners: performance constraints, security edge cases, concurrency bugs, weird data, dependency issues, integration boundaries, and silent failures. Those aren’t “write me a function” problems, and they require deep mental models. A team staffed with people who can generate code without robust understanding will move fast while building fragility into the foundation, and debugging will become a crisis skill rather than a normal competency.
Leadership and Politics. Leadership already has a competence theater problem because people often confuse confidence with clarity. AI increases the supply of polished narratives and makes it easier to produce persuasive text tailored to different audiences. That amplifies a selection pressure toward rhetoric and away from tradeoff-driven reasoning. Over time, you can end up with leaders who are excellent at making things sound coherent and much weaker at navigating the messy constraints and consequences that real leadership is about.
Finance and Operations. AI will make it easier to build dashboards, memos, and models that look credible. In finance and ops, small misunderstanding compounds. If someone doesn’t have a strong grasp of the underlying assumptions, unit economics, constraints, or causal relationships, the output can be elegant but still very wrong. The most dangerous artifacts are the ones that feel complete, because they reduce the urge to check.
Second-order impacts of false fluency
The deeper consequences are structural.
Hiring gets noisier because polished output becomes less diagnostic. Decks get smoother. Interviews get easier to “study for.” Portfolios look impressive. The gap between surface fluency and real capability steadily grows, so the cost of figuring out who can actually reason goes up.
Competence can also start to polarize. Some people will get dramatically better because they use AI to amplify skills they already have and they keep doing the hard thinking. Others will become increasingly dependent because AI becomes a substitute for the reps that used to build their models. That barbell effect creates a fragile middle.
The organizational risk is brittleness. Many teams will look fine as long as the environment is stable and problems are familiar. When something shifts, a vendor breaks, a regulation changes, a new failure mode shows up, the market turns, or an edge case hits, you suddenly need deep understanding in the room. If nobody has it, the org risks coming apart at the seams..
What keeps you out of the fog
The point here isn’t to reject AI, in fact this should underscore just how powerful and effective the technology can be. The point is to emphasize using it in a way that builds competence rather than quietly replacing it.
I’ve found a few simple practices that help check my potential overreliance on any given tool where the space between my ears used to do the heavy lifting.
Two-pass work: use AI for a first draft, then verify and rewrite the reasoning in your own words. The rewriting step matters because it forces you to build the mental model rather than merely consuming the output.
Source discipline: if a claim matters, it needs a primary source and not a circular chain of summaries. Find real anchors for your work beyond the prompt windows.
No-tool reps: choose a slice of your work where you regularly rely on AI and do it manually every once in a while. That keeps the muscle alive and reveals what you’re most prone to outsourcing.
For teams, the highest-leverage move is cultural: reward reasoning and diagnosis, not just polished production. In interviews, give candidates flawed work and see if they can find what’s wrong. In reviews, ask what assumptions drive the conclusion, what breaks the model, and what evidence would change their mind. Make verification normal rather than going the extra mile.
How capable are we when the power goes out? AI is going to make output abundant. That’s the new baseline. The scarce advantage becomes something else: judgment, grounding, verification, and the ability to reason under uncertainty when the answer is not obvious.
In the fog of false fluency, the winners won’t be the people who can produce the most text. They’ll be the people who can still build and test real models in their head, and who treat AI as leverage rather than authority.