AI-Generated Image
You know what used to make something feel real?
A flaw.
A typo. A bad photograph. A voice that sounded a little too human to be a jingle.
AI reproduces those now too.
We spent the first years of generative AI asking whether a machine was capable of making something look convincing. I think we’re past that question. The one in front of us now is harder.
What happens when the signals we use to recognize authenticity become copyable themselves?
What Is the AI Authenticity Problem?
The AI authenticity problem is the growing difficulty of determining where digital content came from, who contributed to it, whether people know AI was involved, and who answers when an authenticity judgment causes harm. Detection addresses only one slice of that problem. Provenance, attribution, human judgment, appeals, and accountability address the rest. AI authenticity governance is the name I’d give to the whole stack: the technical systems, platform rules, regulation, and human judgment we’re building to decide, together, what deserves trust.
Authenticity Is Becoming Copyable
Years ago I used to think about the Xerox machine as a kind of philosophical device. You copied an authentic thing, sold the copy to a hundred thousand people, and the original stayed exactly what it was. The Mona Lisa on a tea towel doesn’t threaten the Mona Lisa in the Louvre.
AI breaks that comfort. The copy isn’t a copy anymore. It’s editable. Personalized. It gets tuned to sound like your favorite creator having a bad day, or a CEO admitting a mistake she never made. And it gets produced at a volume no forger ever managed by hand.
Here’s the part I keep turning over. AI doesn’t only reproduce polish. With the right prompting, it reproduces the tells. Typos. Awkward phrasing. The slightly too-long pause in a voice memo. Emotional cues we used to read as proof of a human on the other end.
I don’t think the interesting story is that AI content looks real. The interesting story is that the signals we relied on to spot the fake are themselves reproducible now. “I know it when I see it” was never a great authentication method. It’s about to get worse.
What happens when the copies are easier to find than the original?
The Crowd Is Part of the Governance System
Before we get to regulators and platforms, I want to talk about something less official: the audience.
You and me, scrolling. We decide who gets believed. Who gets ignored. Who gets reported, who gets cancelled, who loses a brand deal because a screenshot went around before anyone checked it.
That’s real power. It shapes reputation, distribution, and income, often faster than any court or platform review ever manages.
But audiences don’t run on evidence standards. There’s no appeals process for a pile-on. Most people responding to a clip have not seen the source material and are not going to go looking for it. Judgment arrives before investigation, most of the time.
I’m not writing this from outside that crowd. I’m in it. I’ve believed things at first glance that turned out to be wrong, and I’ve moved on to the next post before I found out.
The point isn’t that people are foolish. The point is that the audience has become an informal regulator without any of the institutional safeguards a regulator is supposed to have.
Authenticity is not the same thing as consensus.
What Happens When the Authenticity System Gets It Wrong?
Picture a creator who made something entirely her own. A classifier flags it as AI-generated. The label spreads faster than any correction manages. A platform acts. She appeals into a form that nobody reads for three weeks.
By the time anyone looks closely, the damage already happened.
This isn’t only a creator-monetization problem, either. Widen the frame and the stakes get sharper. A fabricated audio clip of an executive announcing a product recall that never happened. A doctored image of a public official at an event they never attended. A synthetic voice memo, close enough to a real business partner’s, authorizing a wire transfer that shouldn’t go through. None of these need to be common to matter. They need to be possible, cheap to attempt, and hard to unwind once they’ve moved.
I’m not saying the creator-flagging sequence above plays out every time. I’m saying it’s possible, and that possibility is the governance problem. A false authenticity judgment isn’t only a technical miss when it costs someone their income, their following, their business relationship, or their name.
That is the question this whole piece keeps circling back to. Not whether AI was involved. What happens when we’re wrong about it.
Five Problems AI Detection Alone Cannot Answer
I want to lay out five separate questions, because they get flattened into one all the time, and flattening them is where the confusion comes from.
1. Detection: do we know whether content was generated or altered by AI?
2. Attribution: if AI was involved, do we know who is behind the work?
3. Judgment: who decides whether something counts as authentic?
4. Due process: what happens when that decision is wrong?
5. Accountability: who answers for the harm when synthetic content causes it?
Answering the first question tells you nothing about the other four. That’s the trap. Most public conversation about AI authenticity stops at detection, as if a green checkmark or a red flag settles the matter. It doesn’t.
AI Detection: Do We Know If Content Was Generated by AI?
Not reliably, and not by itself. Researchers at CREST, the UK’s Centre for Research and Evidence on Security Threats, made a point this year that undersells itself by how plainly it’s stated: visual appearance no longer serves as a stand-in for authenticity. [1]
Three separate limitations sit behind that sentence, and I think each deserves its own look rather than a blur.
Detectors degrade in the wild. Deepfake detection tools perform well on benchmark datasets, then lose accuracy once they meet the mess of the real world, and the methods for evading detection improve alongside the methods for catching it. [1]
Human perception doesn’t bridge the gap. Research on synthetic faces has found that people struggle to tell them apart from real ones, and in some studies rate the synthetic faces as more trustworthy than the real ones. [1]
The framing itself is too narrow. Treating authenticity as a binary of fake versus real misses the texture of the actual problem. Some AI-assisted content is harmless, even valuable. Some unaltered, completely real footage gets stripped of context and turned into something misleading. Detection was built to answer a narrower question than the one we need answered. [1]
That pushes the field toward a different question. Not “does this look synthetic,” but “where did this come from, and what happened to it along the way.”
What Is the Difference Between AI Detection and Content Provenance?
Detection asks what something looks like. Provenance asks where it came from. That distinction is the technical core of this entire topic.
What is content provenance? Content provenance is information about where digital content came from and how it was created or modified. Provenance helps someone evaluate an asset’s history. It doesn’t establish whether the underlying claims in that asset are true.
What is C2PA? C2PA, the Coalition for Content Provenance and Authenticity, is a standards body whose members include Adobe, Microsoft, Google, OpenAI, Sony, and the BBC. It maintains a technical specification for attaching a cryptographically signed provenance record to a piece of media.
What are Content Credentials? Content Credentials are C2PA’s implementation of that idea, presented to users as a manifest. A Content Credential notes the device or software that produced a file, whether AI was involved, and what editing happened afterward. [3]
Here’s the sentence I think matters most in this entire topic. C2PA is explicit that Content Credentials do not make a value judgment about whether the underlying claims are true. They confirm whether the provenance data is intact, unaltered, and tied to a signer on a trust list. [3] A credential provides evidence about where a file came from, when relevant assertions were made, and what happened to the asset afterward. It doesn’t tell you whether the person behind the content was honest.
Disclosure tells you something happened. Detection estimates whether something happened. Provenance records what happened, as reported by whoever signed it. None of those three is the same as truth.
There’s a practical fragility here too. A screenshot, a re-encode, a print-and-scan, a repost through five platforms- any of these strip the metadata clean off. The field’s response has been layering: a manifest paired with an imperceptible watermark and a content fingerprint, so that if the metadata gets stripped, the watermark gets used to look the record back up. Manifests get removed. Watermarks get attacked. Fingerprints stay ambiguous. Combined, the three cover more of each other’s gaps than any one does alone. None of them, alone or combined, closes the loop entirely. [1] Independent security researchers examining C2PA this year went further, arguing the standard’s current guarantees are narrower than its marketing suggests and shouldn’t yet be treated as sufficient for high-stakes uses like legal evidence.
Provenance is a real advance over pure detection. It is not a verdict machine. Nobody serious is claiming it is, though plenty of secondhand coverage implies it.
Authenticity, Truth, and Trust Are Different Questions
Worth separating these three explicitly, because the article keeps needing all three and they get used interchangeably online.
Authenticity asks where something came from.
Truth asks whether its claims are accurate.
Trust asks whether you have sufficient reason to believe it.
A credential answers the first question. It doesn’t answer the second on its own, and C2PA says as much. [3] The third question, trust, is where human judgment, reputation, and accountability all have to do work that no manifest does for them.
The Trust Stack
I keep coming back to a short list of verbs, because I think they’re doing more work than people give them credit for.
Disclosure tells us AI was involved.
Detection estimates whether AI was involved.
Provenance records origin and transformations.
Attribution identifies who contributed.
Accountability assigns responsibility.
Governance decides what happens next.
Call it the trust stack, if you want a name for it. Six layers, and every one of them addresses a different failure mode. Disclosure fails when nobody bothers to label anything. Detection fails against a determined adversary. Provenance fails when the metadata gets stripped on the way through five reposts. Attribution fails because a manifest proves a file passed through a tool without proving who was accountable for what it said. Accountability fails when nobody agrees who controlled which stage of the pipeline. Governance fails when the appeal process takes three weeks to answer a claim that spread in three minutes.
No single layer solves the whole problem. That’s not a flaw in the design. That might be the honest shape of the problem.
What Does the EU AI Act Say About AI-Generated Content?
Article 50 of the EU AI Act establishes transparency obligations for certain AI systems and certain AI-generated or manipulated content. The rules apply from August 2, 2026. [2] They include machine-readable marking requirements for providers of systems that generate synthetic audio, image, video, or text, and disclosure obligations for deployers involving deepfakes and AI-generated or manipulated text on matters of public interest.
Systems that interact directly with people, like chatbots, carry their own obligation: providers need to disclose that a person is talking to an AI, unless that’s already obvious. Deployers using emotion-recognition or biometric-categorization tools need to flag that use too. [2]
Certain AI systems already placed on the market before August 2, 2026 receive a limited transition for the Article 50(2) marking obligation, running to December 2, 2026. Everything else, including the direct-interaction and deepfake-disclosure duties, started on schedule. [2]
Here’s where I want to be careful, because this is exactly the kind of claim that gets flattened in the retelling. Article 50 does not require every AI-generated post on the internet to carry a label. It applies to specific providers and deployers in specific circumstances, weighted toward systems that reach individuals directly or touch matters of public interest. It’s a transparency law. It tells people when they’re looking at something synthetic or interacting with a system rather than a person.
It does not verify that the content is honest, accurate, or fair. Transparency and authenticity are cousins, not the same relative.
How Are Platforms Deciding What Counts as Authentic?
YouTube has built its own system for deciding what counts as authentic enough to monetize.
In July 2025, the company renamed its “repetitious content” monetization rule to “inauthentic content.” YouTube’s policy requires original and authentic content, and it excludes material that is mass-produced, generic, repetitive, or manipulative. [5] YouTube was clear that the substance didn’t change much; this content was already ineligible for the Partner Program, but the renaming told you where the platform’s head was at.
A year later, in July 2026, YouTube split the policy into three named categories: generic or template-based videos, content designed to be unsatisfying or manipulative, and AI personas presenting themselves as human experts on health, legal issues, finances, or politics. [5] YouTube has said repeatedly that AI tools themselves aren’t the target. A creator using AI to draft, dub, or animate stays eligible, provided a human is adding something the AI alone didn’t produce. What loses monetization is the mass-produced, templated, low-effort version of the same idea, repeated at scale.
X made a parallel move on the money side of the platform. Its Original Content Rewards Program, with terms effective August 7, 2026, replaced the older Revenue Sharing plan and requires an eligible Premium subscription, an account in good standing, and identity verification. [6] What counts as “original” is spelled out in the terms: work a creator wrote, filmed, or designed themselves, or someone else’s material meaningfully analyzed, edited, or added to. A caption slapped on a downloaded clip doesn’t clear the bar anymore. [6]
Neither company has declared a universal definition of authenticity. Both have made originality something with a direct line to a creator’s bank account. That’s worth sitting with. Platform governance isn’t only about what stays up and what gets taken down anymore. It’s increasingly about what gets paid and what doesn’t, and that turns “is this authentic” into an economic question as much as a moral one.
Who Decides Whether Content Is Authentic?
Detection, attribution, and provenance are technical questions with technical answers, incomplete as those answers are. Judgment and due process are governance questions, and governance questions don’t resolve themselves with better cryptography.
Depending on the day, the deciding party might be the creator, the audience, a platform’s automated classifier, a human reviewer, an AI provider’s disclosure system, or a regulator applying a law written before the specific case existed. None of those actors has a monopoly on being right.
What Happens When an AI Authenticity Decision Is Wrong?
The proportionality question deserves more attention than it gets. Not every authenticity decision carries the same weight. A low-stakes automated label, “this post includes AI-generated elements,” doesn’t need a human in the loop every time. A decision that ends someone’s monetization, or accuses a real person of saying something they never said, sits in a different category of consequence entirely.
The rough principle I’d propose: the larger the consequence of getting an authenticity call wrong, the stronger the requirement for a human to have looked at it directly, and the more that person owes the accused a real, documented reason and a real way to push back.
I want to be honest about why this is hard rather than pretend it isn’t. Platforms process authenticity judgments at a volume no human review team matches. Millions of appeals would drown any system built to hear them individually. Proportionality isn’t a solved answer here. It’s the shape of the actual problem, dressed up as a solution.
Accountability runs into a similar snag. When synthetic content causes real harm, a fair number of actors plausibly answer for it: the creator, the AI provider, the platform that hosted it, the advertiser that funded the reach. I don’t think there’s a universal answer to who’s on the hook. I think the better question, asked case by case, is who could prevent the harm, and who had the responsibility to act and didn’t.
This is also where I’d point back to something I’ve written about before: that human judgment becomes the scarce resource once the mechanical parts of a decision get automated. The Death of Busywork Is Creating a Judgment Economy makes that argument in a wider business context. Authenticity governance is one of the sharpest examples of it I’ve come across since.
What If We Overcorrect?
I don’t want to leave the impression that more scrutiny is automatically the answer.
If platforms and regulators demand proof of human authorship for everything, they risk disadvantaging creators who rely on AI for entirely legitimate reasons. Translation tools. Captioning. Accessibility software for people with speech or motor limitations. Synthetic voices used because a creator lost their own. AI-assisted editing that turns a rough recording into something publishable on a limited budget.
None of that is inauthentic in any meaningful sense. But a governance system built to catch bad actors doesn’t always distinguish well between someone gaming the system and someone who needs the tool to participate at all.
Where do we draw the line between “this creator used AI to produce something dishonest” and “this creator used AI to do something they couldn’t otherwise do”? I don’t have a clean answer. I’d rather name the tension than paper over it, because I think it’s the same tension that shows up in a lot of platform enforcement stories, the ones about opaque decisions and creators left guessing what tripped the system in the first place. That’s the terrain I got into in Shadowbanned or Self-Censored?, and I think it applies directly here.
What This Means for Creators
A few practical implications, stated plainly rather than dressed up as a listicle.
Your AI workflow wants documentation, even informally. Not because anyone’s checking today, but because the norms are visibly moving in that direction.
Your original contribution matters more than the tool you used. Platforms are already rewarding demonstrable human input over volume.
Provenance data doubles as useful evidence for you, not only a compliance burden. A record showing what you made and how you made it works in your favor if you’re ever wrongly flagged.
Your audience is increasingly forming opinions about how transparently you use AI, whether or not any platform or regulator asks them to.
What Creators Might Want to Keep
Not a legal requirement. A practical trust habit, for anyone whose income depends on being believed.
Your original idea, in whatever form it started.
Research sources and reference material.
Drafts along the way, not only the final version.
Prompts, where they meaningfully shaped the output.
Major editorial decisions and why you made them.
Original photographs or recordings, unedited.
The human revisions you made after AI involvement.
Final approval, and who gave it.
If you use AI in your creative work, you don’t need to document every keystroke. But you might want a simple record of where your ideas, judgment, research, and decisions entered the process.
I made a free Human Provenance Quick Check for exactly this purpose. It takes about five minutes and gives you a simple way to document your contribution, AI’s role, what you verified, and what evidence you retained.
Get the free Human Provenance Quick Check
Will Human Provenance Become a Creator Advantage?
Here’s a speculative thought I want to flag as exactly that. Speculative.
As AI-assisted production becomes the water everyone swims in, “I never touch AI” stops being a credible claim for most working creators, and it stops being much of a differentiator either. What starts to matter instead is a creator’s willingness to be specific about their own contribution. Where the idea came from. What source material they used. What they changed, and why, and what they’re personally accountable for in the finished piece.
Call it human provenance, though I want to be clear this isn’t an established standard the way C2PA is. It’s a way of thinking about what creators might offer as a trust signal once “made with AI” stops being news. Human-led. AI-assisted. Sources verified. Creator accountable. I don’t know if that becomes a real convention or a nice idea that never catches on. Worth watching either way.
Who Gets to Decide What Is Real?
Let’s go back to the question this piece keeps orbiting.
Who gets to decide?
The detector, some days, though we’ve seen what happens when it’s wrong.
The platform, on a different day, weighing monetization against brand safety.
The regulator, applying rules written to survive more than one product cycle.
The crowd, moving faster than any of the above and answering to none of them.
The creator, who made the thing and knows more about it than anyone else and still isn’t the last word.
None of these systems is complete on its own. I don’t think any combination of them will ever be complete either. What’s worth building toward is systems where a decision comes with a reason attached, where the evidence behind that reason is available to the person it affects, where the consequence is proportional to the confidence behind the call, and where there’s a real door to walk through if the system got it wrong.
We spent the early years of generative AI asking whether a machine was able to make something look human.
The uncomfortable part is we’re building the institutions meant to answer the harder question while still figuring out who deserves the trust to run them.
I think the harder question is already here. What happens when machines get good enough at looking human that we need institutions to tell us what we’re looking at?
And who do we trust to build those institutions?
Gregory H. Bourne writes about AI, governance, culture, business, and the human consequences of technology.
Sources and Further Reading
[1] CREST Security Review. “From Detection to Provenance: Securing Media Authenticity with Content Credentials.” By Shruti Agarwal and Sophie Nightingale. Published April 15, 2026. crestresearch.ac.uk
[2] European Commission. “Transparency obligations under Article 50 of the AI Act” and “Quick Facts: Transparency rules for AI systems.” Guidelines adopted July 20, 2026, obligations effective August 2, 2026. digital-strategy.ec.europa.eu
[3] Coalition for Content Provenance and Authenticity (C2PA). “C2PA and Content Credentials Explainer.” c2pa.org / spec.c2pa.org
[4] Independent security analysis of the C2PA specification, 2026. arxiv.org
[5] YouTube. “YouTube Channel Monetization Policies.” Updated July 15, 2025 and July 16, 2026. support.google.com/youtube
[6] X. “Original Content Rewards Program Terms.” Effective August 7, 2026. legal.x.com
[7] Google Search Central. “A new resource for optimizing for generative AI in Google Search” and “Creating Helpful, Reliable, People-First Content.” developers.google.com