Your CDN Just Set Your AI Policy. Did Anyone in Ecommerce Approve It?

Your AI agent access policy is being written by CDN defaults. Here's who should own it, how to govern search, agent and training bots, and what changes when they start to buy.

Here is the short answer. Your AI agent access policy, meaning the rules for which AI bots and assistants can browse your catalogue, cite your products and complete purchases, is being set right now by CDN defaults and security rules. Your ecommerce team is mostly not involved. That is a governance gap, and the fix is organizational more than technical. Name one person who owns agent access. Give them a decision matrix that covers search, agent and training bots. Review it every quarter against what AI engines actually say about your products.

If you want to go deeper on what to allow, we covered that in what AI shopping agents actually read at Black Friday. This piece covers the question that comes first: who decides, and how often.

What changed: your CDN now has an opinion about AI agents

Cloudflare's July 1 changelog describes the three AI crawler categories. Search crawlers index content to answer questions later, and the site should expect referrals in return. Agent traffic acts in real time for a person, such as chat fetch bots and browser-use tools. Training crawlers take content to train or fine-tune a model. Every plan, including Free, can block each one sitewide, block it only on pages that show ads, or leave it open.

The September 15 date is the part that matters for governance. From that day, new domains onboarding to Cloudflare start with Training and Agent blocked on ad-displaying pages and Search allowed. Cloudflare's September 15 follow-up post went further. It added a "Disallow AI Training" setting and changed what "Block" means: Block and "Block on pages with ads" now cover every training crawler, mixed-use crawlers included. A mixed-use crawler is one bot that does both search and training, and Googlebot, Bingbot and Applebot are the obvious examples. If someone picks the bluntest option without understanding that, a decision meant to protect content could also cut you off from search crawlers.

Cloudflare's own numbers show why this was never just a security question. Cloudflare's blog post "The Internet has a second audience" puts it plainly: fewer than 1% of sites on Cloudflare block search crawlers, while 17% block training. Almost nobody wants to disappear from search. A sizeable minority wants to limit training. The defaults now sit between those two positions. Cloudflare fronts about a fifth of the web, as TechCrunch reported when the change was announced in July, so its defaults effectively set the rules for a large share of online stores.

Cloudflare is not the only vendor in this position. Akamai, Fastly, Shopify apps, WordPress security plugins and in-house WAF rules all make the same kind of choice. Each layer can say yes or no to an AI bot, and each one is usually owned by someone whose job is reducing risk, not growing revenue. In practice, your AI crawler rules end up spread across several layers with no single owner.

Why agent access is a revenue decision, not a firewall setting

Adobe reported that traffic from generative AI tools to U.S. retail sites rose 693.4% year over year in the 2025 holiday season. On its own blog, Adobe said AI referrals "moved from lagging to leading, converting 31% more than other traffic sources." Into 2026, Adobe Analytics' 2026 Q2 AI Traffic Report, published April 16, 2026, found AI traffic to U.S. retail sites grew 393% year over year in the first quarter of the calendar year and converted 42% better than non-AI traffic in March 2026, a record, after converting 38% worse in March 2025. Most of that traffic comes from ChatGPT, as we covered in why ChatGPT drives most AI shopping traffic.

Akamai, which sells bot management and therefore has every reason to sell blocking, has argued the opposite. In its January 2026 AI Pulse predictions, Akamai said blocking will shift to a competitive disadvantage. It described the sensible approach as selective access based on who is behind the traffic and what it is trying to do. The same post noted that AI bots and agents "go where they are wanted." It also predicted that "Why are all the AI agents going to our competitors?" would become a familiar complaint inside companies.

Most retailers have not made an active choice either way. In a July 15, 2026 press release for its commerce State of the Internet report, Akamai said commerce organizations placed more than 90% of their AI bot activity in the "monitor" category. Monitoring is a sensible starting point. Leaving traffic there for months stops being a strategy and turns into a default that nobody has looked at.

The evidence on what blocking costs is still thin, but it points one way. An observational study of 1,058 domains by cloro.dev found that sites blocking PerplexityBot had a median Perplexity citation score of zero, against 1.167 for sites that allowed it. The study cannot prove causation. Still, the effect was specific to each engine, which is what you would expect from a real access effect. One small-business example shows how invisible the problem can be. A Brooklyn law firm found that a default Cloudflare setting had blocked GPTBot, PerplexityBot and ClaudeBot for seven months. After it unblocked them on July 3, 2026, visits from AI tools went from 36 in the five months before to 126 in the four weeks after. It is not an ecommerce brand, and the figures are self-reported. The lesson still applies to any store: you cannot see traffic that never arrives.

Search, Agent, Training: three decisions, not one toggle

Here is the shortest useful version.

• Search bots build the indexes AI answers cite. Examples: OAI-SearchBot (ChatGPT search), Claude-SearchBot, PerplexityBot. Block them and you largely drop out of that engine's answers.
• Agent fetchers visit a page because a person asked. Examples: ChatGPT-User, Claude-User, Perplexity-User and agentic browsers, some of which go on to act autonomously by adding to cart or checking out. OpenAI's documentation says robots.txt rules may not apply to ChatGPT-User because a user starts those actions. That is one more reason the real decision sits in your CDN and WAF, not in a text file.
• Training crawlers collect content for AI training. Examples: GPTBot, ClaudeBot, CCBot, plus Google-Extended as a training opt-out token. This decision shapes what models "know" about your brand over time. It does not decide whether you are cited today.
• Mixed-use crawlers such as Googlebot, Bingbot and Applebot do search and training in one bot. This is where blunt settings cause the most damage.

Thinking in search vs. agent vs. training terms keeps the decision out of the single "all AI bots" bucket that blunt settings encourage.

The decision matrix for your AI agent access policy

Use this as a starting template. The defaults are a reasonable starting point for a typical DTC or mid-market retailer. They are not a rule for every business.

Meet the agent access owner

In most companies this should be the Head of Ecommerce or someone they delegate to. It should not default to the CISO, and most CISOs would rather not own a merchandising call anyway. Security still matters a great deal here. It owns detection, verification, credential management and incident response, and it should hold a veto backed by evidence of abuse. But security should not be setting merchandising policy by leaving defaults untouched.

What matters is clear decision authority and clear accountability. The owner defines which bots get in, what they may do and why. Security handles enforcement and keeps those boundaries tight. CX owns the escalation path when an AI-made purchase goes wrong and a shopper gets in touch.

This role sits naturally next to the people already running AEO and GEO. If you are still working out who that is, our guide on who should run AEO and GEO on your team covers the staffing models. CX needs a seat as well. AI-made purchases create their own returns, disputes and "my AI ordered the wrong size" tickets, and we catalogued the support tickets agent purchases create. That is also why Alhena approaches this from between ecommerce and CX. We do not sell bot blocking. We see what happens on both sides of the decision.

Verified agents make "allow" safer, not optional

Visa launched its Trusted Agent Protocol on October 14, 2025, built with Cloudflare on top of the Web Bot Auth standard. Cloudflare's October 14, 2025 press release said Mastercard is incorporating Web Bot Auth into Mastercard Agent Pay and that American Express will use it in its agentic commerce program. In the same release, Luke Gebb, EVP and Head of Global Innovation at American Express, said the company was "excited to utilize Cloudflare's Web Bot Auth Protocol to help merchants identify trusted agents." In December 2025, Akamai and Visa announced that Trusted Agent Protocol would be supported across Akamai's platform. The goal is to help merchants tell whether a bot is browsing or paying, and to link it to the consumer it represents. For background on how these standards fit together, see how ACP, UCP and MCP fit together.

For governance, this has a practical consequence. "Prefer signed agents on checkout paths" is now a real rule someone can write, approve and enforce. But someone has to own it.

When agents act: from access to authorization

Agentic commerce is moving past browsing toward software that executes tasks: building a cart, applying an offer, paying. Autonomous AI agents now interact with stores through product pages, APIs and standards such as ACP, UCP and MCP, and through payment rails built for external agents. Each step from reading to acting raises the stakes. Securing AI agents at checkout is where AI security and merchandising stop being separate conversations.

Identity first, then scoped permissions

Authentication answers who an agent is. Authorization answers what it is allowed to do. Web Bot Auth, which also underpins Visa's standard, covers the first part: the bot proves its identity with a cryptographic credential instead of a user-agent string anyone can copy. The second part needs explicit permissions. Google's UCP handles some of it through Identity Linking, which uses OAuth so a shopper can link a retailer account and authorize specific scopes, such as managing a checkout or reading orders, and the merchant can specify which operations need a linked account. Payment delegation adds one more layer. Mastercard says its agentic tokens can be restricted by agent, merchant, category, spending limit and timeframe, and Google's AP2 payment standard records what a shopper authorized in signed mandates.

The governing principle is least privilege. Every agent works inside defined boundaries, with the narrowest permissions it needs for one task, for the shortest time that works. In practice that means OAuth tokens that expire quickly, so a leaked token is worth little. It means no shared API keys, service accounts or other credentials handed to external services, and revocation that works the moment a shopper or your team withdraws consent.

Treat every such credential as having a lifecycle, from issue to revocation. Security teams already manage human access with role-based (RBAC) or attribute-based (ABAC) access control, where attribute rules weigh context such as basket value or account age. Bots need the same strict discipline, tied to your data classification and applied at runtime to every API call they make. The more autonomous the agent, the less it should be allowed to do, and nothing external should ever inherit a staff session, an admin token or a direct line into your customer database.

Guardrails for high-impact actions

Not every action carries the same blast radius. Reading a product page is low risk. Applying a loyalty discount, changing a delivery address, deleting a saved payment method, issuing a refund or completing any financial transaction is not, and some of those are irreversible. Set a threshold, such as a basket value or any request that touches PII or stored payment details, above which even an authenticated agent can't execute the action until a human approves it (human-in-the-loop, or HITL) or the shopper explicitly confirms it. Add rate limits so one bot can't hoard inventory or hammer checkout. And keep a kill switch, so security can deny or revoke a bot immediately rather than waiting for the next quarterly review.

The third row is where legal and privacy teams have a stake. Anything that can read order history touches compliance with regulations on personal data and data retention, so privacy and compliance teams should sign off on it before it goes live.

Two threats to know by name

Prompt injection is when text an AI reads, in a review, a product description or a hidden page element, tries to steer its behavior. OWASP ranks it as the top risk for applications built on LLMs. Privilege escalation is when a bot, or someone posing as one, gains permissions it was never granted, often through a broad session it inherited by mistake. Both belong in your threat model. Both argue for keeping actions narrowly scoped, with anything not granted denied by default, and keeping every sensitive request on record, so unauthorized or unintended activity shows up quickly.

A structure for the review

If you want an outside framework, NIST's AI Risk Management Framework makes Govern one of its four core functions, alongside Map, Measure and Manage. It fits this work well: the owner governs, security and CX map the risks, AI Visibility and your audit log measure outcomes, and the quarterly review manages change. Policies define who may do what, and the review checks whether that still holds. That record is also what gives the program auditability. When a partner, a regulator or your CFO asks why a bot was allowed in, the answer should be one search away.

Alhena's own on-site AI is deployed the same way. Deal and discount lookups run through read-only tools, out-of-scope requests escalate to a human with the conversation context attached, and Alhena lists SOC 2, ISO 27001, GDPR and HIPAA among its security and privacy credentials.

How to measure whether it's working

Change a setting, then watch whether ChatGPT, Gemini, Perplexity and Google AI Overviews mention your products more or less, and whether your product cards show up in the answers. Alhena AI Visibility tracks this at the SKU level. It shows which prompts cite you, which competitors appear next to you, and whether a product card appeared. That gives the owner a before-and-after view for every change instead of guesswork. Put the change history and the visibility trend in the same quarterly review, and the debate moves from opinions to evidence.

Your first 90 days

Days 1–30: name the owner and map the control points

• Appoint the agent access owner in writing, with security and CX as named partners.
• List every layer that can admit or refuse AI traffic: CDN bot settings, WAF rules, bot manager, robots.txt, platform apps, security plugins and any agent-facing API or MCP server.
• Record the current setting for Search, Agent and Training on each layer, including anything that changed on or after September 15, 2026.
• Ask security to add AI bot traffic to the existing threat model.
• Take a visibility baseline: citations and product cards for your top 50 commercial prompts.

Days 31–60: run the first Search, Agent, Training review

• Go through the matrix one row at a time, decide, and record who signed off.
• Agree a change rule and approval workflow: no change to AI bot rules goes live without the owner's approval, a written reason and an entry in the audit log.
• Decide which automated actions need sign-off before they run, and which the owner can clear in advance.
• Set a holiday freeze. With peak season weeks away, lock settings by mid-November unless there is a live security incident.

Days 61–90: measure and make it routine

• Compare post-change visibility against the baseline.
• Test that revoking a token actually cuts the bot off.
• Put the quarterly review on the calendar, with a monthly check-in through Q4.
• Draft a one-page policy that new security or platform hires can read in five minutes.

FAQ

The bottom line

Your CDN is going to make this decision either way. The real question is whether someone accountable for revenue sets your AI agent access policy deliberately, writes it down and checks the results every quarter. Name the owner this week, and run your first Search, Agent, Training review before the holiday freeze. Then judge it on what AI engines say about your products, not on what the dashboard toggles show. When agents do arrive, Alhena's AI Shopping Assistant and AI Support Concierge are built to answer them, and the shoppers who follow, accurately.