Franklin AI News Brief

AI providers plan a shared training-preference signal for websites

Key Takeaways

  • A proposed robots.txt signal aims to reduce separate AI-training rules, with different implementation dates across providers.
  • One preference across participating providers Website owners currently face a collection of company-specific choices about AI training.
  • CellCog’s report describes a proposed shared signal that OpenAI, Google and Microsoft say they intend to support.
  • The aim is to let a publisher express a training preference once for participating services.
  • The proposal uses the policy token _usageai-training in robots.txt.

One preference across participating providers

Website owners currently face a collection of company-specific choices about AI training. CellCog’s report describes a proposed shared signal that OpenAI, Google and Microsoft say they intend to support. The aim is to let a publisher express a training preference once for participating services.

The proposal uses the policy token __usage_ai-training in robots.txt. That is a named preference group, rather than a replacement for the entire file. CellCog explains that a wildcard group does not count as expressing this particular preference, and omitting the token does not communicate a training choice under the draft.

The rollout dates matter

The report separates the providers’ plans: OpenAI describes implementation in the coming months, Google points to early 2027, and Microsoft gives February 1, 2027. Those are future commitments, not evidence that the same behavior is already operating across every crawler today.

There is another distinction for publishers to retain. The proposed token concerns training preferences; it is not itself an instruction to stop a crawler fetching pages. A site may need separate crawler-specific settings for that different goal. Microsoft’s additional page-level and grounding preferences also have their own scope.

A proposal with practical limits

CellCog describes an individual Internet-Draft, not an already endorsed final IETF standard. It also notes provisions for more specific company controls and private agreements. A common signal therefore simplifies part of the conversation without making every provider’s implementation identical.

For a publishing team, the useful next step is an inventory: record the current crawler rules, the desired training preference and the implementation dates being monitored. Keep the distinction between an announced control and an observed implementation clear when explaining the change to editors or creators.

Our read

Franklin AI Take

The shared token is a coordination proposal with staggered implementation dates. Keep a dated record of each provider’s actual support before treating one preference as an operational change across the whole industry. Track implementation separately for each participating service during the transition.