Cloudflare and Copilot move the AI-to-web boundary in opposite directions
Cloudflare has added a discover mode to AI Search that removes the requirement that a site publish a sitemap, and the service runs crawling, ingestion, embedding and retrieval from a single command. Microsoft now requires a sign-in for anyone who goes to copilot.microsoft.com or presses the Copilot button on the Bing homepage. Copilot Search answers inside Bing results remain visible without signing in. These are two separate systems. One widens what of the open web gets crawled. The other closes what can be used without an account.
Artificial Intelligence··Midday
Cloudflare AI Search indexes sites without a sitemap
Cloudflare has added a discover mode to AI Search, its service for letting agents and applications search custom data. The mode removes the requirement that a site publish a sitemap. The service runs crawling, ingestion, embedding and retrieval from a single command.[1]
A public endpoint searches several sites without authentication
The service supports multimodal search, and a public endpoint requiring no authentication allows searches across several sites at once. There is also a plugin for users of the EmDash content management system. Cloudflare's own API documentation, developer documentation and open-source projects such as Astro, Vite, Hono and Replicate can be indexed and searched as a single corpus. The service is free during beta. Under the pricing the company describes, embedding and re-ranking stay free with default or selected Workers AI models at general availability, and answer generation and query rewriting are billed by model usage.[1]
Copilot now requires a sign-in for direct access
Users who go to copilot.microsoft.com or press the Copilot button on the Bing homepage must now sign in. Copilot Search answers that appear inside Bing search results remain visible without signing in. Krishna Madhavan, principal product manager for Bing at Microsoft AI, said that going forward all Copilot users are required to log in. Michael Schechter, corporate vice president of search growth and experiences at Microsoft, said Copilot Search does not require signing in.[2]