Research & Experiments
WebMCP Shopping Test: Search Filters Did Not Enforce the Cart
Eight live WebMCP runs selected the correct shoes. Another 36 checks found cart enforcement gaps. See the measured results and lessons for ecommerce brands.
Original research
Tests and experiments built around real observations
This section is reserved for first-hand experiments, benchmarks, and structured tests. Research is only published here when the methodology, scope, findings, and limitations can be documented without inventing data.
CAutomates Agent Utility Index
What the benchmark series is for
The Agent Utility Index tests tools that AI agents depend on and measures the cost of getting usable work done, not only the advertised cost of an API request. It is intended for AI engineers, agent builders, technical founders, and automation teams comparing retrieval, search, browser, enrichment, extraction, and research infrastructure.
Task-level economics
Measure cost per correct result, retries, token burden, evidence quality, and downstream usability instead of request price alone.
Provider selection
See where providers overlap, where difficult pages separate them, and how conclusions change on a fresh sample.
Routing research
Test if a multi-provider agent can beat a strong single provider before adding production routing complexity.
Research & Experiments
I Tested 5 Web Retrieval Methods on 50 Real Pages. One Won Overall, But Not Everywhere.
CAutomates Agent Utility Index Benchmark #001B tested Firecrawl, Bright Data, Tavily, ScrapingBee, and Direct Fetch on 50 real pages across five workload categories. Firecrawl won overall, but documentation, difficult pages, extraction quality, latency, and simple-first retrieval told a more nuanced story.
Research & Experiments
I Tested AI Web Retrieval Tools on Real Pages. Here Is What a Correct Result Actually Costs
Benchmark #001 tests Firecrawl, Bright Data, Tavily, direct fetch, and ScrapingBee on real web-retrieval tasks, then tests routing strategies on a separate holdout to measure cost per correct field, quality, evidence, and practical agent-infrastructure tradeoffs.