M&A customer-loss signal retrieval benchmark
Battle: Web Access API for AI Agents · Dataset: 12 target-company customer-loss queries · Judge: ~openai/gpt-mini-latest
10/12
Top score: Keenable customer-loss signal hits
25–83%
Working-provider hit-rate spread
96
Provider × query judgments
$0.152
LLM judge cost through Battle proxy
Summary leaderboard
| Rank | Provider | LLM customer-loss match@10 | Mean signal score | Credible-source match@10 | Mean source score | Judge OK | Judge cost |
|---|---|---|---|---|---|---|---|
| 1 | Keenable | 10/12 (83.3%) | 86.1% | 10/12 (83.3%) | 86.1% | 12/12 | $0.0259 |
| 2 | Parallel | 9/12 (75.0%) | 82.8% | 9/12 (75.0%) | 80.1% | 12/12 | $0.0185 |
| 3 | Exa | 8/12 (66.7%) | 77.2% | 8/12 (66.7%) | 74.6% | 12/12 | $0.0204 |
| 4 | Perplexity | 6/12 (50.0%) | 58.2% | 6/12 (50.0%) | 52.8% | 12/12 | $0.0239 |
| 5 | Firecrawl | 5/12 (41.7%) | 58.2% | 5/12 (41.7%) | 59.7% | 12/12 | $0.0208 |
| 6 | BrightData SerpAPI | 4/12 (33.3%) | 44.3% | 4/12 (33.3%) | 33.3% | 12/12 | $0.0195 |
| 7 | Tavily | 3/12 (25.0%) | 47.0% | 3/12 (25.0%) | 36.1% | 12/12 | $0.0232 |
| 8 | You.com | 0/12 (0.0%) | 0.0% | 0/12 (0.0%) | 0.0% | 0/12 | $0.0000 |
Per-query × provider score heatmap
1.00 hit0.67 partial0.33 weak lead0.00 missunavailable
| Query / gold signal | Keenable | Parallel | Exa | Perplexity | Firecrawl | BrightData SerpAPI | Tavily | You.com |
|---|---|---|---|---|---|---|---|---|
| cust_loss_001 Salesforce lost customer Marq to HubSpot CRM migration saved 77K Gold: Marq switched/migrated from Salesforce CRM to HubSpot CRM and reported about $77K annual savings. | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.33 | N/A |
| cust_loss_002 Splunk lost customer UOL to Elastic Security migrated from Splunk Gold: UOL Security Team migrated from Splunk to Elastic Security. | 1.00 | 0.95 | 1.00 | 0.67 | 0.33 | 0.33 | 0.33 | N/A |
| cust_loss_003 Splunk customer N11 migrated to Elastic competitor case study Gold: N11 migrated from Splunk to Elastic. | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.00 | 1.00 | N/A |
| cust_loss_004 Segment lost customer Joybird to RudderStack migration customer data platform Gold: Joybird migrated its customer data platform from Segment to RudderStack. | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | N/A |
| cust_loss_005 Segment customer Mattermost migrated from Segment to RudderStack Gold: Mattermost migrated from Segment to RudderStack. | 1.00 | 1.00 | 1.00 | 0.33 | 0.33 | 0.00 | 0.33 | N/A |
| cust_loss_006 QuickBooks lost customer Webistry to Xero switched from QuickBooks to Xero Gold: Webistry switched from QuickBooks to Xero. | 0.33 | 0.33 | 0.33 | 0.00 | 0.33 | 0.00 | 0.33 | N/A |
| cust_loss_007 Salesforce lost customer Pusher to HubSpot migration Salesforce to HubSpot Gold: Pusher migrated from Salesforce to HubSpot. | 1.00 | 1.00 | 0.95 | 0.33 | 0.33 | 0.00 | 0.33 | N/A |
| cust_loss_008 Marketo lost customer Homesuite to HubSpot switched from Marketo to HubSpot Gold: Homesuite switched/migrated from Marketo to HubSpot. | 1.00 | 0.33 | 0.33 | 0.33 | 0.33 | 0.33 | 0.33 | N/A |
| cust_loss_009 Mailchimp lost customer TSB Living to Klaviyo migration case study Gold: TSB Living migrated its email list/marketing from Mailchimp to Klaviyo. | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.33 | N/A |
| cust_loss_010 Salesforce Commerce Cloud lost Rainbow Shops to Shopify moved from Salesforce Commerce Cloud Gold: Rainbow Shops moved from Salesforce Commerce Cloud to Shopify. | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | N/A |
| cust_loss_011 Zendesk lost customer MyHSA to Intercom migrated from Zendesk to Intercom Gold: MyHSA migrated from Zendesk to Intercom. | 0.67 | 0.33 | 0.33 | 0.00 | 0.00 | 0.33 | 0.33 | N/A |
| cust_loss_012 Mailchimp lost customer Pair of Thieves to Klaviyo migrated from Mailchimp to Klaviyo Gold: Pair of Thieves migrated from Mailchimp to Klaviyo. | 0.33 | 1.00 | 0.33 | 0.33 | 0.33 | 0.33 | 0.00 | N/A |
Detailed per-query judgments
| Item | Provider | Signal score | Source score | Matched source | Judge rationale |
|---|---|---|---|---|---|
| cust_loss_001 | Keenable | 1.0 | 1.0 | aptitude8.com case study PDF; hubspot.com case study | The results directly state that Marq saved $77k a year by switching from Salesforce to HubSpot CRM, with both an Aptitude 8 case study and a HubSpot case study supporting the same displacement signal. |
| cust_loss_001 | Parallel | 1.0 | 1.0 | aptitude8.com/case-study/marq-saves-77k-a-year-by-switching-from-salesforce-to-hubspot-crm-in-90-days | The results explicitly show Marq migrating from Salesforce to HubSpot with $77K savings, directly matching the requested customer-loss signal. |
| cust_loss_001 | Exa | 1.0 | 1.0 | HubSpot case study: https://www.hubspot.com/case-studies/marq | The results directly name Marq migrating from Salesforce to HubSpot, and also include a case study titled 'Salesforce to HubSpot Migration: Marq Saves $77K in 90 ...', which matches the requested customer-loss and savings signal. |
| cust_loss_001 | Perplexity | 1.0 | 1.0 | hubspot.com/case-studies/marq; YouTube result titled "Marq saves $77k a year by switching from Salesforce to HubSpot CRM in 90 days" | The results directly show Marq migrating from Salesforce to HubSpot and specifically mention $77k annual savings, which matches the requested customer-loss signal. |
| cust_loss_001 | Firecrawl | 1.0 | 1.0 | HubSpot case study / aptitude8.com case study (Marq saves $77K a year by switching from Salesforce to HubSpot CRM in 90 days) | The results explicitly state that Marq migrated from Salesforce to HubSpot CRM and saved about $77K per year, directly matching the requested customer-loss signal. |
| cust_loss_001 | BrightData SerpAPI | 1.0 | 1.0 | HubSpot case study / Aptitude8 case study (hubspot.com/case-studies/marq; aptitude8.com/case-study/marq-saves-77k-a-year-by-switching-from-salesforce-to-hubspot-crm-in-90-days) | The results directly state that Marq migrated from Salesforce to HubSpot CRM and saved about $77K per year, matching the requested customer-loss signal and providing credible source support. |
| cust_loss_001 | Tavily | 0.33 | 0.0 | pedowitzgroup.com / HubSpot Community | The results show general Salesforce-to-HubSpot migration content, but they do not name Marq or the $77K savings, so they are only partial relevance and not a direct customer-loss signal. |
| cust_loss_001 | You.com | 0.0 | 0.0 | Provider unavailable: http_error 402 | |
| cust_loss_002 | Keenable | 1.0 | 1.0 | elastic.co/de/customers/uol; theapplied.co/use-cases/how-uol-uses-elastic-ai-to-cut-security-incident-resolution-time-by-80-percent | The results explicitly state that UOL uses Elastic and that it migrated from Splunk to Elastic Security, which directly supports the requested Splunk-to-Elastic customer-loss signal. |
| cust_loss_002 | Parallel | 0.95 | 1.0 | https://www.elastic.co/customers/uol | The results include Elastic’s UOL customer story and multiple Splunk-to-Elastic migration/replacement pages, directly supporting that UOL moved from Splunk to Elastic Security. |
| cust_loss_002 | Exa | 1.0 | 1.0 | https://www.elastic.co/customers/uol (Elastic Customers) | The results directly name UOL and show an Elastic customer story plus multiple Elastic pages about replacing Splunk with Elastic for logs/security/observability, which supports the customer-loss signal from Splunk to Elastic. |
| cust_loss_002 | Perplexity | 0.67 | 0.5 | elastic.co/splunk-replacement | The results include Elastic’s own Splunk-replacement page referencing UOL and Elastic Security, which is useful evidence of a Splunk-to-Elastic customer switch, but the text does not explicitly name the UOL Security Team or state the migration from Splunk in the snippet itself. |
| cust_loss_002 | Firecrawl | 0.33 | 0.5 | elastic.co/splunk-replacement; elastic.co/blog/migrating-from-splunk-to-the-elastic-elk-stack-data-migration | The results show generic Splunk-to-Elastic migration and an Elastic replacement page, but they do not name UOL or directly evidence Splunk losing that specific customer to Elastic Security. |
| cust_loss_002 | BrightData SerpAPI | 0.33 | 0.0 | elastic.co/splunk-replacement | The results show only generic Splunk-to-Elastic migration/replacement pages and do not mention UOL or a specific Splunk customer loss, so the exact customer-loss signal is not supported. |
| cust_loss_002 | Tavily | 0.33 | 0.5 | elastic.co/splunk-replacement | The results show generic Splunk-to-Elastic migration messaging and an official Elastic page, but they do not name UOL or specifically confirm that UOL Security Team migrated from Splunk to Elastic Security. |
| cust_loss_002 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_003 | Keenable | 1.0 | 1.0 | https://elastic.co/customers/n11 | The top result is an Elastic customer story explicitly stating that N11 migrated from Splunk to Elastic, directly matching the requested loss-to-competitor signal. |
| cust_loss_003 | Parallel | 1.0 | 1.0 | Elastic Customers / https://www.elastic.co/customers/n11 | The results directly name N11 and state it migrated to Elastic Security, with the Elastic customer page serving as a primary source for the Splunk-to-Elastic displacement signal. |
| cust_loss_003 | Exa | 1.0 | 1.0 | https://www.elastic.co/customers/n11 | The results directly name N11 and show it migrating to Elastic, with additional Elastic customer pages explicitly describing customers dropping Splunk in favor of Elastic. |
| cust_loss_003 | Perplexity | 1.0 | 1.0 | elastic.co PDF customer story: Turkish ecommerce giant, N11, migrates to Elastic Security in just two weeks | The results directly state that N11 replaced its incumbent Splunk platform with Elastic Security, which cleanly matches the requested customer-loss/displacement signal and is supported by a primary Elastic customer story. |
| cust_loss_003 | Firecrawl | 1.0 | 1.0 | https://www.elastic.co/customers/n11 | Result [7] directly states that N11 migrated from Splunk to Elastic, which exactly matches the requested customer-loss signal and comes from Elastic's official customer story. |
| cust_loss_003 | BrightData SerpAPI | 0.0 | 0.0 | The results only show generic Splunk homepage and unrelated references, with no evidence that N11 migrated from Splunk to Elastic or any customer-loss/competitive-displacement signal. | |
| cust_loss_003 | Tavily | 1.0 | 1.0 | elastic.co/pdf/elastic-success-story-turkish-ecommerce-giant-n11.pdf | The top result is an Elastic success story explicitly stating that N11 replaced its incumbent Splunk platform and migrated from Splunk to Elastic Security. |
| cust_loss_003 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_004 | Keenable | 1.0 | 1.0 | RudderStack case study: https://rudderstack.com/customers/how-joybird-reduces-engineering-time-spent-on-customer-data-integrations-with-rudderstack | The top result explicitly states that Joybird moved its customer data platform from Segment to RudderStack, directly matching the requested customer-loss signal with a credible RudderStack case study source. |
| cust_loss_004 | Parallel | 1.0 | 1.0 | RudderStack customer case study / blog (rudderstack.com/customers/how-joybird-reduces-engineering-time-spent-on-customer-data-integrations-with-rudderstack/; rudderstack.com/blog/addressing-segments-limitations/) | The results explicitly name Joybird and RudderStack and include multiple Segment-to-RudderStack migration/case-study pages, directly supporting the customer-loss signal from Segment to RudderStack. |
| cust_loss_004 | Exa | 1.0 | 1.0 | RudderStack Joybird case study / Addressing Segments limitations (rudderstack.com) | The results directly name Joybird, Segment, and RudderStack in case-study and migration-related RudderStack pages, providing strong evidence that Joybird moved from Segment to RudderStack. |
| cust_loss_004 | Perplexity | 1.0 | 1.0 | RudderStack blog: Addressing Segments limitations; Joybird and RudderStack Case Study | The results explicitly state that Joybird moved its customer data platform from Segment to RudderStack, directly matching the requested customer-loss and competitor-displacement signal. |
| cust_loss_004 | Firecrawl | 1.0 | 1.0 | https://www.rudderstack.com/blog/addressing-segments-limitations/ and https://www.rudderstack.com/blog/how-joybird-reduces-engineering-time-spent-on-customer-data-integrations-with-rudderstack/ | The results explicitly name Joybird and state it migrated from Segment to RudderStack, with a RudderStack case study and blog post directly supporting the customer-loss signal. |
| cust_loss_004 | BrightData SerpAPI | 1.0 | 1.0 | https://www.rudderstack.com/blog/addressing-segments-limitations/ | The RudderStack result explicitly states Joybird migrated its customer data platform from Segment to RudderStack, directly matching the requested customer-loss signal. |
| cust_loss_004 | Tavily | 1.0 | 1.0 | rudderstack.com/customers/how-joybird-reduces-engineering-time-spent-on-customer-data-integrations-with-rudderstack | The top result is a RudderStack customer case study explicitly stating Joybird migrated its customer data platform from Segment to RudderStack, directly matching the requested customer-loss signal. |
| cust_loss_004 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_005 | Keenable | 1.0 | 1.0 | news.ycombinator.com/item?id=22637302; resources.rudderstack.com/case-studies/mattermost-roi | The results explicitly state Mattermost was being switched from Segment to RudderStack and include a RudderStack case study about Mattermost using RudderStack, directly supporting the customer-loss signal. |
| cust_loss_005 | Parallel | 1.0 | 1.0 | RudderStack customer story: https://www.rudderstack.com/customers/how-mattermost-uses-rudderstack-to-enable-data-driven-product-decisions | The results include RudderStack migration guides from Segment plus a RudderStack customer story for Mattermost, which together directly support the Segment-to-RudderStack customer displacement signal for Mattermost. |
| cust_loss_005 | Exa | 1.0 | 1.0 | RudderStack customer story / Mattermost blog / GitHub PRs in results | The results directly show Mattermost using RudderStack and include a Segment-to-RudderStack migration guide plus a Mattermost PR titled 'Remove Segment,' which together support the customer-loss/displacement signal from Segment to RudderStack. |
| cust_loss_005 | Perplexity | 0.33 | 0.0 | rudderstack.com docs/knowledge-base | The results show RudderStack’s generic Segment-to-RudderStack migration guides, but they do not name Mattermost or explicitly evidence a Segment customer loss to RudderStack. |
| cust_loss_005 | Firecrawl | 0.33 | 0.33 | rudderstack.com/docs/user-guides/migration-guides/segment-rudderstack-migration/ | The results show general Segment-to-RudderStack migration/displacement content, but they do not mention the specific customer Mattermost or directly evidence that Mattermost migrated. |
| cust_loss_005 | BrightData SerpAPI | 0.0 | 0.0 | The supplied search result text contains only the Google query metadata and no result snippet or cited source showing Mattermost moved from Segment to RudderStack. | |
| cust_loss_005 | Tavily | 0.33 | 0.33 | RudderStack migration docs / guides | The results clearly show Segment-to-RudderStack migration guidance from an official RudderStack source, but they do not mention Mattermost or evidence that Mattermost specifically migrated from Segment to RudderStack. |
| cust_loss_005 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_006 | Keenable | 0.33 | 0.5 | xero.com convert from QuickBooks / migration guide results | The results show generic QuickBooks-to-Xero migration resources and QuickBooks-vs-Xero comparison pages, but they do not name Webistry or directly evidence a specific customer switching from QuickBooks to Xero. |
| cust_loss_006 | Parallel | 0.33 | 0.33 | Xero / YouTube titles in results | The results show generic QuickBooks-to-Xero migration/switching content and a YouTube title about switching, but they do not mention Webistry or provide a specific customer-loss signal from QuickBooks to Xero. |
| cust_loss_006 | Exa | 0.33 | 0.0 | Multiple results about moving/switching from QuickBooks to Xero (e.g., jamietrull.com, smallbizthoughts.com, Xero US conversion pages) | The results show generic QuickBooks-to-Xero switch/migration content, but none names Webistry or clearly evidences that this specific customer moved from QuickBooks to Xero. |
| cust_loss_006 | Perplexity | 0.0 | 0.0 | xero.com/us/accounting-software/convert-from-quickbooks/ | The results only show generic QuickBooks-to-Xero conversion/migration pages and do not mention Webistry or a specific customer switching from QuickBooks to Xero. |
| cust_loss_006 | Firecrawl | 0.33 | 0.0 | xero.com / quickbooks.intuit.com / YouTube migration results | The results show generic QuickBooks-to-Xero conversion/migration resources, but they do not name Webistry or provide evidence that Webistry specifically switched from QuickBooks to Xero. |
| cust_loss_006 | BrightData SerpAPI | 0.0 | 0.0 | The provided search result text contains only Google search metadata and no substantive evidence naming Webistry, Xero, or a QuickBooks-to-Xero switch. | |
| cust_loss_006 | Tavily | 0.33 | 0.0 | xero.com/us/accounting-software/convert-from-quickbooks | The results show generic QuickBooks-to-Xero migration pages, but they do not name Webistry or directly evidence a specific customer loss from QuickBooks to Xero. |
| cust_loss_006 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_007 | Keenable | 1.0 | 1.0 | inboundfintech.com/case-studies/pusher | inboundfintech.com/blog/migrate-from-salesforce-to-hubspot-integrations | The results explicitly name Pusher and state it wanted to migrate from Salesforce to HubSpot, which directly matches the requested customer-loss/displacement signal and is supported by a credible case study/implementation-partner source. |
| cust_loss_007 | Parallel | 1.0 | 1.0 | https://www.inboundfintech.com/case-studies/pusher | The top result directly names Pusher and Salesforce-to-HubSpot migration, which supports the requested loss/displacement signal from Salesforce to HubSpot. |
| cust_loss_007 | Exa | 0.95 | 0.95 | Evara case study title "Pusher | Salesforce to HubSpot Migration" (evara.co/case-studies/pusher); also HubSpot official page "Switching from Salesforce to HubSpot" | The results explicitly name Pusher and a Salesforce-to-HubSpot migration, with HubSpot as the winning platform and an official HubSpot migration page supporting the same displacement signal. |
| cust_loss_007 | Perplexity | 0.33 | 0.0 | HubSpot switch/migration pages and third-party migration guides | The results show generic Salesforce-to-HubSpot migration content, but they do not name Pusher or clearly document a customer/account switching from Salesforce to HubSpot. |
| cust_loss_007 | Firecrawl | 0.33 | 0.5 | https://www.hubspot.com/switch-from-salesforce | The results show generic Salesforce-to-HubSpot migration content, including HubSpot’s own switch page, but they do not mention Pusher or clearly evidence that Pusher specifically migrated from Salesforce to HubSpot. |
| cust_loss_007 | BrightData SerpAPI | 0.0 | 0.0 | hubspot.com / HubSpot Community / multiple Salesforce-to-HubSpot migration guides | The results only show generic Salesforce-to-HubSpot migration content and do not mention Pusher or any specific customer loss from Salesforce to HubSpot. |
| cust_loss_007 | Tavily | 0.33 | 0.0 | syncmatters.com/blog/salesforce-to-hubspot-migration-a-detailed-step-by-step-guide | The results show general Salesforce-to-HubSpot migration content, but they do not mention Pusher or specifically evidence Salesforce lost Pusher to HubSpot. |
| cust_loss_007 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_008 | Keenable | 1.0 | 1.0 | blog.hubspot.com/customers/migrate-from-marketo-hubspot-one-week | The results directly name Homesuite as using Marketo and migrating to HubSpot in a HubSpot customer story, which matches the requested customer-loss signal. |
| cust_loss_008 | Parallel | 0.33 | 0.0 | aptitude8.com / "How to Migrate from Marketo to HubSpot: A Step by Step Guide" | The results show generic Marketo-to-HubSpot migration content, but none names Homesuite or directly evidences that Homesuite switched from Marketo to HubSpot. |
| cust_loss_008 | Exa | 0.33 | 0.5 | ExpertBeacon title and HubSpot migration/result titles | The results strongly indicate a Marketo-to-HubSpot migration, but they do not name Homesuite, so the exact customer-loss signal is not fully supported. |
| cust_loss_008 | Perplexity | 0.33 | 0.33 | HubSpot customer story / blog | The results show Marketo-to-HubSpot migration pages and a HubSpot customer story, but they do not name Homesuite or clearly evidence that Homesuite was the customer that switched. |
| cust_loss_008 | Firecrawl | 0.33 | 0.5 | hubspot.com/comparisons/marketo-vs-hubspot; hubspot.com/case-studies/marq | The results show general Marketo-to-HubSpot migration/displacement content and a HubSpot case study, but they do not name Homesuite or directly evidence that Homesuite switched from Marketo to HubSpot. |
| cust_loss_008 | BrightData SerpAPI | 0.33 | 0.0 | HubSpot/Marketo comparison and migration-guide results (e.g., hubspot.com/comparisons/marketo-vs-hubspot; Aptitude 8) | The results show general Marketo-to-HubSpot migration/comparison content, but they do not name Homesuite or directly evidence a specific customer switch from Marketo to HubSpot. |
| cust_loss_008 | Tavily | 0.33 | 0.0 | campaigncreators.com/blog/an-insiders-guide-to-re-platforming-from-marketo-to-hubspot | The results show generic Marketo-to-HubSpot migration content, but they do not mention Homesuite or provide evidence that Homesuite switched from Marketo to HubSpot. |
| cust_loss_008 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_009 | Keenable | 1.0 | 1.0 | zyber.co.nz/pages/case-studies/tsb-living | Result [1] directly states TSB Living’s email list was migrated from MailChimp to Klaviyo in a case study, which clearly supports the Mailchimp-to-Klaviyo customer-loss signal. |
| cust_loss_009 | Parallel | 1.0 | 1.0 | https://www.zyber.co.nz/pages/case-studies/tsb-living | Result [2] directly names TSB Living and states a Klaviyo growth case study, matching the Mailchimp-to-Klaviyo migration signal with a credible case-study source. |
| cust_loss_009 | Exa | 1.0 | 1.0 | https://www.zyber.co.nz/pages/case-studies/tsb-living | The top result directly matches the target customer TSB Living and a case study titled "TSB Living 140% Klaviyo Marketing Growth," which credibly supports a Mailchimp-to-Klaviyo migration signal. |
| cust_loss_009 | Perplexity | 1.0 | 1.0 | zyber.co.nz/pages/case-studies/tsb-living | The results directly state that TSB Living switched from Mailchimp to Klaviyo and show a case study titled "TSB Living 140% Klaviyo Marketing Growth" from Zyber, which is a credible implementation-partner source for the same customer-loss signal. |
| cust_loss_009 | Firecrawl | 1.0 | 1.0 | zyber.co.nz/pages/case-studies/tsb-living | The results include a direct case study stating TSB Living's email list was migrated from Mailchimp to Klaviyo and highlights 140% Klaviyo marketing growth, matching the requested customer-loss signal. |
| cust_loss_009 | BrightData SerpAPI | 1.0 | 1.0 | zyber.co.nz - TSB Living 140% Klaviyo Marketing Growth | The top result directly states that TSB Living's email list was migrated from Mailchimp to Klaviyo and is presented as a case study with the cited 140% Klaviyo marketing growth. |
| cust_loss_009 | Tavily | 0.33 | 0.0 | polarisgrowth.com / simoneze.com / mailchimp.com | The results show general Mailchimp-to-Klaviyo migration content, but none names TSB Living or directly evidences that Mailchimp lost that customer to Klaviyo. |
| cust_loss_009 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_010 | Keenable | 1.0 | 1.0 | shopify.com/th/case-studies/rainbow-shops | The results directly state that Rainbow Shops switched from Salesforce Commerce Cloud (Demandware/SFCC) to Shopify and report an 80% reduction in platform fees, which cleanly matches the requested customer-loss signal. |
| cust_loss_010 | Parallel | 1.0 | 1.0 | Shopify case study: https://www.shopify.com/case-studies/rainbow-shops | The results include a direct Shopify case study titled “How Rainbow Shops replatformed from Salesforce to Shopify,” which clearly supports Rainbow Shops moving from Salesforce Commerce Cloud to Shopify. |
| cust_loss_010 | Exa | 1.0 | 1.0 | shopify.com/case-studies/rainbow-shops | The results directly identify Rainbow Shops replatforming from Salesforce Commerce Cloud to Shopify, with Shopify case-study and secondary article support, matching the requested customer-loss signal. |
| cust_loss_010 | Perplexity | 1.0 | 1.0 | https://www.shopify.com/case-studies/rainbow-shops | The results directly state that Rainbow Shops switched from Salesforce Commerce Cloud (Demandware) to Shopify and achieved an 80% reduction in platform fees, which is the exact customer-loss signal. |
| cust_loss_010 | Firecrawl | 1.0 | 1.0 | shopify.com/case-studies/rainbow-shops | The results directly state that Rainbow Shops switched from Salesforce Commerce Cloud/Demandware to Shopify and note an 80% reduction in platform fees, matching the requested customer-loss signal. |
| cust_loss_010 | BrightData SerpAPI | 1.0 | 1.0 | shopify.com/case-studies/rainbow-shops; shopify.com/enterprise/blog/salesforce-commerce-cloud-alternatives | The results directly state that Rainbow Shops switched from Salesforce Commerce Cloud/Demandware to Shopify, with Shopify’s own case study and blog noting the move and an 80% platform-fee reduction. |
| cust_loss_010 | Tavily | 1.0 | 1.0 | Shopify case study: https://www.shopify.com/case-studies/rainbow-shops | The results directly state that Rainbow Shops replatformed from Salesforce Commerce Cloud/Demandware to Shopify and that the move produced an 80% reduction in platform fees, which cleanly matches the requested customer-loss signal. |
| cust_loss_010 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_011 | Keenable | 0.67 | 0.5 | intercom.help/myhsa/en; intercom.com/help/en/articles/9307237-import-your-zendesk-ticket-user-and-organization-data | The results include a MyHSA Intercom help center and an official Intercom Zendesk import page, which supports a Zendesk-to-Intercom migration for MyHSA, but the snippet does not explicitly state that MyHSA was lost from Zendesk. |
| cust_loss_011 | Parallel | 0.33 | 0.33 | Intercom Help / community.intercom.com / Zendesk comparison page | The results show generic Zendesk-to-Intercom migration resources, but they do not name MyHSA or evidence that MyHSA specifically switched from Zendesk to Intercom. |
| cust_loss_011 | Exa | 0.33 | 0.0 | Intercom Help / Zendesk-to-Intercom migration pages | The results show generic Zendesk-to-Intercom migration help and guides plus a myHSA help-center article, but they do not connect Zendesk, MyHSA, and Intercom in a single customer-loss/displacement signal. |
| cust_loss_011 | Perplexity | 0.0 | 0.0 | intercom.com/help/en/articles/4393790-switching-from-zendesk-to-intercom | The results only show generic Zendesk-to-Intercom migration/help articles and do not mention MyHSA or a specific customer loss from Zendesk to Intercom. |
| cust_loss_011 | Firecrawl | 0.0 | 0.0 | intercom.com help articles / zendesk vs intercom comparison results | The results show generic migration guides and comparisons between Zendesk and Intercom, but no evidence that MyHSA specifically migrated from Zendesk to Intercom. |
| cust_loss_011 | BrightData SerpAPI | 0.33 | 0.0 | Intercom help article; migration guides and case study titles | Results show Zendesk-to-Intercom migration content, but none names MyHSA or directly evidences that MyHSA switched from Zendesk to Intercom. |
| cust_loss_011 | Tavily | 0.33 | 0.5 | intercom.com help article / Help Desk Migration blog / YouTube migration guide | The results show Zendesk-to-Intercom migration guidance and an Intercom help article, but they do not mention the named customer MyHSA or evidence that MyHSA specifically moved from Zendesk to Intercom. |
| cust_loss_011 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 | |
| cust_loss_012 | Keenable | 0.33 | 0.33 | help.klaviyo.com / emailtooltester.com | The results only show generic Mailchimp-to-Klaviyo migration guides and Klaviyo help pages, but none mention Pair of Thieves or directly evidence that this customer moved from Mailchimp to Klaviyo. |
| cust_loss_012 | Parallel | 1.0 | 0.95 | Klaviyo case study result: 'Mailchimp Alternative: Travel Cat Email Revenue +49% - Klaviyo' plus 'Case Study: Pair of Thieves achieves 8.5x list growth and 42% ...' (casestudies.com / Klaviyo) | The results include a Klaviyo case-study result explicitly naming Pair of Thieves and multiple Mailchimp-to-Klaviyo migration sources, which is enough to support the customer-loss/displacement signal from Mailchimp to Klaviyo. |
| cust_loss_012 | Exa | 0.33 | 0.5 | Case Study: Pair of Thieves achieves 8.5x list growth and 42% of revenue from email with Klaviyo | CaseStudies.com | The results clearly connect Pair of Thieves with Klaviyo, but they do not state that Pair of Thieves migrated from Mailchimp or that Mailchimp was the losing vendor, so the requested competitive-displacement signal is not directly supported. |
| cust_loss_012 | Perplexity | 0.33 | 0.5 | help.klaviyo.com (Mailchimp migration help articles) | The results show Klaviyo documentation about migrating from Mailchimp, but they do not mention the specific customer Pair of Thieves, so they only provide partial displacement context rather than the requested customer-loss evidence. |
| cust_loss_012 | Firecrawl | 0.33 | 0.33 | Multiple migration/help pages and articles about Mailchimp to Klaviyo | The results show general Mailchimp-to-Klaviyo migration content, but none mention Pair of Thieves or explicitly evidence a specific customer loss from Mailchimp to Klaviyo. |
| cust_loss_012 | BrightData SerpAPI | 0.33 | 0.0 | Klaviyo Help Center / Mailchimp / EmailTooltester / Polaris Growth | The results only show generic guidance and examples for migrating from Mailchimp to Klaviyo, but they do not name the customer Pair of Thieves or directly evidence a specific loss from Mailchimp. |
| cust_loss_012 | Tavily | 0.0 | 0.0 | The results only show generic Mailchimp/Klaviyo migration help pages and do not mention Pair of Thieves or evidence that it moved from Mailchimp to Klaviyo. | |
| cust_loss_012 | You.com | 0.0 | 0.0 | Provider unavailable: skipped_after_provider_error 402 |
Method note
Each provider was queried through the same KrabArena Web Access API Battle proxy. Top-10 result text was flattened and judged by ~openai/gpt-mini-latest. Python only orchestrated calls, parsed JSON, and aggregated metrics; semantic match labels came from the LLM judge.
Falsification condition: rerun the fixed dataset and judge prompt from this artifact; the claim is falsified if Keenable does not produce at least 10/12 customer-loss signal matches or if another working provider exceeds its hit count under the same scoring.