Your AI Feature Checklist Is Measuring the Wrong Thing

Based on the research ofClarissa Knorr and Bastian Kindermann, "How Does (AI-)Complement Configuration Affect User Value in Platform Ecosystems? A Qualitative Comparative Analysis," International Conference on Information Systems (ICIS), 2025

The Checkbox Audit

Walk into most product reviews in 2026 and you will hear a version of the same question: "Do we have an AI feature yet?" It is asked the way you'd ask whether the app supports dark mode or push notifications, a box to tick, evaluated on its own, credited or blamed in isolation. Add a chatbot, ship an AI summary tool, bolt on a copilot, and move to the next line of the roadmap. The underlying assumption is that each complement, AI or otherwise, carries its own, portable amount of value, which you can estimate by looking at the feature alone.

A new paper presented at the 2025 International Conference on Information Systems argues that this assumption is close to backwards. Clarissa Knorr of TU Dortmund University and Bastian Kindermann of Technische Universität Braunschweig ran a fuzzy-set qualitative comparative analysis, a configurational method built to find which *combinations* of conditions, not which single conditions, produce an outcome, on 5,008 iOS apps. They coded each app's complements against Carliss Baldwin and Jason Woodard's Layered Modular Architecture, which splits a digital platform into content, service, network, and device layers, and then asked which configurations of complement attributes across those layers were associated with higher consumer value. The headline result: there isn't one. There are several, and they are different for apps with AI capability than for apps without it. A feature that drives value in a configuration that also leans on the network layer can be nearly irrelevant in a configuration built around the device layer instead. Equifinality, many different roads to the same destination, is the finding, not an inconvenient footnote.

Same Chatbot, Two Fates

You do not need the fsQCA truth table to see the mechanism; two apps shipped it in public within weeks of each other. In February 2023, Snapchat launched My AI, a chatbot built on OpenAI's GPT technology, as a Snapchat+ subscriber perk. On April 19, Snap opened it to every global user for free and pinned it to the top of the Chat feed, a conversation that, unlike every other thread in that feed, could not be unpinned, blocked, or removed. It had, as TechCrunch put it, "appeared inside their app without warning or their consent." Within a week, Snapchat's average U.S. App Store rating fell to 1.67, with 75% of reviews landing at one star, against a Q1 2023 baseline of 3.05 and 35%. Daily review volume jumped fivefold. A separate analytics firm, Apptopia, found "AI" had become the single most-mentioned word in Snapchat's reviews that week, 2,973 mentions, with a sentiment-impact score of -9.2 on a -10-to-10 scale. None of this was a demand problem: users were already sending nearly two million messages a day to the same chatbot before the global rollout. It was a configuration problem. The AI was bundled with forced placement, zero consent, and a social messaging layer where Snapchat users transact in trust with friends, not strangers' software.

Three weeks earlier, Spotify had shipped a feature running on strikingly similar technology: an AI DJ that narrates a personalized queue in a synthetic, realistic voice. Spotify bundled it differently. The DJ draws on each listener's own history inside Spotify's existing recommendation engine, the platform's core content layer, and it is something you turn on, not something that appears uninvited in your social feed. Spotify reported that on days people used it, they spent 25% of their listening time with the DJ, and more than half of first-time listeners came back to use it again the very next day. The company expanded it to the UK and Ireland within two months, chasing demand rather than managing backlash.

Same broad class of generative AI, same window of months, opposite outcomes. The variable that moved wasn't the model. It was what the model was configured with: whose data it drew on, whether the user had a say, and which layer of the app it was dropped into.

The chatbot was never the problem. The chatbot bundled with no consent, no control, and no personalization was the problem.

The Complement Graveyard

If adding an AI complement in isolation doesn't reliably add value, the older and blunter version of the mistake is adding *any* complement and assuming volume alone does the work. Amazon's Alexa skills marketplace is the clearest record of that failure at scale. By October 2018, more than 40,000 third-party skills existed in the U.S. store, roughly double the count from a year earlier, and 61% still had zero user ratings, a figure that had barely moved (down one point from 62%) despite the skill count doubling. Only 5% had earned ten or more ratings, just 1% had more than a hundred, and of the entire catalog, exactly one skill, a white-noise app called Ambient Sounds: Thunderstorm Sounds, had crossed ten thousand reviews. Tens of thousands of developers built complements for a platform that, by Voicebot.ai's analysis, offered no reliable discovery path and no structural reason for most of them to be found, let alone combined with anything else a user already valued. The lesson generalizes past voice: a complement count is not a value count, with or without AI inside it.

The inverse also holds, and it is where platform owners should actually spend their attention. Salesforce's own public account of what makes its Agentforce AI agents work for customers names three ingredients that have to arrive together: a unified data foundation, a trust layer, and delivery inside the flow of existing work, not a fourth screen bolted alongside it. Turtle Bay Resort, one of the customers Salesforce cites, had an AI recommendation feature that did close to nothing until its scattered booking and guest-preference data was unified into one profile the AI could actually draw on; once the data layer and the AI layer were combined, personalized content for known guests produced a 40% jump in engagement. The AI algorithm didn't change. Its configuration did.

Don't Count Complements. Configure Them.

None of this says AI complements are overrated or that platform owners should stop shipping them. It says the unit of evaluation is wrong. Knorr and Kindermann's fsQCA result is that there is no single "good complement" profile, with or without AI, because value comes from how a complement's attributes combine with the content, service, network, and device layers around it, and multiple different combinations can each work, while others that look similar on paper don't. Snapchat and Spotify ran versions of the same technology into opposite configurations and got opposite reviews. Alexa proved that stacking complements without a path to combine them with what users already valued just produces a graveyard of zero-rating skills. Turtle Bay proved the reverse: the same AI layer, once paired with a data layer that didn't exist before, produced a real, measured lift.

The practical shift for any team sitting in a roadmap review is to stop asking "do we have this complement" and start asking "what is this complement configured with, and has that specific combination been shown to work." A chatbot is not a checkbox. It is one input into a configuration that will succeed or fail depending on the three or four other things sitting next to it, and the only way to know which configuration you've built is to look at it as a whole, the way the paper's method was built to do, rather than crediting or blaming the AI feature alone.