ChatGPT Didn't Kill the Question. It Grew It.

Based on the research ofRao, Shan, Rivera, Qiu and Wang, "How Generative AI Transforms Questioning Behavior on Q&A Platforms: Evidence from a Natural Experiment with Pilot Usage of ChatGPT," MIS Quarterly, 2026

A Five Day Window Became a Natural Experiment

ChatGPT launched to the public on November 30, 2022. Stack Overflow users began posting its answers almost immediately, and by December 5 the platform's moderators had seen enough: in a policy titled "ChatGPT is banned," they wrote that "the average rate of getting correct answers from ChatGPT is too low" and that the volume of AI generated answers, in the thousands, had "effectively swamped" the site's volunteer based quality curation. The moderators called the measure temporary, a stopgap while the company worked out a permanent stance, but the short period of open access to AI answers on one of the largest Q&A platforms in the world gave researchers something rare: a real before and after, inside the same community, with the same users.

Xinzhi Rao, Guohou Shan, Michael Rivera, Liangfei Qiu, and Nan Wang used that window as a natural experiment. Applying a difference in differences design to Stack Overflow data, they tracked how exposure to generative AI answers changed users' own questioning behavior, measured through the number of questions posted per week, question length, novelty, and the upvotes those questions received. The obvious hypothesis going in was that an instant, always available answer machine would make people rely on it instead of the community, and that the questions still posted to humans would be fewer but sharper, since AI would mop up the routine stuff. That is not what the data showed. Generative AI use was associated with more questions, not fewer, and those questions were longer, more novel, and more upvoted than before.

The Mechanism: AI Clears the Floor, People Climb Higher

The paper's explanation runs through two channels. The first is what the authors call complexity shift: ChatGPT absorbs routine, easily answered queries, the kind that used to pad out Stack Overflow's question volume with "how do I reverse a string in Python" style posts. With that baseline cleared, the questions that still go to the human community skew toward the harder, more idiosyncratic problems that an AI answer could not resolve, which is why the surviving questions look longer and more novel. The second channel is skill enhancement: people who spend time with ChatGPT's responses apparently absorb something about how to frame a problem clearly, carrying that improved question craft back into their posts for the human community, where clarity tends to earn upvotes.

The study also tests who this effect hits hardest. It is weaker among experienced answerers, people who already knew how to ask well and had less to learn from watching an AI model a clean answer, and stronger among users with higher platform reputation scores, the people most invested in the community's judgment of their questions. In other words, generative AI did not flatten engagement into a single pattern. It pushed the platform's more reputation sensitive, more engaged members toward asking bigger, better questions, precisely the opposite of the "AI makes humans lazy" story that circulated when ChatGPT first appeared.

Reconciling the Paradox With Stack Overflow's Own Numbers

This finding sits awkwardly next to a fact that is also real and well documented: Stack Overflow's overall site traffic has fallen sharply since ChatGPT's debut. SimilarWeb reported that Stack Overflow's visits dropped by an average of 6 percent month over month starting in January 2022, worsening to a 13.9 percent year over year decline in March 2023 and a further 17.7 percent drop the following month. Stack Overflow's own director of communications confirmed to Gizmodo that "Stack Overflow's traffic, along with traffic to many other sites, has been impacted by the surge of interest in ChatGPT," while noting the company was already building generative AI features of its own.

Two true things, then: visits to the site have been sliding for years, and the subset of users who actually post questions, when they get hands on access to an AI that answers instantly, post more and harder questions than before. The reconciliation is who left and who stayed. The traffic decline is plausibly driven by the large population of casual searchers and lurkers, people who used to land on Stack Overflow through a Google search for a quick fix and who now get that same quick fix directly from ChatGPT or GitHub Copilot without ever visiting the site. The MIS Quarterly study, by contrast, is tracking engaged community members, the kind of user who posts questions and cares about reputation, during a window when they had AI access but had not yet migrated away. For that population, AI did not substitute for the community. It fed it harder problems and better framed ones.

An AI that answers instantly didn't make people stop asking. It changed what counted as worth asking.

What This Means for Any Platform Betting Against AI

The lazy reading of "AI will kill forums" assumes users are rational substituters: give them a free answer machine and they stop bothering other people. Stack Overflow's own pilot window complicates that story in both directions at once. The platform's traffic numbers support the substitution story at the macro level, since casual visits have been falling for years and Stack Overflow's management has openly acknowledged ChatGPT's role in that decline. But inside the population that still bothers to post, the natural experiment shows the opposite dynamic: instant AI answers raised the bar for what a human question needed to be, and raised how many of those harder questions got asked.

For any operator of a Q&A platform, a support forum, or a knowledge community watching generative AI eat into their traffic, the lesson is not simply "defend against AI" or "integrate AI." It is that the two populations inside your user base, the drive by searchers and the engaged community, will respond to AI access in opposite directions, and a platform strategy built for one will misread the other. Measuring your health only by visits will tell you a decline story. Measuring the depth, novelty, and length of the questions your remaining community actually posts might tell you a growth story running underneath it. Stack Overflow's ban window was an accident of moderation policy, not a designed trial, but it handed researchers, and now platform operators, a clean look at what AI access does to the people who stayed.