Your Ratings Do Not Measure Quality. They Measure Who Holds Power.

Based on the research ofVanessa C. Burbano, Wesley W. Koo, and Jiao Luo, "A Platform Rating System and Vulnerable Workers: Evidence from Field Experiments in Singapore," Organization Science, 2026

In field experiments in Singapore, Vanessa Burbano, Wesley Koo, and Jiao Luo invited foreign domestic workers to rate the employer families who control their housing, wages, and immigration status. Standard two-sided-ratings logic predicts the intended beneficiaries would seize the tool. They did the opposite. Workers told about the rating system disengaged from the platform almost entirely, and the workers most exposed to a difficult employer pulled back the hardest rather than seizing the chance to speak. Employers, by contrast, were largely indifferent to being rated unless the system was framed around a direct benefit to themselves. The system did not neutralize the power structure beneath it. It inherited and encoded it, and the rating was least trustworthy exactly where scrutiny mattered most.

Airbnb: engineering the retaliation channel out before launch

Airbnb's engineering team documented the same pathology inside its own marketplace. Reviews skewed upward, and bad experiences frequently went unreviewed, because each side feared a retaliatory review from the other. A host can tank a guest's ability to book future stays; a guest can tank a host's income. Left alone, the score drifts toward whoever holds more leverage in the next transaction. The skew is measurable: roughly 95 percent of Airbnb listings carry an average rating of 4.5 or 5 stars, against 3.8 for hotels and 4.1 for bed-and-breakfasts, per Zervas, Proserpio, and Byers, whose paper is titled "Where Every Stay Is Above Average." When both sides can punish, almost nobody scores below great.

Airbnb's response was structural, not aspirational. It built a double-blind simultaneous-reveal system: guest and host each get 14 days, neither can see the other's review until both submit or the window closes, and both publish at once. Neither party can tailor a review to the one they received, and neither can dangle a favorable rating in exchange for one back. This is Burbano and colleagues' mechanism engineered away in advance. Rather than hoping symmetry holds, the platform severs the punishment path. The payoff is quantifiable: when Andrey Fradkin, Elena Grewal, and David Holtz studied Airbnb's shift to simultaneous reveal in a field experiment, the change cut the share of five-star reviews by 1.6 percentage points and raised how often guests reviewed at all, direct evidence that the old sequential system had been suppressing honest criticism.

The managerial implication is blunt. Two-sided reviews are honest only to the degree you can stop the rated party from punishing the rater. If your flow lets one side see the other's rating first, or lets the stronger side condition future access on a good review, your scores inflate exactly where you need candor most.

When both sides can punish, almost nobody scores below great, and the score measures leverage, not service.

Upwork: the structural fix helps, and still is not enough

Upwork adopted the same double-blind design, for the same reason: freelancers rate the clients who control their income and future contracts, a direct power gradient the platform wanted to defuse. Neither side sees the other's feedback until both submit or 14 days pass. Upwork went further and built a second, private channel into its core reputation metric. The Job Success Score, the percentage that gates Top Rated status (90 percent or above, roughly the top 10 percent of freelancers), blends public stars with private client feedback the freelancer never sees. The tell is in the gap: a freelancer can collect five public stars on every single contract and still watch their Job Success Score sit well below what those reviews imply, because clients rate honestly only where the freelancer cannot see it and cannot retaliate.

And yet freelancers still routinely hand five public stars to abusive clients. Community threads document a freelancer who rated a "really rough" client five stars out of sheer fear of a bad review, and others describing clients who threaten negative feedback to extract extra unpaid work. This is the Singapore finding replicated where the structural fix already exists. When the rated party controls your livelihood, fear of retaliation can outrun the platform's protection, and the weakest freelancers inflate hardest. Upwork's own two-tier data proves the point: the number it trusts enough to gate its top badge is the private one.

The lesson for operators is that a double-blind mechanism reduces silencing but does not eliminate it when dependence is severe and the rater neither trusts nor understands the protection. You must build the anti-retaliation design and make it credible and legible to the weaker side, and you must watch for coercion in private messages before the rating window ever opens. The on-platform score can look pristine while the real signal is being extorted off the record.

Apple and Amazon: when the platform itself is the rated party

The sharpest inversion appears when the platform is the rated party and the complementor is the weak rater. Developers depend on Apple for their entire distribution and revenue, hand over a 30 percent commission on many transactions, and can have apps rejected or accounts terminated at will. Apple felt the pressure enough to blink: on January 1, 2021, it launched the App Store Small Business Program, halving the commission to 15 percent for developers earning up to 1 million dollars in annual proceeds, while the 30 percent rate stayed in force above that threshold. Yet a lower take does not restore candor. The US House Judiciary antitrust subcommittee's 16-month investigation, reported in October 2020, described developers as afraid of economic retaliation and unwilling to raise App Store concerns publicly; Apple told one developer in writing that going to the press "never helps." The starkest testimony came later: at the April 2021 Senate Judiciary antitrust hearing chaired by Amy Klobuchar, Spotify's chief legal officer Horacio Gutierrez testified that Apple had threatened to remove Spotify from the store as punishment for speaking out. Honest feedback surfaced only once a coalition and a Congressional shield removed the individual retaliation risk. Any "rate your platform" survey, appeals channel, or developer NPS inherits this dynamic at ecosystem scale.

Amazon's third-party sellers show the same silencing among merchants. Sellers depend on the marketplace for survival and can be suspended or de-listed for unstated reasons. On September 26, 2023, the FTC and 17 state attorneys general filed a 172-page antitrust complaint alleging, among other harms, that Amazon's combined fees force many sellers to pay close to 50 percent of their total revenue back to the platform (see the exhibit). Independent tracking sharpens the picture: Marketplace Pulse found that Amazon's average cut of each sale surpassed 50 percent for the first time in 2022, up from roughly 40 percent five years earlier, and that the typical seller now spends about 26 percent of revenue on Amazon fees, up from 19 percent in 2020. The agency argued sellers "reasonably fear that Amazon will use this information to stifle sellers who speak out," and asked the court to shield seller testimony from Amazon's in-house counsel. One seller put it plainly: Amazon sellers are "more scared of Amazon than the U.S. government." A CEO who believed Amazon's pricing policy was anticompetitive stayed silent for fear of retaliation.

The implication for any platform surveying its complementors is that your own power to deplatform is the thing suppressing the signal. Attributable feedback channels systematically under-report your worst problems, and your most dependent participants are the quietest. To hear the truth you must remove attribution or remove the retaliation lever.

The rule any operator can act on

A two-sided rating system converts experience into honest signal only to the extent you can sever the rated party's ability to retaliate, on the platform or off it. Where that channel stays open, the score does not measure quality. It measures the power gradient. The more a rater depends on the rated party for something you do not own (income, rehire, store access, distribution, a visa), the more that rating is silenced or inflated, so your data is least trustworthy exactly where oversight matters most, and your most vulnerable participants are the quietest (see the exhibit). Singapore is the clean proof: strip the protection and engagement collapses toward zero among the very workers the tool was built to help.

Your real design variable is not "do we have ratings." It is "have we structurally broken the retaliation lever," through double-blind reveal, anonymity, independent third-party channels, or binding non-retaliation rules. Airbnb and Upwork engineered it and still only reduced the bias; 95 percent of Airbnb listings remain at the ceiling and Upwork trusts its private score over its public stars. The pre-2020 App Store and the Amazon marketplace left the lever open and governed on an illusion produced by their own coercive power.

Before you trust another satisfaction score, ask who could punish the person who gave it, and design that answer down to zero.