The reverse information paradox: When AI platforms retain too much or too little data

Abstract

When do AI platforms retain too much or too little user data? Users disclose proprietary context to obtain useful generative-AI output, but retention of that context creates exposure and may improve future systems. We formalize this user-side analogue of Arrow’s information paradox. The disclosure cap separates two regimes. Outside the capped region, behavioral adjustment is strong enough for equilibrium harm to fall with retention, so the planner’s optimum is a boundary choice. Within the capped region, the ratio of benefit appropriability to harm internalization governs platform retention: the platform under-retains when the ratio is below one and over-retains when it is above one. When retained inputs also raise AI quality, retention and disclosure become complements once quality feedback is sufficiently strong. Retention regulation corrects the platform’s choice; an interior disclosure choice generally requires an additional disclosure-side instrument to implement the joint first best.