The transaction test: An experimental method for assessing online interfaces

Abstract

This article introduces the transaction test, a novel experimental method to detect whether a website customization can be deceptive. The method measures changes in choice inconsistency with respect to a plain design. In a highly powered online experiment, we assess three dark patterns and one transparency-based remedy in decisions involving data protection. Compared to a placebo condition, dark patterns increase the likelihood of choice inconsistency by an effect in the range of 6-12 percentage points, depending on experimental conditions. The transparency-based remedy fails to mitigate these effects. A second study implements the transaction test in a market experiment. Here, one of the treatments from the online experiment increases inconsistency by 42 pp. Two additional deceptive designs are tested, respectively favoring consistent and inconsistent choices. The Dark Pattern version raises inconsistency by 18 pp while the Bright Pattern reduces it by 15 pp. Our findings demonstrate how the transaction test, by providing an empirical assessment of the legal concept of unfairness, offers a flexible tool for the regulation of dark patterns and other commercial practices.