Algorithmic attention rents

Algorithmic attention rents is a concept developed at University College London's Institute for Innovation and Public Purpose in a series of three papers sponsored by the Omidyar Network, and based on earlier work by Tim O’Reilly. It is used to explain how platforms, with a degree of market power, can degrade the quality of information shown to users in its algorithmic results and recommendations in order to increase their profits above the competitive level. Implications for AI systems is explored. This builds partly on their previous work on digital disclosures, published by the Oxford Review of Economic Policy.

Background and motivation

Algorithmic attention rents draws heavily on Herbert Simon's theory of consumers as “information processors” and the challenges of decision making under extreme information abundance online, which algorithmic-based products, such as Google's Search, help consumers navigate. It builds out this thesis drawing on institutional economics to elaborate on the role of algorithms in making market-like allocations, often without use of the price mechanism or a price-based signal. This approach focuses on the process of decision making online, rather than simply the outcomes of such decisions, as in Neoclassical economics.

The concept's motivation was in the crowding out of optimal “organic” results in Google Search and Amazon's marketplace by inferior paid advertising results. As the phenomenon of advertising in search results grew online, the incentive for third-party websites and firms to pay for advertising, in order to gain access to users’ attention, increased, eventually turning the platform into a “pay to play” model.

Theory

The general theory of algorithmic attention rents, outlined in an overview paper by O’Reilly, Strauss, and Mazzucato, (published in Data & Policy by Cambridge University Press), is that the degrading of algorithmic information quality manifests online as above-normal allocations of user attention to inferior paid advertising or addictive content, generally with a view to getting more advertising revenue from advertisers. Users pay with an “attention rent” (time) and advertisers with a pecuniary rent, making this a multi-sided theory of platform power, following the multi-sided nature of platforms.

In a companion paper by Strauss, O’Reilly, and Mazzucato, focusing on the informational foundations of the concept with respect to Amazon's marketplace, they note that the advertisers who have to pay for visibility in search results, may be the same as the third-party sellers on the platform, with advertising being a pure rent extraction mechanism from these sellers to appear at the top of product search results. The idea is that, by virtue of more of the prime screen space being allocated to advertising, products can no longer compete on their merits, and must pay ever higher advertising fees to gain access to consumers. As organic results get downgraded on the screen in favor of paid advertising products, the algorithm no longer rewards more relevant products. Paid content displaces lower positioned organic results, ultimately, given users strong predisposition to click on the first few search results displayed (see Evidence section).

Attention rents and market power

Attention rents is fundamentally a multi-sided view of a platform's market power, with power over one side used to exploit the other, given the sharing by all sides of a fixed screen space (as a proxy for users’ attention). This contrasts with Jean Tirole's price based theory of how platforms allocate benefits, often leading to a situation where one side pays nothing at all, when cross-side network effects on the platform are strong.

According to the O’Reilly et al.:

In allocating user attention, the platform is also shaping the allocation of economic value between competing stakeholders on the platform, including itself, its users, its third-party supplier ecosystem, and its advertisers. A platform’s third-party producers compete with each other, and advertisers compete with these producers and other advertisers, for a fixed quantum of user attention. Not only is a user’s attention finite, so too is the narrow window onto abundant information provided by the screen, whose interface design is controlled by the platform. Every user attention allocation can thus lead to a pecuniary gain or harm for a firm, website owner, or content creator on another side of the platform. Attention allocations drive value allocations.

According to Strauss et al., information quality plays a key role in measuring market harms online:

In these markets, the level of information and the level of competition are increasingly tied together as greater user monetization necessitates a decline in the relevance of the information results displayed. This may entail showing users a level of information relevance below that which would prevail under more competitive conditions, where the platform had less market power over its ecosystem, and lock-in or stickiness over its users was weaker.

In this framing, market power (or “dominance” in the EU's conception) is defined as “its ability to shape user attention independently of user preferences, user inputs, and the relevance of its third-party ecosystem’s information.” Noting further that:

A “position of strength” is defined by a platform’s ability to direct, over time, significant volumes of user attention within a given market. “Independence” (also called “freedom of action”) is defined here not just by an absence of external pricing pressure (to price competitively), but by an absence of external pressure to show the most relevant available information to the platform. As the European Commission noted in the context of Google Search, it is the algorithm that ultimately sets the competitive benchmark for a platform’s ecosystem – and that a platform’s independence in attention allocations undermines.

Strauss et al. note that both consumer protection and antitrust tools have a role to play, given that many of these harms can occur on platforms irrespective of their size. However, the ability to extract persistently above normal rents from advertisers, will require a degree of platform market power, the authors argue.

Evidence: Amazon's third-party marketplace

Rock, Strauss, O’Reilly, and Mazzucato apply the concept of algorithmic attention rents to Amazon's third-party marketplace, where independent sellers try sell their products under their own branded labels. They examine econometrically the ability of Amazon's search algorithms to gain economic benefits for itself by strategically arranging search results, getting users to click on advertising at the top of the screen, even if inferior quality.

The study focuses on Amazon Marketplace product search results data during 2023 and aims to identify the factors that most significantly influence consumer clicking behavior, combine unique datasets on user clicks with amazon search results. The findings show a strong correlation between the visual prominence of a product on the screen (termed “attention share’’) and the likelihood of it receiving clicks from the user. This was found to be true even in cases where the products were priced higher or had inferior ratings than others but simply had a higher visual prominence. Within the top five search results, which typically include up to four advertisements, the study observed that neither a decrease in product relevancy nor an increase in price substantially deterred consumers from clicking. This behavior indicates, the author argue, that consumers often choose products that are displayed more prominently by Amazon's algorithms, rather than thoroughly searching for the most suitable options. The study confirms the common finding of users having a “position-bias” heuristic (clicks being driven by screen position), which Amazon has been able to exploit.

Empirically, the study's finding on advertising are that almost one third (31.8%) of the top-3 most clicked products in the most popular search results on Amazon's third-party Marketplace are sponsored (advertising) results. The top-3 most clicked advertised products were found to be 17% more expensive than organic ones ($19.3 vs. $16.5) and one-third less relevant (organic rank of 4 vs. 3). They also found considerable duplication of products in search results on Amazon due to it permitting multiple ads from a single seller. One-quarter of product search results on the first page were found to be adverts, leading to 48.3% of advertised results having at least one duplicate organic result on the first page, and 93.6% of top-3 most clicked ads being duplicated.

Reception and relationship to other popular concepts

The leading U.S. antitrust scholar Herbert Hovenkamp (author of the Antitrust Law textbook) has called the paper by Strauss et al. “a superb and I think very important paper on information costs, search algorithms, abundant but imperfect information, and the role of New Institutional Economics, focusing mainly on Amazon.”

Algorithmic rents is closely linked to Cory Doctorow's “enshittification” of platforms problem. For Doctorow attention rents” are when “Enshittification comes out of the barrel of an algorithm.” Noting:

The “attention rents” referenced in the paper’s title are bait-and-switch scams in which a platform deliberately enshittifies its recommendations, search results or feeds to show you things that are not the thing you asked to see, expect to see, or want to see. They don’t do this out of sadism! The point is to extract rent — from you (wasted time, suboptimal outcomes) and from business customers (extracting rents for “boosting,” jumbling good results in among scammy or low-quality results).

The work was covered in the Financial Times by Rana Forooohar, in the context of “surveillance capitalism”, quoting from the paper:

“the more fundamental problem that regulators need to address is that mechanisms by which platforms measure and manage user attention are poorly understood.”

For O’Reilly and his co-authors, “effective regulation depends on enhanced disclosures.”

Subsequent development

Marianna Mazzucato and Ilan Strauss have written in Project Syndicate on Facebook's feed algorithms and their displacement of content which meets user preferences, with “recommended”, addictive, content. This is more beneficial to the platform since it increases a user's engagement and time spent on platform. Focusing on feed and recommendation, rather than search algorithms, they allocate much of the blame for algorithmic rents to platforms’ optimizing for short-term user engagement. They provide five policy recommendations to ensure algorithms optimize for long-term benefits rather than short-term ones, focusing on algorithmic disclosures (including a monetization narrative) in public firms annual Form 10-K report, long-term orientated A/B testing, user influence over ranking, and use of public AI to monitor the quality of acceptable content promoted by a platform's advertising algorithms.