The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power

This next chapter focuses on the concept of rendition, the name Zuboff uses for the strategies that bring ubiquitous computing to life. One illustration of the dynamics of the dynamics of rendition is the iRobot company’s use of their Roomba. The Roomba, a cleaning robot, is a foundational machine in iRobot’s mission to establish “smart homes,” in which automation systems monitor and control home appliances. In 2017, iRobot drew controversy when their plans were made public: The company aimed to sell floor plans of customer homes drawn from Roomba’s mapping capabilities to Google, Amazon, or Apple. In preparation for these sales, iRobot revamped their Roomba to include a camera, advanced sensors, and mapping software.

Despite concerns from privacy experts, iRobot got away with such a plan because CEO Colin Angle insisted that it all relied on customer choice; if people didn't want the floorplans of their homes sold, they could simply opt out. Yet, Zuboff points out, when one tries to opt-out, many of the “smart” features of the Roomba such as phone controls of the vacuum, scheduled cleanings, and automatic software updates—the main reasons that people purchase these devices—stop working. Other smart home products, like the SleepIQ mattress or the Nest thermostat, operate under the same caveat.

Zuboff estimates that homes will be the next target of surveillance capitalism. She refers to the practice of collecting ever more detailed information about citizens as “Economies of depth.” Economies of depth will exert their power in everyday lives as companies turn seemingly innocuous objects like toothbrushes, lightbulbs, and ovens into “smart” technology to render citizens’ lives into profitable data. One way in which economies of depth are already well at work in our lives is through  phones. Smartphones are rich sources of location data for surveillance capitalists. Location services are one of the most significant sources of behavioral surplus for Google.

Firms will defend their collections of behavioral surplus gleaned from these location strategies by claiming that it is all kept as metadata stored in large, anonymous collections, which cannot be used to identify individuals. However, Zuboff argues that when supposedly anonymous metadata is cross-referenced with public record information like birthday, zip code, or sex, it is easy to “de-anonymize” this location data.

Chapter 8 draws to a close by analyzing the power dynamics at work through rendition. Zuboff refers to surveillance capitalism as an intimidating regime and argues that rendition necessitates ignorant consumers who are blind to the true power dynamics behind services such as smart home technology.

The next chapter explores the concept of personalization and its relationship to ubiquitous computing and economies of depth. Personal digital assistants learn from their users and become highly personalized to fit different users’ habits and needs. While personalization is presented as a progressive benefit to consumers, Zuboff argues that personalization is a method used by surveillance capitalism to further secure streams of behavioral surplus data—and mine them at new personal depths.

Unsurprisingly, Google was at the forefront of personal digital assistants. The firm invented Google Now, a digital assistant equipped with a predictive search that combined every system Google had ever invented—including voice search and machine intelligence—to analyze one’s digital habits to predict what a user will ask. Products like Google Now signaled a new type of prediction product that brought surveillance capitalism’s goals of ubiquitous computing to life. In the race to further perfect the personal digital assistant, companies began folding in voice recognition technology to deepen user interactions with these technologies. Exploitative applications of voice recognition technology did not end with digital personal assistants, however. Companies like Samsung and Vizio sold smart TVs that recorded audio of everything said in their vicinity. Genesis Toys marketed products that allowed the toys to capture and process what a child said.

One of the most effective examples of rendition is affective computing, which involves a computer observing humans’ voices and physiological responses and linking them to particular emotions. MIT Professor Rosalind Picard was one prominent intellectual who became invested in affective computing. While Picard admitted that privacy guidelines were necessary to ensure such technology was not abused by governments or others in power, surveillance capitalism latched onto the work of Picard and used it to its own benefit. Facebook, for example, applied for an emotion detection patent in 2014 to gauge user interests in different pages and content. Zuboff notes that Picard later ended up as a CEO as a company called Affectiva, which designed a system of machine intelligence called MindReader in the hopes of creating technology that could identify moods and emotions in autistic children.

Affectiva, however, received significant interest by market research firms and advertisers, and the company took a more commercial turn. Picard led Affectiva to embrace Emotion AI, an artificial intelligence aiming to compile an emotional data repository storing data from video games, conversations, and online videos. Picard even expressed interest in creating an “emotional chip” which could be installed and used to constantly assess the emotions of users on their computers, phones, TVs, and other devices. Although Picard’s intentions with affective computing started off as benign, Zuboff observes that her vision was “subordinated to the larger aims of surveillance capitalism, [and] the thrust of the affective project changed as if distorted in a fun-house mirror” (276). At the chapter’s conclusion, Picard is used as an example to further emphasize how surveillance capitalism overtakes ingenuity and corrupts it into an exploiting force.

Chapter 10 is largely focused on actuation and the impacts of economies of action on human psychology. Building off of the concept of ubiquitous computing, actuation involves ubiquitous action, wherein surveillance capitalism aims to alter one’s choices and behaviors in the real world to increase predictive products’ performances and overall revenue.

The first actuation strategy is tuning, which can include subliminal messages or “nudges.” These occur on websites and user interfaces that are specifically designed to elicit certain patterns of behavior and engagement. The second strategy, herding, is achieved through controlling elements of an environment to force people down a certain path of behavior; for example, a smart TV can be programmed to shut off at a particular hour to make a viewer stop watching and go to bed to encourage healthy sleep schedules. Conditioning is the final strategy of actuation. Inspired by the work of Harvard behaviorist B. F. Skinner, conditioning involves a chain of action in which a stimulus initiates a desired behavior. The behavior is then reinforced, thus shaping specific behavioral patterns.

To illustrate how actuation strategies work, Zuboff turns to the Pokémon Go app, owned by Niantic Labs—which, notably, receives funding from Google and was created by John Hanke, the previous head of Google’s Street View. Hanke identified the mobile app as a significant potential source of behavioral surplus and a promising route to explore actuation methods. The app helped Niantic learn how to condition, herd, and nudge behaviors at an unprecedented scale. Most importantly, Pokémon Go was among the first and most successful applications to modify behaviors outside in real life.

The pattern of conditioning was built right into the game’s framework: A user downloads the app, which then uses GPS and the smartphone camera to hunt Pokémon creatures that “appear” in a user’s surroundings via augmented reality. When a user captures a Pokémon, they are rewarded with game currency and game points. Niantic began sponsoring locations like bars and restaurants, where Pokémon would pop up in large number, thus attracting app users. Niantic then amassed a network of collaborators who, alongside the app creators, enjoyed increased profits as a result of the app’s behavioral modification strategies.

Zuboff then rewinds her chapter’s perspective to the 1970s, where the conceptual roots of behavioral modification lay. She traces the science of manipulating behavior at scale to the American government during the Cold War. Anxieties over the ideological spread of Communism from the East spurred the CIA to create MKUltra, a secret set of experiments that sought to identify methods of behavioral modification and control. The Constitutional Rights Subcommittee convened in 1971 to address public concerns over such research, eventually arguing that the First Amendment must protect one’s rights to independent thought and that privacy must extend to protecting citizens’ thoughts, emotions, and behavior.

The Subcommittee’s arguments lead to a rewriting of psychologists' ethical codes. The National Research Act in 1974 would eventually lead to the Common Rule dictating protections of research subjects in federal studies, and Congress’ creation of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. Zuboff points out that it is interesting to look back on this history of behavioral modification and see how past anxieties were centered around state abuse of behavioral modification; corporate abuses of behaviors were essentially unimaginable. Little did the Senators and privacy rights activists of the 1970s know that the power they fought so hard to defeat would merely respawn again in a different form: surveillance capitalism.

Chapter 11 is preoccupied with the role of free will in individuals’ futures. Surveillance capitalism seeks to corral and curtail freedom of thought and behavior and is therefore a significant threat to humanity’s future, according to the author. In an effort to gather society’s collective bearings, fully understand the threat to our humanity, and face the challenge head-on, Zuboff outlines 16 reasons explaining why surveillance capitalism has been able to succeed. These reasons include surveillance capitalism’s unprecedented nature, its use of the language of conquering invaders, the Cold War historical context that facilitated its rise, and the extent to which surveillance capitalists like Google are shielded from government regulation and scrutiny. Zuboff also points to the dispossession cycle, whereby firms push incursions into public privacy until they are challenged at the government level, which leads to public relations ministrations and surface-level changes to buy time to adjust strategy behind the scenes to continue behavioral data extraction/camouflage it in a new way. Firms also encourage users to become dependent on these services, either to fulfill perceived needs or to feel included.

Moreover, surveillance capitalists present themselves as master innovators and hard workers; people identify with and grow to love their success stories, even aspiring to become leaders of this kind themselves. Surveillance capitalists also present themselves as authorities on the future of our society and the leaders of our collective progress. Many believe in and look up to their subsequent “expertise”. Finally, surveillance capitalism has grown so prevalent that it is essentially impossible to escape. People are essentially left with no alternatives, leading to a sense of “inevitablism” around surveillance capitalism.

In response to these explanations, Zuboff demands laws that reject and dismantle the functions of surveillance capitalism. Furthermore, she argues there must be a collective social movement representing a withdrawal of agreement to surveillance capitalism’s existence.