This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American
Imagine a world governed by smart technologies engineered to achieve three distinct yet interrelated normative ends: optimized transactional efficiency, resource productivity and human happiness. We could have congestion-free roads—no stop and go, no road rage! Instantaneous, personalized entertainment—no need to search or browse! Successful social interactions—no misunderstanding or missed cues! No surprise ailments, no failures, no missed opportunities! Heck, no surprises of any kind! There are so many imperfections in our world that smart technology could fix.
We do not live in such a world, but the technologies required for it to exist are already being rapidly developed and deployed. Take, for example, the Internet of Things (IoT)—big data, sensors, algorithms, artificial intelligence and various other related technologies. Their promoters make seductive promises. Supposedly, smart phones, grids, cars, homes, clothing and so on will make our lives easier, better, happier.
These claims are rooted deeply in a smart-tech utopian vision that builds from prior techno-utopian visions such as cyber-utopianism as well as from economic-utopian visions such as the Coasean idea of friction-free, perfectly efficient markets and the Taylorist vision of scientifically managed, perfectly productive workers. In our modern digital networked world these visions creep well beyond their original contexts of idealized internet, markets and workplaces. Smart-tech can manage much more of our lives.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
In Reengineering Humanity Evan Selinger and I explain the logic embedded in these visions applies to a much wider range of actions that depend on human labor (time, attention, effort, etcetera), whether driving a car, caring for one’s children, exercising our bodies and minds, or any other human activity. In the not-so-distant future supposedly smart technological systems may be deployed to maximize human productivity throughout our lives. Smart-tech utopianism is driving social and technological progress in the 21st century, yet it seems doomed to end in tyranny.
Here’s why:
Much of what matters about being human would be lost in the smart-tech world. Smart-tech utopianism entails minimizing various costs associated with humans being human. Although counterintuitive to economists and engineers trained to think in terms of efficiency, humans need friction. Whereas congestion on roads may be pure waste, congestion in our lives may not be. Often it’s productive in ways that are hard to measure, at least in dollars or pleasure. Congestion is sometimes an opportunity to socialize or to stop and think about what you’re waiting for.
Consider, for example, time spent waiting in a checkout line at the grocery store. Sometimes it’s a hassle, but sometimes it’s time well spent talking with friends and even strangers. Sometimes it’s also a buffer that allows you time to second-guess a purchase, anticipate regret, exercise your willpower and make a better decision. Efficiency mongers too easily dismiss the value of searching, browsing, being bored, being lost, failing, missing out, daydreaming, being surprised, going off script.
Social engineers armed with oceans of behavioral data see predictably irrational humans as subjects to nudge toward rationality and smart tech as an efficient tool for nudging. They fail, however, to appreciate that a smart-tech world of perfect rationality is a dystopian nightmare wherein humans are reduced to automatons—perhaps “happy” automatons, which is hardly consolation.
There’s more that matters about being human than happiness (although of course happiness counts, too). Sometimes being irrational is the product of the very human tendency to have conflicts among our personal values, and sometimes it is because we contextualize problems and see meaning where the efficiency monger and social engineer only see a transaction cost to be eliminated.
Most people do not subscribe fully or exclusively to the ideals embedded in the smart-tech utopian vision, yet they’ll have no choice but to live by those ideals. They might prioritize values other than efficiency, productivity and happiness. If given the choice, they would not choose to “plug in” to the smart-tech world. Yet that choice will never be given. There’s not going to be a “plug” or a vote. Most people have little to no say in the development and deployment of smart tech, much less in the complex political economy of world building; it’s hardly democratic—although some will argue folks have a say in markets, as consumers.
But this seems laughable. If our experience in the past decade is instructive, preferences for supposedly smart tech will likely be shaped by puffed up claims and blatant omissions. All of the supposedly “free” services and content online actually come at price.
Does it seem reasonable to conclude consumers chose ubiquitous surveillance? No, not really. That’s an argument that only makes sense when made in hindsight with blinders on to ignore inconvenient facts about political economy, techno-social engineering and human psychology. Did people take supposedly free stuff they were given, gradually develop expectations, preferences and habits via experiences in a highly engineered digital networked environment, and become dependent on surveillance capitalism?
Yes, that seems a more reasonable description of the past two decades. Regardless, in many contexts people don’t and won’t have a say at all. Powerful investors have pursued and will continue to pursue their utopian vision of a world governed by smart-tech systems. Because seamless interconnection and ubiquitous deployment are critical features, we should anticipate, evaluate and (I would argue) resist a future where everyone is always switched on by default—in short, tyranny by smart tech.
Much as techno-realism was needed to counter the cyber-libertarian utopianism in the 1990s, we need to get real about supposedly smart tech and the many contentious trade-offs hidden by its utopianism. Nothing less than the fate of humanity is at stake.