Amy Webb, futurist, NYU professor, and award-winning author, has spent much of the last decade researching, discussing, and meeting with people and organizations about artificial intelligence. “We’ve reached a fever pitch in all things AI,” she says. Now it’s time to step back to see where it’s going.
This is the task of her new book, The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity, where she takes a bird’s-eye view of trends that, she warns, have put the development of technology on a dangerous path. In the US, Google, Microsoft, Amazon, Facebook, IBM, and Apple (the “G-MAFIA”) are hamstrung by the relentless short-term demands of a capitalistic market, making long-term, thoughtful planning for AI impossible. In China, Tencent, Alibaba, and Baidu are consolidating and mining massive amounts of data to fuel the government’s authoritarian ambitions.
If we don’t change this trajectory, argues Webb, we could be headed straight for catastrophe. But there is still time to act, and a role for everyone to play. MIT Technology Review sat down with her to speak about why she’s worried and what she thinks we can do about it.
The following Q&A has been condensed and edited for clarity.
You mention that we’re currently seeing a convergence of worrying technological, political, and economic trends. Can you expand on what the technological trends are?
When you talk to a researcher working in the field, they will tell you that it’s going to be a very long time before we see a lot of the promises that have been made about AI: things like full automation in vehicles, absolute recognition, or artificial general intelligence—AGI—systems that are capable of cognition and more human-like thought.
From my vantage point, looking to the horizon for the day we have some kind of walking, talking machine, or a machine with a disembodied voice making autonomous decisions, somewhat misses the point. We’re already seeing billions of tiny advancements that will have a compounding effect over time and lead to systems that can autonomously make many decisions at once.
The DeepMind team, for example, has been hard at work on teaching machines how to beat humans playing games. They’ve leapt pretty far ahead in areas like hierarchical reinforcement learning and multitask learning. The latest version of its AlphaGo algorithm, AlphaZero, is capable of learning how to play three games at once without a human in the loop. That’s a fairly big leap. There’s also the very new field of generative adversarial networks, where with a decent-sized corpus of images, you can now generate human faces that looked very, very realistic.
These advancements are not as sexy or as exciting as what we’ve been told about AGI. But if you’re able to take the 40,000-foot view, you can see that we’re heading into a situation in which systems will be making choices for us. And we have to stop and ask ourselves what happens when those systems put aside human strategy in favor of something totally unknown to us.
What about the political and economic trends? Can you describe the ones most worrying to you?
In the United States, the free flow of ideas can spread unencumbered. This is the way Silicon Valley was founded. It has bred both competition and innovation, which is how we got to where we are now with AI among other kinds of technologies.
However, in the US, we also suffer from a tragic lack of foresight. Rather than create a grand strategy for AI or for our long-term futures, the federal government has stripped funding from science and tech research. So the money must come from the private sector. But investors also expect some kind of return. That’s a problem. You can’t schedule your R&D breakthroughs when you’re working on fundamental technology and research. It would be terrific if the big tech companies had the luxury of working really hard without having to produce an annual conference to show off their latest and greatest whiz-bang thing. Instead, we now have countless examples of bad decisions that somebody in the G-MAFIA made, probably because they were working fast. We’re starting to see the negative effects of the tension between doing research that’s in the best interest of humanity and making investors happy.
Now that would be bad enough as it is, right? But this is all happening at the same time that there’s a tremendous amount of power being consolidated in China. China has a sovereign wealth fund devoted to fundamental basic research in AI. They’re throwing huge amounts of money at AI. And they have a totally different idea than the US when it comes to privacy and data. This means they have way more data that can be mined and refined. With a central authority, it’s super easy for the government to test and build AI services that incorporate data from 1.3 billion people. And that’s just within their own country.
Then they have the Belt and Road Initiative, which looks like a traditional infrastructure program but is partly also digital. It isn’t just about building roads and bridges; it’s also about building 5G networks and laying fiber, and about mining and refining data abroad. The deployment of these technologies is a risk to people who care about things like freedom of speech and Western democratic ideals.
Why should we be striving for Western democratic ideals?
It’s a great question. I’ve lived in China, in Japan, and obviously, in the United States. And you could look at the state of our country right now and what’s happening in China and wonder, is that really the worst thing? China’s social credit scoring system, by the way, sounds freakish and awful to Americans, but what a lot of people don’t realize is that self-reporting and monitoring behavior within villages and communities has been part of Chinese culture forever. The social credit score kind of just automates that. So yeah, that’s a terrific question.
I guess what I would say is if I were to look at the idealized version of Chinese communism and the idealized version of Western democracy, I would pick the Western democracy because I think there is a better opportunity for the free flow of ideas and for everyday people to succeed. I think that giving people incentives for individual and personal achievement is a great way to lift up a society, helps us reach our individual potential.
With the direction that the world is headed with AI today, is that a fair comparison? Should we be comparing the idealized versions of Chinese communism with Western democracy, or the worst versions of the two?
That’s a great question because you could argue that pieces of the AI ecosystem are already impacting our Western democratic ideals in a truly negative way. Obviously, everything that’s happened with Facebook serves as an example. But also look what’s going on with the anti-vaxxer community. They’re spreading totally incorrect information about vaccines and basic science. Our American traditions will say freedom of speech, platforms are platforms, we need to let people express themselves. Well, the challenge with that is that algorithms are making choices about editorial content that are leading people to make very bad decisions and getting children sick as a result.
The problem is our technology has become more and more sophisticated, but our thinking on what is free speech and what does a free market economy look like has not become as sophisticated. We tend to resort to very basic interpretations: Free speech means all speech is free unless it butts up against libel law, and that’s the end of the story. That’s not the end of the story. We need to start having a more sophisticated and intelligent conversation about our current laws, our emerging technology, and how we can get those two to meet in the middle.
In other words, you have faith that we will evolve from where we are now to a more idealized version of Western democracy. And you would much prefer that to idealized Chinese communism.
Yeah, I have faith that it’s possible. My huge concern is that everybody is waiting, that we’re dragging our heels and it’s going to take a true catastrophe to make people take action, as though the place we’ve arrived at isn’t catastrophic. But the fact that measles is back in the state of Washington to me is a catastrophic outcome. So is what’s happened in the wake of the election. Regardless of what side of the political spectrum you’re on, I cannot imagine anybody today thinks that the current political climate is good for our futures.
So I absolutely believe that there is a path forward. But we need to get together and bridge the gap between Silicon Valley and DC so that we can all steer the boat in the same direction.
What do you recommend government, companies, universities, and individual consumers do?
The developmental track of AI is a problem, and every one of us has a stake. You, me, my dad, my next-door neighbor, the guy at the Starbucks that I’m walking past right now. So what should everyday people do? Be more aware of who’s using your data and how. Take a few minutes to read work written by smart people and spend a couple minutes to figure out what it is we’re really talking about. Before you sign your life away and start sharing photos of your children, do that in an informed manner. If you’re okay with what it implies and what it could mean later on, fine, but at least have that knowledge first.
Businesses and investors can’t expect to rush a product over and over again. It is setting us up for problems down the road. So they can do things like shore up their hiring processes, significantly increase their efforts to improve inclusivity, and make sure their staff are more representative of what the real world looks like. They can also put on some brakes. Any investment that’s made into an AI company or project or whatever it might be should also include funding and time for checking things like risk and bias.
Universities must create space in their programs for hybrid degrees. They should incentivize CS students to study comparative literature, world religions, microeconomics, cultural anthropology and similar courses in other departments. They should champion dual degree programs in computer science and international relations, theology, political science, philosophy, public health, education and the like. Ethics should not be taught as a stand-alone class, something to simply check off a list. Schools must incentivize even tenured professors to weave complicated discussions of bias, risk, philosophy, religion, gender, and ethics in their courses.
One of my biggest recommendations is the formation of GAIA, what I call the Global Alliance on Intelligence Augmentation. At the moment people around the world have very different attitudes and approaches when it comes to data collection and sharing, what can and should be automated, and what a future with more generally intelligent systems might look like. So I think we should create some kind of central organization that can develop global norms and standards, some kind of guardrails to imbue not just American or Chinese ideals inside AI systems, but worldviews that are much more representative of everybody.
Most of all, we have to be willing to think about this much longer-term, not just five years from now. We need to stop saying, “Well, we can’t predict the future, so let’s not worry about it right now.” It’s true, we can’t predict the future. But we can certainly do a better job of planning for it.
An abridged version of this story originally appeared in our AI newsletter The Algorithm. To have it directly delivered to your inbox, sign up here for free.
How the Dumb Design of a WWII Plane Led to the Macintosh
The B-17 Flying Fortress rolled off the drawing board and onto the runway in a mere 12 months, just in time to become the fearsome workhorse of the US Air Force during World War II. Its astounding toughness made pilots adore it: The B-17 could roar through angry squalls of shrapnel and bullets, emerging pockmarked…
The B-17 Flying Fortress rolled off the drawing board and onto the runway in a mere 12 months, just in time to become the fearsome workhorse of the US Air Force during World War II. Its astounding toughness made pilots adore it: The B-17 could roar through angry squalls of shrapnel and bullets, emerging pockmarked but still airworthy. It was a symbol of American ingenuity, held aloft by four engines, bristling with a dozen machine guns.
Imagine being a pilot of that mighty plane. You know your primary enemy—the Germans and Japanese in your gunsights. But you have another enemy that you can’t see, and it strikes at the most baffling times. Say you’re easing in for another routine landing. You reach down to deploy your landing gear. Suddenly, you hear the scream of metal tearing into the tarmac. You’re rag-dolling around the cockpit while your plane skitters across the runway. A thought flickers across your mind about the gunners below and the other crew: “Whatever has happened to them now, it’s my fault.” When your plane finally lurches to a halt, you wonder to yourself: “How on earth did my plane just crash when everything was going fine? What have I done?”
For all the triumph of America’s new planes and tanks during World War II, a silent reaper stalked the battlefield: accidental deaths and mysterious crashes that no amount of training ever seemed to fix. And it wasn’t until the end of the war that the Air Force finally resolved to figure out what had happened.
To do that, the Air Force called upon a young psychologist at the Aero Medical Laboratory at Wright-Patterson Air Force Base near Dayton, Ohio. Paul Fitts was a handsome man with a soft Tennessee drawl, analytically minded but with a shiny wave of Brylcreemed hair, Elvis-like, which projected a certain suave nonconformity. Decades later, he’d become known as one of the Air Force’s great minds, the person tasked with hardest, weirdest problems—such as figuring out why people saw UFOs.
For now though, he was still trying to make his name with a newly minted PhD in experimental psychology. Having an advanced degree in psychology was still a novelty; with that novelty came a certain authority. Fitts was supposed to know how people think. But his true talent is to realize that he doesn’t.
When the thousands of reports about plane crashes landed on Fitts’s desk, he could have easily looked at them and concluded that they were all the pilot’s fault—that these fools should have never been flying at all. That conclusion would have been in keeping with the times. The original incident reports themselves would typically say “pilot error,” and for decades no more explanation was needed. This was, in fact, the cutting edge of psychology at the time. Because so many new draftees were flooding into the armed forces, psychologists had begun to devise aptitude tests that would find the perfect job for every soldier. If a plane crashed, the prevailing assumption was: That person should not have been flying the plane. Or perhaps they should have simply been better trained. It was their fault.
But as Fitts pored over the Air Force’s crash data, he realized that if “accident prone” pilots really were the cause, there would be randomness in what went wrong in the cockpit. These kinds of people would get hung on anything they operated. It was in their nature to take risks, to let their minds wander while landing a plane. But Fitts didn’t see noise; he saw a pattern. And when he went to talk to the people involved about what actually happened, they told of how confused and terrified they’d been, how little they understood in the seconds when death seemed certain.
The examples slid back and forth on a scale of tragedy to tragicomic: pilots who slammed their planes into the ground after misreading a dial; pilots who fell from the sky never knowing which direction was up; the pilots of B-17s who came in for smooth landings and yet somehow never deployed their landing gear. And others still, who got trapped in a maze of absurdity, like the one who, having jumped into a brand-new plane during a bombing raid by the Japanese, found the instruments completely rearranged. Sweaty with stress, unable to think of anything else to do, he simply ran the plane up and down the runway until the attack ended.
Fitts’ data showed that during one 22-month period of the war, the Air Force reported an astounding 457 crashes just like the one in which our imaginary pilot hit the runway thinking everything was fine. But the culprit was maddeningly obvious for anyone with the patience to look. Fitts’ colleague Alfonse Chapanis did the looking. When he started investigating the airplanes themselves, talking to people about them, sitting in the cockpits, he also didn’t see evidence of poor training. He saw, instead, the impossibility of flying these planes at all. Instead of “pilot error,” he saw what he called, for the first time, “designer error.”
The reason why all those pilots were crashing when their B-17s were easing into a landing was that the flaps and landing gear controls looked exactly the same. The pilots were simply reaching for the landing gear, thinking they were ready to land. And instead, they were pulling the wing flaps, slowing their descent, and driving their planes into the ground with the landing gear still tucked in. Chapanis came up with an ingenious solution: He created a system of distinctively shaped knobs and levers that made it easy to distinguish all the controls of the plane merely by feel, so that there’s no chance of confusion even if you’re flying in the dark.
By law, that ingenious bit of design—known as shape coding—still governs landing gear and wing flaps in every airplane today. And the underlying idea is all around you: It’s why the buttons on your videogame controller are differently shaped, with subtle texture differences so you can tell which is which. It’s why the dials and knobs in your car are all slightly different, depending on what they do. And it’s the reason your virtual buttons on your smartphone adhere to a pattern language.
But Chapanis and Fitts were proposing something deeper than a solution for airplane crashes. Faced with the prospect of soldiers losing their lives to poorly designed machinery, they invented a new paradigm for viewing human behavior. That paradigm lies behind the user-friendly world that we live in every day. They realized that it was absurd to train people to operate a machine and assume they would act perfectly under perfect conditions.
Instead, designing better machines meant figuring how people acted without thinking, in the fog of everyday life, which might never be perfect. You couldn’t assume humans to be perfectly rational sponges for training. You had to take them as they were: distracted, confused, irrational under duress. Only by imagining them at their most limited could you design machines that wouldn’t fail them.
This new paradigm took root slowly at first. But by 1984—four decades after Chapanis and Fitts conducted their first studies—Apple was touting a computer for the rest of us in one of its first print ads for the Macintosh: “On a particularly bright day in Cupertino, California, some particularly bright engineers had a particularly bright idea: Since computers are so smart, wouldn’t it make sense to teach computers about people, instead of teaching people about computers? So it was that those very engineers worked long days and nights and a few legal holidays, teaching silicon chips all about people. How they make mistakes and change their minds. How they refer to file folders and save old phone numbers. How they labor for their livelihoods, and doodle in their spare time.” (Emphasis mine.) And that easy-to-digest language molded the smartphones and seamless technology we live with today.
Along the long and winding path to a user-friendly world, Fitts and Chapanis laid the most important brick. They realized that as much as humans might learn, they would always be prone to err—and they inevitably brought presuppositions about how things should work to everything they used. This wasn’t something you could teach of existence. In some sense, our limitations and preconceptions are what it means to be human—and only by understanding those presumptions could you design a better world.
Today, this paradigm shift has produced trillions in economic value. We now presume that apps that reorder the entire economy should require no instruction manual at all; some of the most advanced computers ever made now come with only cursory instructions that say little more than “turn it on.” This is one of the great achievements of the last century of technological progress, with a place right alongside GPS, Arpanet, and the personal computer itself.
It’s also an achievement that remains unappreciated because we assume this is the way things should be. But with the assumption that even new technologies need absolutely no explaining comes a dark side: When new gadgets make assumptions about how we behave, they force unseen choices upon us. They don’t merely defer to our desires. They shape them.
User friendliness is simply the fit between the objects around us and the ways we behave. So while we might think that the user-friendly world is one of making user-friendly things, the bigger truth is that design doesn’t rely on artifacts; it relies on our patterns. The truest material for making new things isn’t aluminum or carbon fiber. It’s behavior. And today, our behavior is being shaped and molded in ways both magical and mystifying, precisely because it happens so seamlessly.
I got a taste of this seductive, user-friendly magic recently, when I went to Miami to tour a full-scale replica of Carnival Cruise’s so-called Ocean Medallion experience. I began my tour in a fake living room, with two of the best-looking project staffers pretending to be husband and wife, showing me how the whole thing was supposed to go.
Using the app, you could reserve all your activities way before you boarded the ship. And once on board, all you needed was to carry was a disk the size of a quarter; using that, any one of the 4,000 touchscreens on the ship could beam you personalized information, such which way you needed to go for your next reservation. The experience recalled not just scenes from Her and Minority Report, but computer-science manifestos from the late 1980s that imagined a suite of gadgets that would adapt to who you are, morphing to your needs in the moment.
Behind the curtains, in the makeshift workspace, a giant whiteboard wall was covered with a sprawling map of all the inputs that flow into some 100 different algorithms that crunch every bit of a passenger’s preference behavior to create something called the “Personal Genome.” If Jessica from Dayton wanted sunscreen and a mai tai, she could order them on her phone, and a steward would deliver them in person, anywhere across the sprawling ship.
The server would greet Jessica by name, and maybe ask if she was excited about her kitesurfing lesson. Over dinner, if Jessica wanted to plan an excursion with friends, she could pull up her phone and get recommendations based on the overlapping tastes of the people she was sitting with. If only some people like fitness and others love history, then maybe they’ll all like a walking tour of the market at the next port.
Jessica’s Personal Genome would be recalculated three times a second by 100 different algorithms using millions of data points that encompassed nearly anything she did on the ship: How long she lingered on a recommendation for a sightseeing tour; the options that she didn’t linger on at all; how long she’d actually spent in various parts of the ship; and what’s nearby at that very moment or happening soon. If, while in her room, she had watched one of Carnival’s slickly produced travel shows and seen something about a market tour at one her ports of call, she’d later get a recommendation for that exact same tour when the time was right. “Social engagement is one of the things being calculated, and so is the nuance of the context,” one of the executives giving me the tour said.
Subscribe to WIRED and stay smart with more of your favorite writers.
It was like having a right-click for the real world. Standing on the mocked-up sundeck, knowing that whatever I wanted would find me, and that whatever I might want would find its way either onto the app or the screens that lit up around the cruise ship as I walked around, it wasn’t hard to see how many other businesses might try to do the same thing. In the era following World War II, the idea that designers could make the world easier to understand was a breakthrough.
But today, “I understand what I should do” has become “I don’t need to think at all.” For businesses, intuitiveness has now become mandatory, because there are fortunes to be made by making things just a tad more frictionless. “One way to view this is creating this kind of frictionless experience is an option. Another way to look at it is that there’s no choice,” said John Padgett, the Carnival executive who had shepherded the Ocean Medallion to life. “For millennials, value is important. But hassle is more important, because the era they’ve grow up in. It’s table stakes. You have to be hassle-free to get them to participate.”
By that logic, the real world was getting to be disappointing when compared with the frictionless ease of this increasingly virtual world. Taken as a whole, Carnival’s vision for seamless customer service that can anticipate your every whim was like an Uber for everything, powered by Netflix recommendations for meatspace. And these are in fact the experiences that many more designers will soon be striving for: invisible, everywhere, perfectly tailored, with no edges between one place and the next. Padgett described this as a “market of one,” in which everything you saw would be only the thing you want.
The Market of One suggests to me a break point in the very idea of user friendliness. When Chapanis and Fitts were laying the seeds of the user-friendly world, they had to find the principles that underlie how we expect the world to behave. They had to preach the idea that products built on our assumptions about how things should work would eventually make even the most complex things easy to understand.
Steve Jobs’ dream of a “bicycle for the mind”—a universal tool that might expand the reach of anyone—has arrived. High technology has made our lives easier; made us better at our jobs, and created jobs that never existed before; it has made the people we care about closer to us. But friction also has value: It’s friction that makes us question whether we do in fact need the thing we want. Friction is the path to introspection. Infinite ease quickly becomes the path of least resistance; it saps our free will, making us submit to someone else’s guess about who we are. We can’t let that pass. We have to become cannier, more critical consumers of the user-friendly world. Otherwise, we risk blundering into more crashes that we’ll only understand after the worst has already happened.
Excerpted from USER FRIENDLY: How the Hidden Rules of Design Are Changing the Way We Live, Work, and Play by Cliff Kuang with Robert Fabricant. Published by MCD, an imprint of Farrar, Straus and Giroux November 19th 2019. Copyright © 2019 by Cliff Kuang and Robert Fabricant. All rights reserved.
When you buy something using the retail links in our stories, we may earn a small affiliate commission. Read more about how this works.
More Great WIRED Stories
- The super-optimized dirt that helps keep racehorses safe
- The 12 best foreign horror movies you can stream right now
- VSCO girls are just banal Victorian archetypes
- Google’s .new shortcuts are here to simplify your life
- The delicate ethics of using facial recognition in schools
- 👁 Prepare for the deepfake era of video; plus, check out the latest news on AI
- 💻 Upgrade your work game with our Gear team’s favorite laptops, keyboards, typing alternatives, and noise-canceling headphones
A Tesla Cybertruck Mishap, a Massive Data Leak, and More News
Hackers are stealing and Elon is squealing, but first: a cartoon about subscription dreams.Here’s the news you need to know, in two minutes or less.Want to receive this two-minute roundup as an email every weekday? Sign up here!Today’s NewsMeet the Tesla Cybertruck, Elon Musk’s Ford-fighting pickup truckTesla CEO Elon Musk last night unveiled his newest…
Hackers are stealing and Elon is squealing, but first: a cartoon about subscription dreams.
Here’s the news you need to know, in two minutes or less.
Want to receive this two-minute roundup as an email every weekday? Sign up here!
Meet the Tesla Cybertruck, Elon Musk’s Ford-fighting pickup truck
Tesla CEO Elon Musk last night unveiled his newest baby, an all-electric pickup called the Tesla Cybertruck. He demonstrated that it can take a sledgehammer to the door with nary a scratch, and he also accidentally demonstrated that it can’t take a ball to the window. But behind the showmanship and Elon’s audible disbelief at the onstage mishap is a truck with a 500-mile range and the torque that comes from an electric motor. It represents an important new market expansion for Tesla. Now it just has to actually put the darn thing into production.
1.2 billion records found exposed online in a single server
Hackers have long used stolen personal data to break into accounts and wreak havoc. And a dark web researcher found one data trove sitting exposed on an unsecured server. The 1.2 billion records don’t include passwords, credit card numbers, or Social Security numbers, but they do contain cell phone numbers, social media profiles, and email addresses—a great start for someone trying to steal your identity.
Fast Fact: 2025
That’s the year NASA expects to launch the first dedicated mission to Europa, where water vapor was recently discovered. The mission to Jupiter’s moon will involve peering beneath Europa’s icy shell for evidence of life.
WIRED Recommends: The Gadget Lab Newsletter
First of all, you should sign up for WIRED’s Gadget Lab newsletter, because every Thursday you’ll get the best stories about the coolest gadgets right in your inbox. Second of all, it will give you access to early Black Friday and Cyber Monday deals so you can get your shopping done early.
News You Can Use:
Here’s how to hide nasty replies to your tweets on Twitter.
This daily roundup is available as a newsletter. You can sign up right here to make sure you get the news delivered fresh to your inbox every weekday!