In 2001: A Space Odyssey, Arthur C. Clarke came very close to predicting the future in one of his laws. He wrote that “Any sufficiently advanced technology is indistinguishable from magic.”

In today’s world, in which technological innovation seems to be moving forward at warp speed, the revered science fiction writer seems to be closer to the truth than we might think…

Technology is in the air

In just ten or so short years, technology has gradually become invisible. The cumbersome computers that once took up space on our desks have morphed into tablets and smartphones. Bulky cathode-ray tubes have yielded floor space to flat-screen TVs, that some people even use as works of art. The wires that once connected telephones to the network have disappeared, leaving behind WiFi waves and 4G. Technology is now omnipresent in our daily lives. But it has moved out of sight.

Screens themselves also seem to be disappearing, giving way to connected speakers and voice command technology. We are more surrounded by technology than ever, but let’s face it… we can no longer see it. The ultimate expression of this disappearance is Amazon Go. In reality, there is nothing technological about the customer experience at Amazon’s checkout-free supermarket. You go in, take what you need from the shelves, fill your basket and leave. All the shop’s electronic equipment – sensors, cameras, and of course computers – is hidden behind the scenes, out of customers’ view.

From their perspective the shopping experience is no more “digital” than buying a lemon on a Friday evening from your local grocer’s. All the technology that Amazon uses has actually become completely transparent.

But there is still a magical element about this shop, in which there are no human interactions at all. Helping yourself, leaving the shop, and seeing your bank account automatically debited with the correct amount is the most positive brand of magic that modern technology can offer!

Power and Data

The magical Amazon Go experience owes its appeal to a whole host of technological innovations that we now place under the umbrella of “Artificial Intelligence”. Amazon Go uses weight sensors that detect when a product leaves a shelf. Cameras follow buyers and identify their movements. And there are computers that can link up these sources of information and determine who has purchased what.

In a nutshell, the boom in information transfer and processing capabilities is what has made Amazon Go possible. While just a few years ago, we were still strugging to analyse complex statistics, the increase in computers’ processing abilities and the deployment of high-speed networks (fibre and soon 5G) mean that we can now process images or videos in real time. Computers have learnt how to handle eminently complex data on our behalf.
Image interpretation is commonplace, and opens up a plethora of new possiblities. If only to mention some of the most striking: the facial recognition experiments that were carried out in Shenzen, China: detection of cheating during exams, identification of a criminal inside a crowded stadium, etc. not to mention the attribution of a “social” score to inhabitants, depending on their behaviour.

Here again, we are talking about transparent technology that has a very real impact on people’s lives.

The end of empathy

In fact we are seeing the impact of accelerating technology in our everyday experiences already. What criteria can be used to deny a Chinese citizen access to an international flight? And most of all, what can the citizen do to argue against these criteria?

Doesn’t basing administrative decisions on thousands of statistics mean casting shadows on real life?
In the United States, local authorities are using AI to decide how social benefits should be attributed. An increasing number of cases are being processed “digitally”, based on solely objective criteria. By developing fully automatic, and therefore “objective” systems, the authorities are actually creating discord.

Most federal employees using these systems have found them to be a way of relinquishing the burden of responsibility: “Our system has denied you access to this loan”. For beneficiaries, the programmes are seen as the end of empathy and human understanding. A refusal that is backed up with the humanity and compassion of a real-life personal explanation is easier to accept than when the decision is generated by a heartless source of artificial intelligence, which allows no response or opposition to its arguments.

Lost in digitalisation

In addition to Arthur C. Clarke’s “magic”, there is a fear of a certain modern illiteracy, the human mind becoming unable to understand the ins and outs of a decision. Because obviously, the ability of computers to store and process information bears no relation to human intelligence.

When faced with a decision involving an algorithm that impacts our daily lives, we don’t know how to react. Simply because we can’t understand and discuss the various factors involved in the decision. A counsellor can present arguments, even though they may be clumsy, whereas AI remains cold and explains nothing.

In fact, that’s the problem. The very criteria that make AI such an efficient mediator are – by definition – too complex to be understood by the people who are affected by them. How can the Shenzhen resident who is refused access to an international flight understand the reasons for this refusal? And above all, is the person given the opportunity to know how their actions may impact their visa request before it all begins?

And that’s what digital illiteracy is: not understanding the impact that technology has on our daily lives, and feeling that we are losing all control. It’s a type of curse.

While progress in artificial intelligence has aroused fear about the destruction of humanity – known as the Skynet syndrome – we worry somewhat less about the stranglehold that algorythms have on our daily lives. No longer understanding the world around us, how we interact and make decisions, and above all what impact our actions may have, are increasing dangers for our society.

While Arthur C. Clarke did predict the rise of magic, he couldn’t have known that it could be of the black variety.

Translated into English by Ruth Simpson

This article is part of Serviceplan’s Twelve #5 issue.

This page is available in FR