This site is intended to be viewed with javascript enabled; please upgrade or reset your browser.

See here for more details.

Literature and Ideas

Engines of Death

On 15 September 1830, George Stephenson was driving his newly built steam engine, the Rocket, on towards Manchester. He had been celebrating the official opening of a new railway route, the world’s first inter-city line, which was to bind together the two industrial megacities of North-West England, Liverpool and Manchester. It was an unprecedented feat of engineering, but the heavens offered little in the way of thanks: stormclouds had gathered and the engine was beaten by a merciless swarm of hailstones. Stephenson was racing through the piercing rain, reaching almost 40 MPH (breaking, in the process, the legal limit of 17 MPH that was subsequently set for drivers, breaking even the world speed record), but he was not fleeing simply from the spiteful skies, the unremitting storm that surrounded him: there had been an injury on the line.

A local Member of Parliament, William Huskisson, had fallen onto the opposing track during a scheduled stop and his leg had been shattered by the weight of the oncoming Rocket. Having left him to convalesce in the small town of Eccles (where it was considered that there would be less chance of a mob forming to attack the assembled dignitaries, including the much loathed Prime Minister, the Duke of Wellington), Stephenson went on to Manchester to fetch surgical help. All to no avail, as within hours the poor man had perished in pangs of torturous pain. And so the opening of the track had simultaneously produced the first fatality of what was to become the railway age.

The unveiling of the memorial to William Huskisson MP.

Except that firsts are rarely really firsts — and this one wasn’t. Despite what the Oxford Dictionary of National Biography, amongst other such serious sources, may have us believe. At least two people had died on the Liverpool–Manchester line even before it had officially opened, one of whose identities was recorded only as that of a blind, American beggar-woman. But who remembers the down and out, whose dismal lives are forgotten well before any final coup de châssis? Maybe we’ll remember Elaine Herzberg, who lived on the streets of urban Arizona and this week fell under a self-driving car, the first fatality of our now driverless age.

Yet, of course, she wasn’t. She was the first pedestrian to have been killed. Joshua Brown, a former US Navy SEAL, a thrillseeker and a technophile, had already died behind the wheel of his semi-autonomous Tesla two years earlier. The onboard computer system failed to differentiate between the bright white of the van that he swiftly ploughed into and the sunlight surrounding it. Herzberg was the victim of an equal but opposite error when crossing the street (where, truth be told, she ought not to have crossed) in front of a car from the fleet of Uber’s self-driving roadtest, the black of her shadow indistinguishable from night.

The systems work, when they work, by building constantly updating models of their surroundings, a virtual world of bounding boxes, often based on LiDAR scans that provide accurate positions for nearby vehicles and people. Training data is first fed into the systems (for example, hundreds of thousands of images with the bounding boxes already found by human hand and eye) and then, through practice, the artificial intelligence begins to recognise similar objects on real roads. Models, however, are never complete representations of the external world, they are always a necessary simplification, and if the artificial model should ever depart too significantly from the true scene existing outside the glass, sheet metal and plastic, wrong moves can be made by the purblind machine.

Cars as seen by computers.
Image from Playment.

In the short video that has since been released to the public, presumably in order to prove the company’s and the car’s innocence, Herzberg’s silhouette and her jeans lit by the headlights are visible for hardly a snatch of a second before the screen fades to absolute black and we know that she has been hit, almost certainly too short a time for a human (including the one sat in the car, monitoring its operation) to react.

Every death should be grieved, but it must be too much to say that every death is avoidable. We may even think that there ought to be greater outrage at those road deaths where a conscious person runs down another conscious person — around 1.3 million of them every year, a perfectly quotidian peril — than where a machine merely malfunctions; it may, in fact, be the very lack of agency in the latter case that frightens us most. Can you scream and plead, abuse and supplicate a misfiring black box to leave you your life? Will the robots hold your hand and call for an ambulance when they hear the bones crunch, or will they just hit and run? Either way, they’ll certainly be designed to be safer, much safer, than the average flawed sapien has ever been behind the wheel, because customers and regulators will insist on it. Yet there’s still something worse about being run over by a misplaced semicolon in a vast body of corporate-owned code than being hit by a bastard with a name, a face, a family, a date and place of birth, a history, a conscience and a brain.

A still from the video released by Uber.

Though there won’t be any misplaced semicolons, because the algorithms will be worked and written and tested, rewritten and tested and tested again, and again until every last bug has been exterminated. But the algorithms will have axioms, underlying assumptions. And these will be the assumptions of professional ethicists, of people like Philippa Foot who devised one of the most influential of thought experiments, inspired by the age of steam, where out-of-control railway trolleys career dangerously down tracks and the dilemmas consist in whether or not to reroute them with the pull of a lever or the push of an obstacle, weighing up the costs and the casualties (and, most importantly, the moral faults and culpabilities) that each choice or omission could involve. If a malfunctioning car — to give an artificial but not atypical example — can turn left onto oncoming traffic and deliver death to a handful of passengers, creating a mile-long pile-up that could block the arterial road for the rest of the day, or turn right through the school gates and crush a lone girl, killing her and her alone, which way should it turn? The utilitarian, who applies a cold economic calculus to matters of life and limb, would always turn right, but would the human? Would you?

Let’s assume, too, that there are at least some zero-sum scenarios where the course of action that most protects the safety of the autonomous vehicle’s passengers is one that does so at a cost to other drivers or pedestrians. The autonomous vehicle may “decide” to swerve towards a nearby cyclist rather than hit a larger obstacle in the road that could severely injure its passengers onboard. If the system learns from experience, would it not tend to do what most pleases the consumer? And who would choose a car that prioritises the safety of anyone but themselves? As seen in the case of chatbots that spit terms of abuse, our machines can easily pick up the biases, prejudices and vices of their users, unless they are hard-coded with precise, explicit prohibitions against every sort of bad behaviour that they may conceivably meet.

Maybe these are idle questions, because most of us hardly even want to see these cars on our roads anyway. A recent American study found that a majority of respondents were at least “somewhat” worried about driverless technology: a small but significant preponderance of the worriers and the naysayers over the optimists. But how often have majorities and their opinions ruled in democracies? There is money to be made in the efficient movement of goods, and profit lost in paying drivers to do it. Indeed, the perfect business ought really to have no employees at all, since they always end up on the wrong side of the balance sheet; ours, however, is not a perfect world and almost no actually existing business has managed to reach that Platonic ideal of pure profitability. Nonetheless, and primarily through the power of the Internet and its super-connectivity, there are now billion-dollar enterprises with no more than a hundred programmers on the payroll (Craigslist, for example, still has fewer than fifty employees), as close to nothing as could be explained away by a rounding error. The driverless revolution will, likewise, extend this trend into the haulage, transportation and distribution industries.

Which will surely reward the shareholders of the companies who are best able, and fastest, to utilise the technology. While the self-employed contractors who worked, in some cases, 60-hour weeks to make ends meet and, in so doing, allowed Uber to grow to oligopolistic status will be searching for other cakes from which to collect crumbs. Almost 200,000 drivers worldwide currently work for Uber; if the gamble pays off, this figure will fall to near-zero in a matter of years. These affronted lay-offs could form a phalanx of vocal opposition to the spread of automation, and it will be interesting to see whether and how their silence is bought.

However, it’s all too easy to forget how heated was the opposition to the laying of railway lines. Progress tends to rewrite the past as an inexorable race to the present, and that isn’t an unpleasant illusion to uphold — illusions, by nature, rarely are unpleasant. We can only hope that there always remain at least some obstinate people prepared to dig and to discover the long-buried complexities, rather than be seduced by those more meretricious simplicities. The Duke of Wellington, for example, in the decade after he witnessed Huskisson’s death, described rail travel as “the Vulgarest, most indelicate, most inconvenient, most injurious to the Health of any mode of conveyance that I have seen in any part of the World!” Not one to mince words. Yes, engines were fast and efficient, but the smog they produced was foul and the roar deafening. What the Duke hated most of all, however, was other people: the fact that he would be mobbed by middle-class snobs and busybodies at every station. Stagecoaches were slow and (even with the new steel-sprung suspension) often uncomfortable, but at least they were private. In that respect, they were comparable to our cars, each one an island.

Promotional poster produced by British Railways (1948–1965),
exalting the natural beauty of the Peak District.

The art critic and social reformer John Ruskin famously decried the black scar on the landscape left as the cast-iron tracks cut across his beloved Peak District (which in 1951 managed, all the same, to become the first National Park in the UK), turning the valley between Buxton and Bakewell from a sanctuary as “divine as the vale of Tempe” into a mundane, industrial thoroughfare. (Tempe is a verdant region of Thessaly, in mainland Greece, that was prized by ancient poets and favoured by the gods for its natural beauty; ironically, it is also the name of the city in Arizona where Herzberg lived and died.) The very fact that people could move from one town to another so easily meant that the hitherto untamed landscapes in between would inevitably lose their romance: through a carriage window, a beauty spot becomes a blur. These arguments live on in the contemporary complaints that globalisation denudes our cities and countries of their cherished idiosyncrasies, their charms, as many of the same big businesses construct their strip malls, warehouses and near-identical stores on top of our neighbourhoods. That is, if we ever look up long enough from the apps on our phone to notice.

Just as with current worries about automation destroying or stripping the dignity from blue-collar jobs, the railways, said some, not only needlessly uglified the landscape but brutalised the workers who built them. Navvies on the Liverpool–Manchester line, for instance, who put down their tools and went on strike in support of a sacked colleague were charged with breach of contract and soon sentenced to hard labour, working the prison’s treadmill for six hours a day. Henry David Thoreau, with a particularly memorable metaphor, expressed his concern with the partial pictures that progress likes to paint:

Did you ever think what those sleepers are that underlie the railroad? Each one is a man, an Irishman, or a Yankee man. The rails are laid on them, and they are covered with sand, and the cars run smoothly over them. They are sound sleepers, I assure you. And every few years a new lot is laid down and run over; so that, if some have the pleasure of riding on a rail, others have the misfortune to be ridden upon.

Walden

Our cities were transformed by the first wave of car ownership; in Britain, perhaps the decisive decade was the 1950s, when the motorways began to be built. Roads were widened, infrastructure centralised and populations dispersed. Needless to say, the self-driving car will precipitate changes that are just as far-reaching, most of which we can hardly now predict. We are told that the increased efficiency of automated driving will necessitate less fuel being used, even before electrification, and fewer pollutants being emitted. This may be so, but increased uptake is bound to negate these effects, as we’ve seen with previous efficiencies: more vehicles on the road will mean more pollution. Not only will passengers not need a drivers’ licence, but there need not even be passengers. Imagine a world where those wealthy enough can command whole armadas of unpeopled automobiles (or drones, which are much the same technology, raised to the third dimension) to speed around town, collecting deliveries on behalf of their owners. Imagine, too, the congestion.

Now imagine cars where the front seats are facing the seats behind, rather than the road, perhaps with a table in between, so that the passengers can chat and eat and troll the Twitterati, file tax returns and peruse fake news on plugged-in devices. Imagine a time when commuting has become an entertainment industry. Maybe all the windows will be permanently tinted. Will there be wretched souls throwing themselves under the wheels in the reassuring knowledge that no driver will be emotionally harmed by the sight of their suicide? For the railway has long been associated with self-destruction much more than the kind of fatal accidents that inaugurated its birth. Already by the 1870s, Anna Karenina provided a tragic example for copycat readers. A woman redeeming her marital missteps by mangling herself under the advances of warm steel. Sophia Tolstoy herself attempted to leap under a train shortly after the publication of her husband’s great novel; at least, she had hatched the plan, but was soon persuaded not to put it into practice. There’s an inevitability of the tracks, and the timetables, that evokes grand notions of fate — and fateful decisions. Computered cars will allow us to bring even higher levels of order to the world, but humans, given the chance, will always find their ways to reintroduce chaos.

The techno-utopians tell us that they can bring about a world of unprecedented safety, efficiency and ease. They tell us that 90% of all deaths on the road are the result of human error (for cars don’t kill us, drivers do), and that so long, therefore, as we hesitate in giving over control to the machines and their owners, so long as we put obstacles on the road that they’re building to our brighter future, we are responsible for the needless harms of the present. Perhaps they’re right, but what if that seductive story is a simplification of the truth? What if driverless cars are like so many of the other exciting technologies to have solved a pressing problem of the day only to leave more, and more intractable, problems in their wake? It’s one thing to be killed by human error or evil, but another to lose a loved one to a crash created by algorithms designed to make the best of all possible choices in the best of all possible worlds. For now, in our imperfect present, let us spend a moment to remember those fatal footnotes in our perpetual history of progress.

No comments — you could leave the first.

Please login or register to leave a comment.