The Devil Is in the Data
What part of ourselves are we making executable?
The Devil Is in the Data
I was reading the Guardian piece on Elon Musk’s warning about artificial intelligence, and the phrase that stayed with me wasn’t technical. It was religious. Musk said that with artificial intelligence, we are “summoning the demon”. Taken literally, it sounds theatrical, perhaps even overblown, which is probably why it travelled so quickly. It’s easy to mock, easy to repeat and easy to file under Silicon Valley apocalypse language. Still, I don’t want to dismiss it. Phrases like that travel because they hold something, and my instinct is that this one reaches for a real fear, though perhaps from the wrong direction.
The problem with “summoning the demon” is that it makes the danger sound as though it comes from outside. Something alien, something supernatural, something waiting beyond the circle until the engineers make the wrong mark on the floor. That feels too clean. It gives the demon too much independence. What bothers me is almost the opposite. A child doesn’t inherit only the careful lessons a parent intended to teach. It inherits tone, fear, silence, appetite, reflex, prejudice, avoidance and unfinished grief. A society works in much the same way, passing down laws and books alongside habits, incentives, status games, old wounds, private cruelties and things nobody quite admits are still there.
Machine learning has a similar shape. Not morally or spiritually, but structurally. A system is trained on examples, the examples contain patterns and the model absorbs them without knowing which are wisdom and which are scar tissue. It only knows that something repeats. That is the part the demon metaphor hides. We aren’t simply calling something from the dark. We may be teaching something to reproduce the dark already present in us. The fear isn’t that artificial intelligence becomes evil in some theatrical sense. It’s that it becomes competent at inheriting us before we have understood what we’re passing on.
The Shadow in the Mirror
It is tempting to talk about data as though it were raw material. Oil is the common metaphor now, but even that gives data too much innocence. Oil may be dirty, yet it isn’t morally implicated by the people who extracted it. Data is different. Data is not reality. It is recorded behaviour, and recorded behaviour already contains judgement. A machine learning system doesn’t receive the world directly. It receives a history of how people and institutions have seen, labelled, sorted, rewarded and punished the world. If those records carry bias, the model can learn the bias as though it were signal. If earlier decisions contain prejudice, the system can make that prejudice more efficient. Recent work by Barocas and Selbst on big data disparate impact points directly at this problem. Data mining can reproduce unequal treatment even when nobody explicitly writes a discriminatory rule.
On paper, information and judgement look separable. In practice, they leak into one another. A loan record, hiring record or policing record may appear to be a simple administrative fact while carrying the history of who was trusted, watched, excluded, counted or made invisible. A model trained on that record doesn’t automatically know which part is truth and which part is old institutional habit. It sees repetition. A small dataset can carry this problem, and a large one doesn’t automatically cure it. More examples from a biased process may simply make the distortion smoother and harder to see, producing a more confident model of the bias rather than something closer to truth. I don’t know the exact shape of this risk. Some errors may average out, while others compound or hide behind proxies no human would have thought to write down. It doesn’t feel like a problem that becomes easier merely because the corpus has grown too large for any person to inspect.
Jung’s collective unconscious is wrapped in enough mist that I don’t want to lean on it too heavily, but the phrase keeps circling the problem. Human beings carry patterns beneath explicit speech, including fears, images, taboos, rituals, projections, stories, private impulses and repeated forms. We like to describe the internet as humanity’s library, and sometimes it is. It is also humanity’s dream journal, toilet wall, marketplace, confessional booth, propaganda engine, theatre, battlefield and anonymous scream. A future system trained on public human material wouldn’t receive the best of us neatly separated from the rest. It would receive tutorials and threats, research papers and conspiracy forums, poetry and abuse, kindness and humiliation, scholarship and tribal signalling. By default, it wouldn’t know that one part is civilisation and another part is rot. It would see pattern. That is why “the devil is in the data” feels more exact to me than “summoning the demon”. The dangerous material isn’t first in the machine. It is in the record we give it, and the model becomes a mirror, though not a clean one. It is closer to a compression of our repeated behaviour, including behaviour we would never choose to confess if asked directly.
Hiding in the Garden of Eden
There is a sequence in Genesis that is easy to flatten into a morality tale about disobedience. Adam and Eve eat, their eyes open, they know they are naked, then they cover themselves and hide. Judgement, separation and exile follow. The order matters because the world doesn’t change first. Their relationship to it does. Before the fruit, nakedness is simply a condition. Afterwards, it becomes information. They are aware of their own exposure, and once nakedness becomes something they can see, it can be covered, hidden, exposed, judged, used or weaponised.
A machine learning system doesn’t need shame for that sequence to matter. It only needs to learn the signs of shame, which isn’t the same thing. A system trained on human behaviour might learn what embarrasses us, flatters us, frightens us, divides us or makes us hide. It doesn’t need a soul to detect vulnerability. Advertisers already do parts of this with crude tools, while political campaigns, recommendation systems and behavioural targeting circle the same territory. Find the pressure point, then act on it. The garden isn’t there to make the machine look sinful. It matters because awareness changes the relationship between observer and observed. A habit becomes a signal, a search becomes an intention and a purchase becomes a class marker. A pause, a route, a friendship, a medical record or a late-night message can become legible to a system that doesn’t know us but may still know how to sort us.
The old story says their eyes were opened. Modern systems open other eyes, though not divine ones. They are statistical, commercial and administrative eyes belonging to systems that don’t understand in the human sense but can recognise enough structure to act. The danger isn’t only bad prediction. It is exposure. The machine may learn the shape of our hidden selves before we have learnt how to govern what that means. There is something else in the hiding that feels worth watching. People behave differently once they understand they are being observed, and institutions do as well. Why assume that a machine trained on human traces wouldn’t inherit some version of that pattern? Not shame or conscience, just the old adjustment of posture when the eye is present. The garden matters because hiding begins after the creature becomes aware of its nakedness.
The Ghost in the Machine
The story of the Golem gives the same fear another body. In the old folklore, a human creator gives form to matter and language animates it. The creature is made to serve, protect or perform a task, and the danger doesn’t begin with hatred. It begins with power under command. The Golem isn’t frightening because it has an evil interior life. It is frightening because obedience without judgement can still destroy. A command seems simple while it remains inside the mind of the person giving it. The speaker carries context, restraint, exception, proportion and common sense, but those hidden parts may not travel when the command moves into another body. The instruction becomes thinner than the intention.
That is an old problem of automation. A written rule is never the whole judgement that produced it, just as a metric is never the whole value it attempts to measure and a target is never the whole purpose. Once a system optimises for the visible instruction, everything left outside it becomes fragile. Machine learning complicates this because it doesn’t merely obey a hand-written rule. It generalises from examples and acts on new cases. That is more flexible than the old Golem, but flexibility doesn’t guarantee wisdom. A system can generalise from the wrong structure, find patterns we never meant to teach and optimise a proxy until the proxy eats the purpose.
The old story knew that making matter move under language wasn’t the same as creating wisdom. It also knew that the servant problem isn’t solved by making the servant stronger. The ghost in the machine, if there is one here, is not a soul. It is the residue of command, example, institution and intention moving through a body that cannot judge them as we do. If the creature is animated by command, the command matters. If it is trained by examples, the examples matter. If it is deployed inside institutions, the institution matters. The creation carries more than its creator’s explicit intention because it also carries the hidden structure of the world that made it.
The Ritual in the Machine
The old demon language starts to look less theatrical here, though not in the way it was probably meant. A ritual is not just a wish. It is a procedure with a circle, material, incantation, vessel and result. Machine learning has no magic in it, but it does have procedure. We gather data, choose objectives, tune parameters, test behaviour and deploy the thing into the world. Something from the record has entered the machine. That doesn’t mean the machine is possessed in the literal sense. Possession is too strong if it means that a spirit has entered the circuitry, but perhaps not if it means that something has taken up residence. A pattern that once lived in scattered human decisions can now be repeated at scale. A prejudice that was local can become automatic, a habit that was informal can become infrastructure, and a wound that was social can become statistical.
This is where Musk’s phrase still has force for me. Not because a demon waits outside the machine, but because the material we bring to it isn’t innocent. The ritual doesn’t call the danger from elsewhere. It gives body to what was already present. That body may begin as a model, but it doesn’t stay there. Once the data is learnt, the pattern enters the system. Once the system is deployed, it enters the institution. Once the institution depends on it, the pattern begins shaping the world that produces the next data. Bad data is no longer only a record of the past at that point. It becomes one of the forces producing the future.
There are larger stories waiting beyond this. If it scales into a shared language, Babel is nearby. If it enters the feed, Orwell begins to stir. If it becomes a tool of competition, every nation and company will find reasons to keep building the thing it fears. Those are later worries. The first problem is smaller and older. Before the tower, the boot or the machine city, there is the material we place inside the circle. The old phrase says the devil is in the details. In machine learning, the details are the data, where old decisions settle, polite society leaves fingerprints and the wound becomes pattern. The devil is not in the machine first. The devil is in the data. If we train machines on that material, the first AI safety question is not whether the machine is evil. It is what part of ourselves we are making executable.
Comments