“The temple is not the market. That is what makes it the temple.” – Anonymous
Much has been said of late about the rise of autonomous weapon systems, unmanned aerial vehicles, and the increased automation of war more broadly. Alongside this monumental technological shift, there have been increasing calls for governance, accountability, traceability, and above all, the insistence that there always remain a human in the loop.
But this phrase, now something of a policy mantra, is often treated as self-explanatory when it is anything but.
The deeper question, and the proper focus of this article, is not whether a human should remain “in the loop,” but rather, what kind of human that human should be. Any human whatsoever? Or, rather, a particular kind of human; one formed by certain practices, disciplines, habits, loyalties, and ways of being in the world? What competencies, what experiences, what virtues, what memories, what scars, what hard-earned wisdom are presupposed in the one or ones who are entrusted to exercise meaningful judgment over life-and-death decisions mediated by these increasingly complex and increasingly opaque technological systems?
Herein lies a danger that is often missed. It is not only the familiar fear that our machines of war could one day suddenly go haywire and annihilate humanity in some singular, apocalyptic event. Rather, the deeper and far more sinister risk is much slower, more subtle, and arguably more pervasive: that in orienting ourselves ever more toward quantification, efficiency, and automated optimization, in war and in life in general, we risk gradually eroding the very capacities, attributes, and values that make meaningful human judgement even possible.
For the machines need not kill us outright in order to end humanity as we know it if they instead radically reshape us into beings who can no longer remember what it is to be human. The end result is all the same.
This, we might describe as the Wall-E problem; not a sudden life extinguishing technological catastrophe, but a slow anthropological and spiritual decay; a world in which human beings are increasingly passive, optimized, and technologically mediated, while the space of lived judgment, agency, and direct encounter is progressively outsourced to systems designed for efficiency and convenience rather than virtue or purpose.
What is striking in all of this is not the technological systems or capacities themselves, but the worldview that underwrites it. War, politics, and moral judgment are increasingly approached as problems of system design rather than practices requiring virtue or formation of character. The underlying assumption is that the right systems architecture will compensate for the absence of the right kind of person.
But this is precisely what is in question.
A system can only be as humane or virtuous or courageous as the human beings who reside within it. And yet, in modern, contemporary society we have little serious account of what such humans are supposed to be or resemble.
What, then, is the antidote?
We might begin by recalling older languages that modernity has largely forgotten: phronesis, practical wisdom; virtue; judgment shaped by experience rather than by pure abstraction. The distinction between those who know life only through models and abstractions and those who have encountered it directly. The difference between first-hand experience and second-hand simulation.
There is, for example, a meaningful contrast between the “first-hander” versus the person who knows only of mediated reality. One learns from contact with the world as it resists one’s expectations; the other learns from representations and idealizations that rarely if ever push back. In this sense, modern life increasingly risks becoming a “copy of a copy of a copy,” in which reality is flattened into metrics, metrics into algorithms, algorithms into entire ways of life.
Consider, by analogy, the stark contrast between hunting or fishing for one’s food and receiving factory-farmed, fully processed food-products via the latest delivery app. In one case, there is exposure to risk, patience, bodily engagement, competency, and the irreducible contingency of the natural world. In the other, food becomes a product of logistical abstraction, available at the tap of a screen. Nothing about this is inherently immoral. But it is formative. It shapes the kind of agent one becomes.
Similarly, decision-making in war and ethics can become overwhelmed by casuistry, hypotheticals, and model-driven, algorithmic abstractions. The result is often not deeper wisdom, but often a kind of paralysis or detachment, a drowning in scenarios that have no grounding in lived reality.
Here the distinction between ideal and non-ideal theory becomes relevant. Ideal theory tends to operate at a level of abstraction that presumes away precisely the conditions under which human judgment is most needed. Non-ideal theory brings us back to constraint, imperfection, and lived limitation. But even this distinction can be too academic if it is not anchored in the formation of persons capable of acting under duress, time pressures, and radical uncertainty.
What is needed is not only better models, but better humans within these models.
The deeper concern, then, is not simply technological, but anthropological and moral. We risk entering a condition that thinker, René Guénon, once diagnosed as the “reign of quantity”; a world in which what can be measured increasingly displaces what actually matters. In such a world, the immeasurable, the qualitative, the ineffable, the sublime, is not merely ignored; it is actively eroded or worse; dismissed altogether.
And yet it is precisely in the immeasurable where judgment, prudence, courage, restraint, compassion, and “the human” in the loop must ultimately reside.
Indeed, there is a violence we do to reality when we insist on over-measuring it. And there is a further violence we do when we forget that not everything that matters can be measured. As the poet, William Wordsworth, once warned, “we murder to dissect.” The attempt to render all things into mere concept or number can therefore end up destroying the very subject we most value.
One can see this dynamic in contemporary attempts to optimize friendship, attention, or social media clout, treating human relationships as bloodless systems to be tuned, scaled, and improved. But friendship is not that kind of thing; neither is trust, nor judgment.
At its limit, such singular logic produces clear absurdities: the idea that one could ‘optimize’ friendship by trading one friend for two, two for four, four for eight, and so on, as though friendship was nothing more than a portfolio to be maximized. Indeed, if we encountered someone who thought this way about friendship, then most of us would conclude that such a person didn’t understand the idea of friendship at all. Yet this pervasive ethic, of never-ending optimization and efficiency maximization has seeped into the bones of nearly every facet of day-to-day human life, to include war and killing.
Returning then to the subject at hand, the question, then, is not whether we will have humans in the loop. We will. The question is whether those humans will still be recognizably human in the relevant senses we care about: capable of humility, solidarity, restraint, empathy, courage, discernment, and moral attention not capturable by spreadsheets, algorithms, or the latest data analytics.
Tradition, ritual, ceremony, physical trials, and shared cultural practices of old once played an important role in forming such humans. They created spaces resistant to total quantification; spaces in which judgment, trust, and grit could be inwardly cultivated rather than externally outsourced. In an age of dwindling attention and competing attention economies, such spaces, such ports in the informational storm, are increasingly rare, and therefore increasingly important.
To insist on meaningful human control, then, is not merely a regulatory requirement. It is a philosophical claim about the irreducibility of human judgment. But that claim must be matched by practices that preserve the conditions under which such judgment is still possible.
We should therefore be cautious. In our effort to defend humanity through increasingly automated systems, we may inadvertently reshape humanity in order to fit the systems we have built. And in doing so, we risk producing agents who are efficient, compliant, and ‘optimized’ in every sense of the word, but no longer capable of the kinds of judgments that make “meaningful human control” meaningful or human in the first place.
In defending humanity, we must also equally remember how to preserve it as well.
For the deepest danger is not that our weapons become autonomous; it is that we ultimately cede our autonomy to them and become indistinguishable from the weapons themselves.