Personal Autonomy and Autonomous Systems: Sameness and Difference

This chapter aims to clarify the transformative effects on the concept of personal autonomy resulting from extending the notion to Artificial Intelligence (AI) technologies, particularly in the defense domain. The complexity of this task is increased by the fact that philosophers have offered a wide range of competing accounts of the autonomous agent’s special relation to their own desires and values. I will draw on the concept of autonomy implied in John Martin Fisher and Mark Ravizza’s control theory, which, while explaining the theory and practice of autonomy, also identifies two conditions that cannot be appropriately satisfied by autonomous systems, but only by persons who interact and communicate with one another. Despite the superficial sameness between personal autonomy and autonomous systems, control theory allows us to pinpoint an important distance in their underlying features. The seeming similarity masks indeed a profound conceptual divergence between autonomy as a condition of moral agency and autonomy understood as the characterization of a system’s operational capacities. This supports, at least in principle, the moral argument that entrusting machines with authority over human life and death is ethically wrong, regardless of how advanced the technology might be.