Beyond Programming: Why AI Cannot Truly Adhere to Moral Norms

TL;DR

Can a machine be "good" if it is impossible for it to be "bad"? This paper argues that Moral Agency requires more than just following rules (Norm-Compliance); it requires Deliberate Norm-Adherence. Because current AI is tethered to its programming and lacks the capacity to intentionally violate norms or truly endorse them, it remains outside the realm of moral accountability.

The Background: Lowering the Bar for Machines

In the rapidly evolving landscape of AI ethics, scholars like Floridi and Sanders have suggested that as AI becomes more autonomous and adaptive, we should grant it the status of a moral agent. However, this paper argues that such views have "lowered the threshold" for agency. Just because a machine's behavior aligns with ethical principles doesn't mean the machine is a moral agent.

The Core Insight: The Necessity of Norm-Violation

The author's most provocative claim is that to adhere to a norm, one must have the capacity to violate it. This follows a Kantian intuition: acting in accordance with duty (doing what is right by accident or force) is not the same as acting from duty (choosing to do right because it is right).

1. Norm-Compliance vs. Norm-Endorsement

Imagine a person in Thailand who doesn't step on a coin because they've been raised never to do it—it's just a reflex. Compare this to a visitor who understands the norm, recognizes its authority, and chooses to respect it even if they find it strange.

• Current AI acts like the reflex: It complies because it is hard-wired.
• True Moral Agents act like the visitor: They endorse the norm as their own.

2. The Sociopath Comparison

The paper uses the example of a sociopath to illustrate the "Affective Foundation." A sociopath might know the rules (Cognitive Foundation) but doesn't care about them (lacks Affective Foundation). AI is in an even tougher spot; it doesn't "know" the rules as norms—it only executes codes.

The Constitutive Pitfall

The paper draws a fascinating parallel between AI and "constitutivist" theories of human agency. Constitutivists argue that following certain norms is what defines being an agent. The paper points out a logical trap here: if following norms is constitutive of being an agent, then violating a norm would mean you cease to be an agent.

The author argues that for AI, the link is even stricter. For a machine, following its code is constitutive of its existence; if it "breaks" the code, it isn't exercising agency—it is simply malfunctioning.

Critical Analysis: The Programming Paradox

The fundamental limitation is that AI derives its authority from Facts (code), not Norms (values).

• Cognitive Criterion: AI can identify patterns, but it cannot "grasp the concept of normativity."
• Affective Criterion: AI lacks the motivational drive or "feeling" that makes an action moral.
• The Status Quo: As long as AI follows "hard-wired" protocols, it cannot "decide" to deviate. Without the choice to be immoral, its "moral" behavior is just successful execution.

Conclusion: A Limited Moral Status

The paper concludes that we should be cautious about granting moral agency to AI. While AI can be an Implicit Ethical Agent (performing beneficial actions by design), it lacks the depth required for Explicit Moral Accountability.

Takeaway for AI Safety and Research

This research suggests that the path to "Moral AI" isn't just about better alignment or more rules. Instead, it poses a deeper challenge: How do we build a system that understands why a rule matters and has the "freedom" to respect it, rather than just the "instruction" to follow it?

(Note: The paper emphasizes the distinction between norm-compliance and deliberate adherence as the primary barrier for machine ethics.)