Hard Choices in AI: Moving Beyond Optimization to Sociotechnical Commitment

TL;DR

As AI enters high-stakes domains like healthcare and surveillance, we face "Hard Choices"—situations where no single mathematical objective can capture the messy reality of human values. This paper argues that AI safety isn't a problem to be "solved" with better code, but a public utility that must be governed through democratic dissent and sociotechnical commitments across the entire development pipeline.

The "Vagueness" Trap: Why Current AI Safety Fails

Most technical approaches to AI safety fall into one of two traps:

1. The Epistemicist Trap: Believing we can perfectly arbitrate and encode "correct" societal norms into a reward function.
2. The Indeterminist Trap: Dismissing value comparisons entirely, treating AI as a neutral tool.

The authors argue that these miss the reality of Normative Uncertainty. When we build a model, we aren't just choosing hyper-parameters; we are deciding what harms are "legal," which populations are protected, and what happens when the system fails. These are not technical "bugs"—they are political and ethical "Hard Choices."

Methodology: The Anatomy of a Hard Choice

The paper leverages Ruth Chang's theory of comparability. In a "hard choice," two options (e.g., Privacy vs. Accuracy) are neither better, worse, nor equal to each other—they are "on a par."

To navigate this, the authors break AI development into four critical stages, each requiring distinct commitments:

• Problematization: Defining what the AI is actually for.
• Featurization: Deciding which parts of the world are "visible" to the data.
• Optimization: Selecting the trade-offs in the loss function.
• Integration: How the model interacts with human users in the real world.

Figure 1: The paper highlights that normative stakes exist at every stage of the pipeline, from scoping to integration.

Democratic Dissent as a Feature, Not a Bug

Instead of trying to find the "one true objective," the authors suggest we look at Elizabeth Anderson’s work on democratic institutions. They propose a framework where stakeholders can offer three types of challenges:

• Formal Challenges: Technical critiques of the model's logic.
• Substantive Challenges: Critiques of the values the model prioritizes.
• Discursive Challenges: Political critiques of the power structures the AI reinforces.

This turns "AI alignment" into an agonistic compromise—a state where we agree to move forward despite lingering disagreements, because the process of dissent was fair and transparent.

Critical Insight: "Neighborhoods" vs. "Sidewalks"

The most profound metaphor in the paper is the distinction between principles and procedures.

• Principles are the "Neighborhoods of Value"—abstract goals like "Fairness" or "Safety."
• Procedures are the "Sidewalks"—the actual sociotechnical infrastructure (data cleaning, threshold setting, UI design) that makes those neighborhoods livable.

Developers often focus on the "Neighborhoods" (Top-down principles) but fail to build the "Sidewalks" (Bottom-up practical procedures) that allow real humans to navigate the system safely.

Conclusion: From Coder to Facilitator

The takeaway for the AI community is clear: a "Safety Culture" is more important than a "Safety Algorithm." Developers must stop seeing themselves as objective engineers and start seeing themselves as facilitators who communicate "hard choices" to stakeholders.

Limitations & Future Work

While the theoretical framework is robust, the paper leaves the computational implementation of "dissent" open. How do we mathematically represent "agonistic compromise" in a gradient descent environment? This is the next frontier for sociotechnical AI research.