Reasoning with Legal Cases: Why AI Still Needs the Human Touch for Analogy

TL;DR

Is legal reasoning just applying a set of rules, or is it about finding the right analogy? This paper argues that while AI has moved toward "Rule Application" for efficiency, it loses the ability to handle "Hard Cases." The authors suggest that for truly novel situations, AI fails because it lacks the "Common Sense" required to map real-world facts to legal categories.

Background: The Shift from Analogy to Rules

Since the late 1980s, AI and Law research has undergone a quiet transformation. Early systems like HYPO and CATO treated legal reasoning as a search for the best analogy—comparing cases to see which side had a "stronger" set of factors. However, since the late 90s, the field has leaned toward Rule Application, where precedents are distilled into a "theory" or a set of defeasible rules before a new case even arrives.

The Core Conflict: Why "Rules" Are Not Enough

The authors, Atkinson and Bench-Capon, highlight a critical flaw in the rule-based approach: The rules eventually "run out."

In Jurisprudence, "Hard Cases" arise when:

1. Gaps Exist: No existing rule covers a unique combination of facts.
2. Magnitude/Switching Points: It’s unclear at what point a factor (like "closeness of relationship") tips from favoring the plaintiff to favoring the defendant.
3. Ascription of Factors: The biggest challenge is not comparing factors, but deciding if they exist in the first place. Is a kindergarten teacher "like" a mother in a negligence case? Rules can't tell you; only an analogy can.

Methodology: Analogy at the Fact Level

The paper posits that we must distinguish between two types of analogy:

• Analogy between Cases: Comparing Case A (factors {f1, f2}) to Case B (factors {f1, f3}).
• Analogy between Real-World Elements: Comparing a "Motor Home" to a "Fixed Dwelling" to determine if the "Expectation of Privacy" factor applies.

(The evolution of AI and Law approaches from HYPO to modern ADF methodologies)

The authors use the case of California v. Carney to illustrate this. The court had to decide if a motor home was more like a vehicle (lower privacy) or a dwelling (higher privacy). This wasn't about rule application; it was about choosing a conceptualization of an object.

The "Ontology" Bottleneck

To make these analogies, a system needs a "Common Sense" ontology—a massive database of how the world works (e.g., "people sleep in dwellings," "vehicles move on roads").

The authors critique the feasibility of this:

• The Scale Problem: Projects like CYC have tried to map common sense for decades with limited success.
• The Subjectivity Problem: In Law, there is no "correct" ontology. The Plaintiff and Defendant offer competing ontologies, and the judge chooses one. Therefore, the "correct" mapping is only known after the decision is made.

Machine Learning: A False Hope?

The paper briefly addresses the rise of Machine Learning (ML) in predicting court outcomes (with success rates around 70-79%). However, it notes significant drawbacks:

1. Transparency: ML "word weights" are not legal "reasons."
2. Evolution: Law changes based on social values; ML models trained on the past might perpetuate outdated biases or fail to recognize a landmark shift in legal thinking.

Critical Insight & Conclusion

The ultimate takeaway is a reality check for "AI Judges." While symbolic AI is excellent at simulating existing law and teaching students, it hits a wall when faced with novel situations.

The Verdict: AI can support lawyers by organizing and retrieving precedents, but the creative act of forming a new analogy to bridge the gap between the messy real world and the structured world of law remains a uniquely human capability.

Takeaways for the Future:

• Hybrid Systems: Future research should focus on combining the interpretative power of LLMs with the formal rigor of Argumentation Frameworks.
• Limits of Automation: We must accept that "Hard Cases" require the exercise of judicial discretion that cannot be pre-programmed into a static ontology.