Can Backward Attribution Methods Be Unified as a Two-Player Game on a Network Graph?

Attribution maps become projections of trajectories

The paper's core move is to stop treating an attribution map as the primary object. Instead, the object of study is a distribution over weighted backward trajectories through the network, and the attribution map is only that distribution's projection onto the input states. This reframing is what lets methods that look like unrelated recipes be compared on one footing.

The starting point is the ReLU Net Game of Gaubert and Vlassopoulos, which shows that a ReLU network's forward output is the value of a two-player Stopping Game on its graph: layers are rounds, neurons are states, input values are terminal payoffs, and the scalar output f(x) is the game value. The authors reinterpret the backward pass as a distribution of weighted game trajectories through that same graph.

To do this they introduce a Stopping Decomposition that splits every weight by sign, W = W+ − W−. At each state of the extended graph, players decide whether to stop or continue and where to route. Their equilibrium policies induce a distribution over backward trajectories, and the discounted occupation measure of that distribution at the input states is the attribution map. The decomposition matters because it separates paths with positive and negative contributions to the prediction, information the original game formulation averages out.

Two games recover gradients and alpha-beta-LRP

The framework rests on two constructions. The Stopping Game (SG) adapts the ReLU Net Game to the Stopping Decomposition, and its occupation measures recover the Jacobian. The Routing Game (RG) is a novel entropy-regularized game, and its occupation measures at temperature τ = 1 recover the full alpha-beta-LRP family on ReLU networks.

This is the sense in which the unification is exact rather than analogical: propagation rules become equilibrium policies, and the attribution map is recovered from the discounted occupation measure of the induced trajectory distribution, an integral over the path distribution assigning relevance to each game state. The alpha-beta-LRP parameters have game-space duals: (α, β) set occupation-measure path weights, while the ε-stabilizer becomes a stop option with payoff log(ε).

The authors are explicit that these are representation theorems, not mechanistic inevitabilities. The architecture does not by itself determine one game, policy class, or discount, and other games may give other insights. That caveat is what keeps the unification a framework for comparison rather than a claim that one calculation is the true one.

Explanation goals become game-theoretic knobs

Once the backward pass is a game, desired explanation properties can be written as game-theoretic concepts and translated back into concrete changes to the backward rules. The paper names localisation focus, robustness to input noise, and stable attention routing as examples, mapping them to policy regularization, risk aversion, and extended action sets respectively.

The analytical modes make this concrete. A local temperature τ on the linear-state Gibbs readout sharpens routing when τ < 1 and diffuses it when τ > 1, at no measured cost beyond small timing shifts (VGG-16 0.035 to 0.037 s per image, ResNet-50 0.089 to 0.094 s per image). Activation-side risk aversion λ uses ADF variance propagation to attenuate routes through high-variance units, costing one ADF pass (ResNet-50 0.089 to 0.140 s per image). A softmax-shift risk aversion λsm downweights high-variance keys in attention routing, also one ADF pass (ViT-B/16 0.097 to 0.172 s per image). Equivariant spatial averaging over random rotations, translations, and flips multiplies cost by N (ViT-B/16, N = 50: 4.93 s per image, or 8.66 s with λsm).

Each mode recovers the anchor at its default (τ = 1, λ = 0, gate off, λent = 0), and none carries an exactness claim. The authors' suggested order of use is to start at the anchor, vary τ first because it is free, then add risk aversion when maps should discount routes through high-variance units, and use equivariant averaging on ViTs only when token-grid artifacts matter.

One adapted alpha-beta-LRP wins on ViT-B/16 localisation

The headline empirical result is a case study rather than a broad benchmark. On ViT-B/16, one selected adaptation of alpha-beta-LRP outperforms prior transformer-specific backward methods across all considered localisation metrics. The same case study also covers VGG-16 and ResNet-50.

Getting LRP to work on a ViT required specific operator-level conventions, collectively called the DAVE-style conditioned forward pass and augmented with QK gradient detachment. LayerNorm statistics are computed from a detached copy of the input, so the backward pass sees only a pointwise map and LRP conservation is preserved; GELU gates are conditioned the same way; softmax outputs are fully detached; and the QK-matmul backward hooks return zero gradients so no relevance flows through Q or K. All explanatory mass through the attention block therefore flows through the Value tensor.

The authors state plainly that these conventions implement Value-only attribution conditioned on the realized attention matrix, as in CP-LRP, and are not the rules of AttnLRP, which propagates relevance through the softmax and the QK product as well. The composite Value rule is the only attention-Value rule in the released code, and the sequential variant was not used for any reported number.

Trajectory distance separates calculations that maps hide

Because the trajectory distribution is the object of study, the paper can define a distance between two attribution calculations rather than between two attribution maps. The Hellinger distance between trajectory distributions separates trained from randomized calculations under cascading weight randomization, even when their attribution maps look alike.

The authors position this as a complement to the parameter-insensitivity sanity check: it gives a calculation-level readout, and it detects randomization collapse even when projected maps remain similar. The distance is computable in one backward pass. First results are described as promising: H separates intra- from inter-class image pairs for at least one game on every architecture and increases monotonically with Gaussian input noise. The authors also note that its empirical utility will require calculation-level benchmarks, so this is a diagnostic proposal with initial evidence, not an established standard.

Exactness stops at ReLU and conditioned attention

The scope conditions are specific and worth keeping attached to the results. Exact results cover ReLU networks and attention conditioned on the realized attention pattern, LayerNorm statistics, and GELU gates. The analytical modes are deformations around these exact anchors and carry no exactness claim of their own.

The evaluation has its own boundary. Human-mask agreement is only one evaluation axis, and sharper routing can remove network-relevant evidence outside the mask. Within this scope, one adapted ViT-B/16 alpha-beta-LRP configuration outperforms all baselines, including prior transformer-specific backward methods, in attribution localization, while the LRP rules remain more faithful. Whether this transfers to other modalities, and whether metric gains translate to downstream tasks, remains future work.

The equivariant averaging mode is also explicitly optional: the transforms do not form a group, so the average is a smoothing mode without an exactness claim.

Prefer the trajectory view when maps look alike

The paper's most useful practical distinction is not that one rule beats another on a leaderboard. It is that two attribution calculations can produce similar-looking maps while differing in the distribution of backward trajectories that generated them. When the question is whether an explanation is stable under weight randomization, the trajectory-level Hellinger distance is the paper's proposed instrument, and it is designed to catch collapse that map-level comparison misses.

When the question is localisation on a ViT, the paper's evidence points to a narrower priority: start from the alpha-beta-LRP anchor with the conditioned forward pass, vary the free temperature τ first to move between focused and diffuse routing, and add risk aversion only when routes through high-variance units should be discounted, accepting one ADF pass. Equivariant averaging is worth its N-fold cost only when token-grid artifacts are the problem being solved.

The framework's value is therefore conditional on which question is being asked. For comparing backward calculations at the level of their routing laws, the game formulation supplies exact anchors for gradients and the full alpha-beta-LRP family on ReLU networks. For claiming that a sharper map is a better explanation, the paper's own caveat applies: sharper routing can remove network-relevant evidence outside the human mask, and human-mask agreement is only one axis.

Sources used in this answer