Which cognitive steps do users intend to delegate to LLM reasoning?

Nearly half of reasoning steps were judged AI-initiated

The paper's central measurement is not whether users liked the AI's output, but who initiated each intermediate step. Participants reviewed their own completed tasks step by step and judged whether they had asked for that step or the AI had taken it on by itself. Aggregated over 992 annotations, 48.6% of steps were judged AI-initiated.

That headline number splits into four delegation enactments. Intended Delegation — the user prompted the work and the AI handled it as wanted — covered 43.6% of steps. Serendipitous Delegation, where the AI acted unprompted but the contribution was wanted, covered 30.4%. Unintended Delegation covered 18.1%, and Regretted Delegation, where the user had prompted the work but later wanted it handled differently, covered 7.8%.

The distinction matters because a single prompt can fan out into many steps. Delegating a task through a prompt did not mean the user had asked for every reasoning step required to complete it, so 'I delegated the task' and 'I intended this step' are different claims measured at different levels.

AI initiative rose from 31.1% to 61.4% within a turn

AI initiation was not evenly spread through a response. Participants judged 31.1% of steps in the first fifth of a turn as AI-initiated, compared with 61.4% in the final fifth.

Part of that rise tracked a shift in what kind of work was being done: reasoning moved away from Framing toward Developing, Reviewing, and Reprocessing, which were more often attributed to AI initiative. But composition alone predicted an increase only from 38.0% to 52.6%, below the observed 61.4%. The paper therefore attributes the increase to two things at once — changing cognitive work and greater AI initiative within the same kinds of work later in the turn.

Unintended Delegation also became more common later in a turn, but an AI-initiated step was not significantly more likely to be unwanted later. The paper reads this as more steps being attributed to AI initiative rather than as later AI initiative being worse.

Unwanted and minor steps were more often AI-initiated

AI initiative was not automatically a problem. Of the AI-initiated steps, 62.7% contributed as users wanted, which the paper classifies as Serendipitous Delegation; 37.3% did not.

The asymmetry appears when steps are split by how users judged them. 70.0% of unwanted steps were AI-initiated, compared with 41.1% of wanted steps, and 75.8% of minor steps were AI-initiated, compared with 44.4% of critical steps. Both differences were significant at p < .001.

These are shares within different denominators — unwanted versus wanted steps, minor versus critical steps — so they describe where AI initiative concentrates, not how often AI initiative is unwanted. The 62.7% figure answers the second question and the 70.0% figure answers the first.

Some kinds of cognitive work invite unwanted initiative

Delegation enactment varied by the type of cognitive work. Understanding and Planning were most often explicitly prompted and completed as wanted, with 72% and 54% classified as Intended Delegation. AI-initiated work was often welcomed in Synthesizing, Brainstorming, and Analyzing. Unintended Delegation was most common in Repeating (44%), Narrowing (38%), Generating (28%), and Gathering (25%).

The paper shows these differences are not just a restatement of how often the AI took initiative. Analyzing and Gathering had nearly identical AI-initiation rates (51% and 50%), yet 26% of AI-initiated Analyzing steps were unwanted, compared with 51% of Gathering steps. Generating produced the largest volume of unintended cases: 61 steps, or 34% of all Unintended Delegation.

Cognitive work was also unevenly distributed overall. Generating was the most frequent code (21.7%), followed by Analyzing (14.2%), Planning (12.4%), and Understanding (12.1%). The prominence of Generating partly reflects how tool use appears in traces: each artifact-producing tool call, such as generating one slide of a pitch deck, counted as its own step, and 78% of the 167 tool-call steps were coded Generating.

Users wanted full delegation for 63.9% of steps

For most steps (63.9%), participants wanted the AI to proceed without consulting or notifying them. The remaining 358 steps (36.1%) called for user involvement. Among those, 63% called for consultation before the AI acted, 29% for adjusting how much the AI did (18% toward doing less and 11% toward doing more), and 8% for being informed afterward.

How delegation had actually unfolded strongly shaped the involvement users wanted (Cramér's V = .475, p < .001). For Overreaching Delegation — critical work the AI took on that did not contribute as wanted — 82% called for consultation before the AI acted. For Unnecessary Delegation, where the AI took on minor work the user did not want, 53% called for the AI to do less. For Regretted Delegation, where the user had prompted the work but later wanted it handled differently, 62% preferred consultation.

Successful initiative did not always mean users wanted full autonomy. Among Beneficial Autonomy steps, 28% still called for user involvement, and 23 of the 29 requests to be informed after the AI acted occurred in these cases. Users sometimes wanted visibility into AI-initiated work even when its contribution was what they wanted.

Desired protocols depend on the type of cognitive work

Preferred protocols also varied across cognitive work (Cramér's V = .268, p < .001). Generating drew the greatest demand for involvement (62%), primarily through consultation before acting (55%). Repeating drew a similar level of demand (56%), but participants instead wanted the AI to do less (44%). Narrowing fell between these patterns, with participants wanting either consultation (32%) or less AI involvement (21%). Understanding (15%), Planning (21%), and Recalling (26%) drew relatively little demand for involvement.

These differences were not simply reactions to unwanted outcomes. Even among steps whose contribution matched what participants wanted, the desire for involvement continued to vary by cognitive work: among satisfactory steps, only 6% of Recalling and Understanding steps called for involvement, compared with 34% of Generating and 30% of Brainstorming steps.

The paper organizes these preferences into four desired delegation protocols ordered by how much autonomy the AI gets: Full Delegation (silent), Disclosed Delegation (post-action notification), Calibrated Delegation (in-action adjustment of how much the AI does), and Consultative Delegation (pre-action consultation). The protocols differ in whether and when the user is brought in — not at all, after the AI acts, as work unfolds, or before it proceeds.

Step-level annotations came from 24 users and three tasks

The evidence base is 24 frequent LLM users, each completing one of three predefined tasks representing Creation, Information, or Advice — the major categories of LLM-assisted knowledge work identified in prior work. Participants used a custom interface for task completion and retrospective review, producing 992 step-level annotations.

Taxonomies were developed on 567 data points and then applied to 425 held-out points from the 12 participants not used during development, with four participants from each task. Two LLMs from different providers annotated the held-out data using the finalized taxonomy definitions. For Desired Delegation Protocol, the two annotators agreed on 94.1% (Cohen's κ = .876), with Fleiss' κ = .853 across all annotators in a 60-instance validation sample. For Cognitive Work, agreement was 72.7% (Cohen's κ = .686), with Fleiss' κ = .656. Disagreements were adjudicated by two authors through discussion.

All significance tests accounted for within-participant dependence using permutation tests based on the χ² statistic. For step-level tests, labels were shuffled within participants; for position tests controlling for cognitive-work composition, labels were shuffled within participant-by-category cells; for participant-level tests, participants were permuted across conditions.

Where these distributions stop transferring

The findings come from 24 frequent LLM users and three predefined tasks, one per category, so differences between Creation, Information, and Advice may also reflect differences between those specific tasks. The tasks were short and carried fewer consequences and organizational constraints than many forms of knowledge work, and participants used one model, one interface, and a limited set of tools.

Trace format directly shaped the reported distributions. Because the PowerPoint tool represented separate artifact-producing calls as separate steps, the share of Generating in the Creation task was inflated — a reminder that model behavior, system prompts, trace format, and available tools all change which steps appear.

Participants evaluated each step retrospectively after seeing the output, which helped them connect intermediate work to its effects but may also have introduced hindsight and outcome bias. Reviewing every step after a task also differs from being interrupted mid-task, and stated preferences for consultation, adjustment, or disclosure may change when those mechanisms impose real costs. The design sketches in the paper are proposals, not evaluated interfaces.

The taxonomies also simplify work that can serve more than one function: each step received one primary cognitive-work code, and agreement for Cognitive Work was lower than for Desired Delegation Protocol. The paper calls for applying the taxonomies across populations, domains, tasks, models, and agent architectures before treating these distributions as general patterns.

Watch the work type, not just the step count

The most decision-relevant split in this paper is not how many steps the AI took on, but which kinds of work those steps were and how the user judged the outcome. Two work types with nearly identical AI-initiation rates — Analyzing at 51% and Gathering at 50% — produced very different rates of unwanted initiative (26% versus 51% of AI-initiated steps). A count of AI-initiated steps alone would have missed that difference entirely.

The same logic applies to the protocols users wanted. Generating and Repeating drew similar overall demand for involvement (62% and 56%), but the requested remedy differed: consultation before acting for Generating, doing less for Repeating. Treating 'wants more involvement' as one preference would collapse two distinct requests.

The paper's own priority follows from this: because 28% of Beneficial Autonomy steps still called for user involvement, and because unwanted and minor steps were disproportionately AI-initiated, the useful target is not blanket oversight of every step but inspectable, revisable AI-initiated decisions on the work types where users actually want a say. The paper's design sketches — labeling steps by cognitive-work type and attributed initiative, summarizing how many steps in a reply were AI-initiated, and letting users carry a protocol choice into later tasks — are aimed at exactly that target rather than at reviewing everything.

Sources used in this answer