io_uring: free the task context after a grace period

A task other than the owner can free a request, io_ring_exit_work()
running the local task_work of a DEFER_TASKRUN ring for example, or
io_do_iopoll() reaping completions. io_put_task() then takes the else
branch:

    percpu_counter_sub(&tctx->inflight, 1);
    if (unlikely(atomic_read(&tctx->in_cancel)))
            wake_up(&tctx->wait);
    put_task_struct(tctx->task);

The first line is what lets the owner leave io_uring_cancel_generic(),
and on exec it frees the tctx right after. Nothing holds that off for
the remaining three lines, the task reference only covers a tctx freed
along with the task_struct. It used to work out because
io_uring_del_tctx_node() took the uring_lock that the freeing side holds
across io_put_task(), until that moved to tctx_lock.

Free the tctx with kfree_rcu(), and enter an RCU read section in
io_put_task() before the count is dropped.

Fixes: 5623eb1ed0 ("io_uring/tctx: add separate lock for list of tctx's in ctx")
Signed-off-by: Chris Mason <[email protected]>
Signed-off-by: Jens Axboe <[email protected]>
This commit is contained in:
Chris Mason
2026-10-28 10:50:12 -06:00
committed by Jens Axboe
parent 8543a39f03
commit 9f5315d3f4
3 changed files with 6 additions and 1 deletions
+3
View File
@@ -161,6 +161,9 @@ struct io_uring_task {
/* drains ->task_list once the task can no longer run task_work */
struct work_struct fallback_work;
/* the tctx is freed after a grace period, see io_put_task() */
struct rcu_head rcu;
struct { /* task_work */
struct mpscq task_list;
struct callback_head task_work;
+2
View File
@@ -634,6 +634,8 @@ static inline void io_put_task(struct io_kiocb *req)
if (likely(tctx->task == current)) {
tctx->cached_refs++;
} else {
guard(rcu)();
percpu_counter_sub(&tctx->inflight, 1);
if (unlikely(atomic_read(&tctx->in_cancel)))
wake_up(&tctx->wait);
+1 -1
View File
@@ -64,7 +64,7 @@ void io_uring_free_tctx(struct task_struct *tsk)
WARN_ON_ONCE(tctx->cached_refs);
percpu_counter_destroy(&tctx->inflight);
kfree(tctx);
kfree_rcu(tctx, rcu);
tsk->io_uring = NULL;
}
}