Files
Chris Mason 360395e8db io_uring/sqpoll: don't wait for the thread to exit
io_sq_thread_stop() wakes the SQPOLL thread and waits for it to exit.
The thread belongs to the thread group that created the ring, so a
SIGSTOP or the cgroup freezer holds it in get_signal(), and the wake up
doesn't change that. io_ring_exit_work() then sleeps in that wait until
someone continues or kills the process. iou_exit has 64 slots, so 64
closed SQPOLL rings of a stopped process keep every other ring from
being freed, for everyone.

Don't wait. Set the stop bit, wake the thread, and let it exit when it
gets to run again. The sqd can now outlive its last ring, so give it a
second refcount with one reference for the rings and one for the thread,
and free it with the last one. The thread can't reach a freed ring,
rings leave ctx_list before dropping their reference, and the io-wq hash
is refcounted.

Fixes: 16efa4fce3 ("io_uring: allow IO worker threads to be frozen")
Signed-off-by: Chris Mason <[email protected]>
Signed-off-by: Jens Axboe <[email protected]>
2026-10-28 10:50:19 -06:00

40 lines
1.1 KiB
C

// SPDX-License-Identifier: GPL-2.0
struct io_sq_data {
refcount_t refs;
/* one for all the rings, one for the thread: the last put frees */
refcount_t free_refs;
atomic_t park_pending;
struct mutex lock;
/* ctx's that are using this sqd */
struct list_head ctx_list;
struct task_struct __rcu *thread;
struct wait_queue_head wait;
unsigned sq_thread_idle;
int sq_cpu;
pid_t task_pid;
pid_t task_tgid;
u64 work_time;
unsigned long state;
};
int io_sq_offload_create(struct io_ring_ctx *ctx, struct io_uring_params *p);
void io_sq_thread_finish(struct io_ring_ctx *ctx);
void io_sq_thread_stop(struct io_sq_data *sqd);
void io_sq_thread_park(struct io_sq_data *sqd);
void io_sq_thread_unpark(struct io_sq_data *sqd);
void io_put_sq_data(struct io_sq_data *sqd);
void io_sqpoll_wait_sq(struct io_ring_ctx *ctx);
int io_sqpoll_wq_cpu_affinity(struct io_ring_ctx *ctx, cpumask_var_t mask);
u64 io_sq_cpu_usec(struct task_struct *tsk);
static inline struct task_struct *sqpoll_task_locked(struct io_sq_data *sqd)
{
return rcu_dereference_protected(sqd->thread,
lockdep_is_held(&sqd->lock));
}