io_uring/poll: avoid retry wakeup loss when dropping ownership

If a wakeup sets IO_POLL_RETRY_FLAG between the owning task consuming
the existing instance of IO_POLL_RETRY_FLAG and the loop check
of io_poll_check_events(), then we stall on processing the request
until an unrelated wakeup comes along.

Using atomic_cmpxchg instead of atomic_sub_return to guarantee both the
count v and poll_refs are 0 and no wake-ups are missed.

Cc: [email protected]
Fixes: a26a35e901 ("io_uring: make poll refs more robust")
Signed-off-by: Juanlu Herrero <[email protected]>
Assisted-by: LLM
Suggested-by: Anand Natarajan <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Jens Axboe <[email protected]>
This commit is contained in:
Juanlu Herrero
2026-10-15 14:17:58 -06:00
committed by Jens Axboe
parent 98bc044d5a
commit 937d38dcac
+2 -1
View File
@@ -330,7 +330,8 @@ static int io_poll_check_events(struct io_kiocb *req, io_tw_token_t tw)
* Release all references, retry if someone tried to restart
* task_work while we were executing it.
*/
} while (atomic_sub_return(v, &req->poll_refs) & IO_POLL_REF_MASK);
v &= IO_POLL_REF_MASK;
} while (atomic_cmpxchg(&req->poll_refs, v, 0) != v);
io_napi_add(req);
return IOU_POLL_NO_ACTION;