From eab627dcd18a18a44a2cbee47df3058d3637ba0d Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Fri, 16 Oct 2026 18:22:10 +0300 Subject: [PATCH 01/26] NFSv4.1: retain MDS I/O until both thresholds are met When both size and I/O thresholds are configured, pnfs_within_mdsthreshold() uses size && io to decide whether to keep I/O on the MDS. Both variables indicate an unmet threshold, so reaching only one threshold returns false and may trigger pNFS layout acquisition while the other threshold remains unmet. Use logical OR instead of AND to stay with MDS I/O until both thresholds are met, as recommended by RFC 8881 Section 5.12.6. The new RFC contains the same threshold selection rule as RFC 5661 which is referenced in function comment. Fixes: d23d61c8d351 ("NFSv4.1 test the mdsthreshold hint parameters") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- fs/nfs/pnfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 6dc00233e837..14f4a4b12ffd 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -2086,7 +2086,7 @@ static bool pnfs_within_mdsthreshold(struct nfs_open_context *ctx, break; } if (size_set && io_set) { - if (size && io) + if (size || io) ret = true; } else if (size || io) ret = true; From 2ffcf17d26dabef5793aeadf75a23c595a72be08 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Sun, 18 Oct 2026 15:38:41 +0300 Subject: [PATCH 02/26] NFS: honor UTIME_NOW atime with explicit mtime When utimensat() requests UTIME_NOW for atime and an explicit mtime, the VFS sets ATTR_ATIME but not ATTR_ATIME_SET (as it should). The delegated explicit-mtime path calls nfs_set_timestamps_to_ts(), which skips atime, but nfs_setattr() then clears ATTR_ATIME, allowing the syscall to succeed without updating atime. The delegated explicit-mtime path already updates ctime locally. The VFS also supplies the requested atime value in ia_atime, even when ATTR_ATIME_SET is absent. Check ATTR_ATIME instead of ATTR_ATIME_SET to also apply the UTIME_NOW atime value supplied by the VFS. Fixes: aba41e90aade ("NFSv4.2: fix setattr caching of TIME_[MODIFY|ACCESS]_SET when timestamps are delegated") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- fs/nfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c index e0a2e92952af..23e8b62da080 100644 --- a/fs/nfs/inode.c +++ b/fs/nfs/inode.c @@ -650,7 +650,7 @@ static void nfs_set_timestamps_to_ts(struct inode *inode, struct iattr *attr) inode_maybe_inc_iversion(inode, updated); cache_flags |= NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME; } - if (attr->ia_valid & ATTR_ATIME_SET) { + if (attr->ia_valid & ATTR_ATIME) { inode_set_atime_to_ts(inode, attr->ia_atime); cache_flags |= NFS_INO_INVALID_ATIME; } From a38d7cc8c90f0d093550487c2905b51db0be1522 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Sun, 18 Oct 2026 17:15:28 +0300 Subject: [PATCH 03/26] NFS: update ctime for atime-only UTIME_NOW requests When an NFS client holds a timestamp delegation, utimensat() with atime set to UTIME_NOW and mtime set to UTIME_OMIT reaches nfs_setattr() with ATTR_TIMES_SET but without ATTR_ATIME_SET. There may be other paths that can reach here as well. The delegated-atime fast path treats this as an implicit atime update, clears ATTR_ATIME and returns success without updating ctime. Add check for ATTR_TIMES_SET along with ATTR_ATIME_SET (OR) so explicit timestamp requests use the normal SETATTR path while incidental atime updates can keep using the local shortcut. Fixes: e12912d94137 ("NFSv4: Add support for delegated atime and mtime attributes") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- fs/nfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c index 23e8b62da080..9839087cf373 100644 --- a/fs/nfs/inode.c +++ b/fs/nfs/inode.c @@ -746,7 +746,7 @@ nfs_setattr(const struct mnt_idmap *idmap, struct dentry *dentry, } else if (nfs_have_delegated_atime(inode) && attr->ia_valid & ATTR_ATIME && !(attr->ia_valid & ATTR_MTIME)) { - if (!(attr->ia_valid & ATTR_ATIME_SET)) { + if (!(attr->ia_valid & (ATTR_ATIME_SET | ATTR_TIMES_SET))) { nfs_update_delegated_atime(inode); attr->ia_valid &= ~ATTR_ATIME; } From b9d518902ff1969f93d2b4787951d98e50ab0181 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Sun, 18 Oct 2026 21:30:42 +0300 Subject: [PATCH 04/26] NFS: Fix inverted cross-directory rename check nfs_rename_is_unsafe_cross_dir() returns false for cross-directory renames, causing nfs_rename() to skip nfs_sync_inode() even when filehandles may expire on rename. Same-directory renames may instead trigger unnecessary synchronization. Current condition directly contradicts the function name/comments. Return false for same-directory renames so cross-directory renames reach the existing filehandle-volatility check. Keep the conservative fallback unchanged.. Fixes: dcd21b609d4ab ("NFS: Avoid flushing data while holding directory locks in nfs_rename()") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- fs/nfs/dir.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 08a940e63152..91c40c2fdac5 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -2735,7 +2735,7 @@ static bool nfs_rename_is_unsafe_cross_dir(struct dentry *old_dentry, { struct nfs_server *server = NFS_SB(old_dentry->d_sb); - if (old_dentry->d_parent != new_dentry->d_parent) + if (old_dentry->d_parent == new_dentry->d_parent) return false; if (server->fh_expire_type & NFS_FH_RENAME_UNSAFE) return !(server->fh_expire_type & NFS_FH_NOEXPIRE_WITH_OPEN); From 89b91409b2342e7629b1271eba315a0f06495247 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Mon, 19 Oct 2026 15:26:30 +0300 Subject: [PATCH 05/26] NFS: restore directory index when resuming readdir nfs_readdir() restores the cached folio index and cookie when resuming directory iteration, but leaves current_index at zero. In 32-bit index mode, nfs_readdir_search_for_cookie() then reports a folio-relative offset instead of the absolute entry position. Restore current_index from the saved logical folio index before searching the cache. This solution relies on the current behavior of nfs_readdir_seek_next_array(), where folio_index advances by one and current_index advances by nfs_readdir_array_maxentries() for each full, nonterminal folio. Fixes: f648022faa68e ("NFS: Convert readdir page cache to use a cookie based index") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- fs/nfs/dir.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 91c40c2fdac5..49437037f948 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -1253,6 +1253,8 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) spin_lock(&file->f_lock); desc->dir_cookie = dir_ctx->dir_cookie; desc->folio_index = dir_ctx->page_index; + desc->current_index = (loff_t)desc->folio_index * + nfs_readdir_array_maxentries(); desc->last_cookie = dir_ctx->last_cookie; desc->attr_gencount = dir_ctx->attr_gencount; desc->eof = dir_ctx->eof; From 702d55edc017632a0c39c1d4bdd579bdbadfd730 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Tue, 20 Oct 2026 10:14:55 +0300 Subject: [PATCH 06/26] SUNRPC: wake poll waiters when an empty pipe is unlinked When an empty rpc_pipefs pipe is unlinked, rpc_purge_list() returns without waking the pipe wait queue. An existing epoll waiter can remain blocked even though rpc_pipe_poll() reports EPOLLHUP and EPOLLERR for the disconnected pipe. Wake the wait queue after clearing RPC_I(inode)->pipe so existing poll waiters are notified regardless of whether messages were purged. Fixes: 9842ef3557ab ("SUNRPC: rpc_timeout_upcall_queue should not sleep") Signed-off-by: Avi Weiss Reviewed-by: Jeff Layton Reviewed-by: NeilBrown Signed-off-by: Anna Schumaker --- net/sunrpc/rpc_pipe.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/sunrpc/rpc_pipe.c b/net/sunrpc/rpc_pipe.c index 9d349cfbc483..68b4e040a77d 100644 --- a/net/sunrpc/rpc_pipe.c +++ b/net/sunrpc/rpc_pipe.c @@ -191,6 +191,7 @@ rpc_close_pipes(struct dentry *dentry) cancel_delayed_work_sync(&pipe->queue_timeout); rpc_inode_setowner(inode, NULL); RPC_I(inode)->pipe = NULL; + wake_up(&RPC_I(inode)->waitq); inode_unlock(inode); } From 0cd6e563938a5b2f88ce0008eda32f9617c9d81f Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Mon, 19 Oct 2026 13:11:17 +0300 Subject: [PATCH 07/26] sunrpc: instantiate rpc_pipefs dentry before publishing it rpc_mkpipe_dentry() publishes pipe->dentry before d_make_persistent() attaches its inode. A concurrent rpc_queue_upcall() or rpc_timeout_upcall_queue() can conceivably dereference the dentry while d_inode() is NULL. Attach the inode before publishing the dentry, restoring the ordering used before the rpc_pipefs conversion refactor. Fixes: 946e2256775e ("convert rpc_pipefs") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- net/sunrpc/rpc_pipe.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/sunrpc/rpc_pipe.c b/net/sunrpc/rpc_pipe.c index 68b4e040a77d..f14cae295280 100644 --- a/net/sunrpc/rpc_pipe.c +++ b/net/sunrpc/rpc_pipe.c @@ -666,8 +666,8 @@ int rpc_mkpipe_dentry(struct dentry *parent, const char *name, rpci->private = private; rpci->pipe = pipe; rpc_inode_setowner(inode, private); - pipe->dentry = dentry; // borrowed d_make_persistent(dentry, inode); + pipe->dentry = dentry; // borrowed fsnotify_create(dir, dentry); simple_done_creating(dentry); return 0; From 30150b94c70eff4bf916d7d367ecf75de25d5337 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Thu, 22 Oct 2026 19:55:14 +0300 Subject: [PATCH 08/26] SUNRPC: preserve fatal network errors during RPC binding call_bind_status() initializes status to -EIO. When an unbound transport's port lookup fails with -ENETDOWN or -ENETUNREACH and RPC_TASK_NETUNREACH_FATAL is set, the function exits the switch without updating status. The RPC task then reports -EIO instead of the network error. This can occur during NFSv3 port discovery with nconnect, when an additional transport has not yet bound. Preserve task->tk_status in the fatal branch. Leave the nonfatal retry handling unchanged. Fixes: 9827144bfb2b ("NFS: Treat ENETUNREACH errors as fatal in containers") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- net/sunrpc/clnt.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 6cedc824cf82..73f8469767d3 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2125,8 +2125,10 @@ call_bind_status(struct rpc_task *task) goto retry_timeout; case -ENETDOWN: case -ENETUNREACH: - if (task->tk_flags & RPC_TASK_NETUNREACH_FATAL) + if (task->tk_flags & RPC_TASK_NETUNREACH_FATAL) { + status = task->tk_status; break; + } fallthrough; case -ECONNREFUSED: /* connection problems */ case -ECONNRESET: From 5b0ac1f82201c2ab2d9c31ed568cc782bac6a399 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Thu, 22 Oct 2026 21:43:16 +0300 Subject: [PATCH 09/26] SUNRPC: preserve timeout policy when cloning clients rpc_clone_client() and rpc_clone_client_set_auth() leave the timeout argument unset, causing rpc_new_client() to use the transport defaults instead of the parent's configured timeout. This also affects pNFS data-server clients and changes their request timeout and initial RTT calculations. Use the parent's cl_timeout when no timeout is specified. Preserve explicitly supplied timeout overrides. Fixes: 1b63a75180c6 ("SUNRPC: Refactor rpc_clone_client()") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- net/sunrpc/clnt.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 73f8469767d3..3d9d7c930f35 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -653,6 +653,8 @@ static struct rpc_clnt *__rpc_clone_client(struct rpc_create_args *args, } args->servername = xprt->servername; args->nodename = clnt->cl_nodename; + if (!args->timeout) + args->timeout = clnt->cl_timeout; new = rpc_new_client(args, xps, xprt, clnt); if (IS_ERR(new)) From 7783534ce5fee57adf1b9025745c9af2e1ce2ccd Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Thu, 22 Oct 2026 21:43:17 +0300 Subject: [PATCH 10/26] SUNRPC: preserve transport security policy when cloning clients RPC client cloning does not copy the parent's transport security policy into the new client. Clones consequently lose their TLS policy and certificate/key serials, even though the shared transport remains protected. When NFSv4 migration replaces a cloned client's TLS transport, the missing security policy causes xs_setup_tcp_tls() to return -EACCES. Copy the parent's cl_xprtsec into the clone arguments before creating the new client. Fixes: 500053191297 ("SUNRPC: Plumb an API for setting transport layer security") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- net/sunrpc/clnt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 3d9d7c930f35..ba6fdd70414d 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -653,6 +653,7 @@ static struct rpc_clnt *__rpc_clone_client(struct rpc_create_args *args, } args->servername = xprt->servername; args->nodename = clnt->cl_nodename; + args->xprtsec = clnt->cl_xprtsec; if (!args->timeout) args->timeout = clnt->cl_timeout; From 0502c7e7f171a8c9c892bdef89e6aa6f44d7b2a3 Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Fri, 23 Oct 2026 11:48:19 +0300 Subject: [PATCH 11/26] SUNRPC: Fix TCP-with-TLS transport class identification xs_setup_tcp_tls() assigns the ordinary TCP transport class to a TCP-with-TLS transport. When rpc_clnt_add_xprt() is called with an unspecified transport type, it inherits the class identifier and can create an unencrypted TCP transport. Looks like a copy-paste error from the commit 3 years ago. NFSv4 trunk discovery uses an unspecified transport type when adding server-advertised addresses. If the additional transport passes server-identity validation, it can be admitted to the client's transport switch. An isolated loopback test reproduced plaintext application RPC transmission for TLS- and mTLS-configured clients, including an unpinned TLS-client RPC that selected the added transport. Assign the TCP-with-TLS class instead. With the fix, the plaintext-only test peer receives TLS negotiation probes but no application payloads. Fixes: 75eb6af7acdf ("SUNRPC: Add a TCP-with-TLS RPC transport class") Cc: stable@vger.kernel.org Signed-off-by: Avi Weiss Reviewed-by: Benjamin Coddington Signed-off-by: Anna Schumaker --- net/sunrpc/xprtsock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 027b37de694b..173e3b0fd7a3 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -3501,7 +3501,7 @@ static struct rpc_xprt *xs_setup_tcp_tls(struct xprt_create *args) transport = container_of(xprt, struct sock_xprt, xprt); xprt->prot = IPPROTO_TCP; - xprt->xprt_class = &xs_tcp_transport; + xprt->xprt_class = &xs_tcp_tls_transport; xprt->max_payload = RPC_MAX_FRAGMENT_SIZE; xprt->bind_timeout = XS_BIND_TO; From 8acd94f02a960c8936df25a7af9c02c8ea87064d Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Sun, 25 Oct 2026 14:59:24 +0200 Subject: [PATCH 12/26] SUNRPC: Balance switch queue length on transport removal When call_connect_status() moves a task off a transport marked for removal, it decrements xprt->queuelen but leaves the switch's xps_queuelen unchanged. The stale count persists after the task finishes and skews multipath transport selection. Decrement xps_queuelen alongside xprt->queuelen, matching the normal transport release path. Fixes: 6f081693e7b2 ("sunrpc: remove an offlined xprt using sysfs") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- net/sunrpc/clnt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index ba6fdd70414d..6fc36e516c1c 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2262,6 +2262,7 @@ call_connect_status(struct rpc_task *task) xprt_release(task); value = atomic_long_dec_return(&xprt->queuelen); + atomic_long_dec(&xps->xps_queuelen); if (value == 0) rpc_xprt_switch_remove_xprt(xps, saved, true); From f14b157168ac106cfac6d577e08726488b1b886c Mon Sep 17 00:00:00 2001 From: Avi Weiss Date: Tue, 27 Oct 2026 06:56:43 +0200 Subject: [PATCH 13/26] SUNRPC: Balance queue accounting when relocating a task When rpc_task_set_transport() relocates a movable task from an offline transport, it releases the request and drops the transport reference without decrementing the old transport's queuelen or the switch's xps_queuelen. Releasing the task later only accounts for the replacement transport. The stale counts affect transport selection and can prevent xprt_delete_locked() from removing an otherwise idle transport. Use rpc_task_release_xprt() to balance the previous assignment before selecting a replacement. Pass clnt explicitly because rpc_task_set_client() can assign a transport before tk_client is initialized. Fixes: 82ee41b85cef ("SUNRPC don't resend a task on an offlined transport") Signed-off-by: Avi Weiss Signed-off-by: Anna Schumaker --- net/sunrpc/clnt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 6fc36e516c1c..33e64c0225f3 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -1197,7 +1197,7 @@ void rpc_task_set_transport(struct rpc_task *task, struct rpc_clnt *clnt) (task->tk_flags & RPC_TASK_MOVEABLE))) return; xprt_release(task); - xprt_put(task->tk_xprt); + rpc_task_release_xprt(clnt, task->tk_xprt); } if (task->tk_flags & RPC_TASK_NO_ROUND_ROBIN) task->tk_xprt = rpc_task_get_first_xprt(clnt); From 000a87910213562ff09f5bc38cd366ce34fccf71 Mon Sep 17 00:00:00 2001 From: Mike Snitzer Date: Mon, 2 Nov 2026 16:52:18 -0500 Subject: [PATCH 14/26] NFS/localio: hold the client across an asynchronous LOCALIO probe nfs_local_probe_async() queues the client's own cl_local_probe_work without a reference on the client, and nfs_free_client() does not cancel it. The work takes a reference only once it runs. If the last nfs_put_client() comes first, for example at umount after an NFSv3 I/O queued a throttled probe, the client is freed with its work queued, and the work then runs on freed memory. Take the reference when the work is queued, and drop it when the work has run. If the work is already queued, it already holds a reference, and the one just taken is dropped. Fixes: 76d4cb6345da ("nfs: probe for LOCALIO when v4 client reconnects to server") Signed-off-by: Mike Snitzer Signed-off-by: Anna Schumaker --- fs/nfs/localio.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/fs/nfs/localio.c b/fs/nfs/localio.c index cdada6cbb13b..d5859e8c2a2a 100644 --- a/fs/nfs/localio.c +++ b/fs/nfs/localio.c @@ -203,15 +203,22 @@ void nfs_local_probe_async_work(struct work_struct *work) struct nfs_client *clp = container_of(work, struct nfs_client, cl_local_probe_work); - if (!refcount_inc_not_zero(&clp->cl_count)) - return; nfs_local_probe(clp); + /* The reference nfs_local_probe_async() took for this work */ nfs_put_client(clp); } +/* + * The work is part of the client, so it holds a reference on the client + * from when it is queued until it has run: the last nfs_put_client() + * cannot free the client under a queued probe. + */ void nfs_local_probe_async(struct nfs_client *clp) { - queue_work(nfsiod_workqueue, &clp->cl_local_probe_work); + if (!refcount_inc_not_zero(&clp->cl_count)) + return; + if (!queue_work(nfsiod_workqueue, &clp->cl_local_probe_work)) + nfs_put_client(clp); /* already queued, with its reference */ } EXPORT_SYMBOL_GPL(nfs_local_probe_async); From b3497ac6e96effd5450b820de171e1137223d36c Mon Sep 17 00:00:00 2001 From: Mike Snitzer Date: Mon, 2 Nov 2026 16:52:19 -0500 Subject: [PATCH 15/26] NFS/localio: probe again for LOCALIO when localio_enabled is set to Y localio_enabled is checked on every I/O. Setting it to N sends the next I/O of every client over the network, and setting it back to Y resumes LOCALIO for a client that uses it. But a client that probed while localio_enabled was N was disabled by the probe, and nothing probes it again: NFSv3 probes again only when nfs3_localio_probe_throttle is set, and NFSv4 only after state recovery. Such a client stays on the network after localio_enabled is set to Y. Give localio_enabled a set handler that, on a change from N to Y, queues a probe for every ready client, in every network namespace, that does not use LOCALIO. On the module command line there are no clients yet, and nothing is probed. Tested with NFSv3 and NFSv4.2 on a loopback mount: a client mounted with localio_enabled=N sent its writes over the network, and after localio_enabled was set to Y it sent none. Signed-off-by: Mike Snitzer Signed-off-by: Anna Schumaker --- Documentation/filesystems/nfs/localio.rst | 3 ++ fs/nfs/localio.c | 51 ++++++++++++++++++++++- 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/Documentation/filesystems/nfs/localio.rst b/Documentation/filesystems/nfs/localio.rst index 79808b37d745..e8f44332cc80 100644 --- a/Documentation/filesystems/nfs/localio.rst +++ b/Documentation/filesystems/nfs/localio.rst @@ -312,6 +312,9 @@ Module Parameters /sys/module/nfs/parameters/localio_enabled (bool) controls if LOCALIO is enabled, defaults to Y. If client and server are local but 'localio_enabled' is set to N then LOCALIO will not be used. +It is checked on every I/O: setting it to N sends the next I/O of every +client over the network, and setting it back to Y resumes LOCALIO. A +client that probed while it was N is probed again when it is set to Y. /sys/module/nfs/parameters/localio_O_DIRECT_semantics (bool) controls if O_DIRECT extends down to the underlying filesystem, defaults diff --git a/fs/nfs/localio.c b/fs/nfs/localio.c index d5859e8c2a2a..80a496d6fe0d 100644 --- a/fs/nfs/localio.c +++ b/fs/nfs/localio.c @@ -25,6 +25,7 @@ #include #include "internal.h" +#include "netns.h" #include "pnfs.h" #include "nfstrace.h" @@ -55,7 +56,31 @@ struct nfs_local_fsync_ctx { }; static bool localio_enabled __read_mostly = true; -module_param(localio_enabled, bool, 0644); + +static void nfs_local_probe_all(void); + +/* + * localio_enabled is checked on every I/O, so turning it off stops LOCALIO + * at once and turning it back on resumes it for clients that use it. A + * client that probed while it was off was disabled, and nothing probes it + * again: turning it on probes every client that does not use LOCALIO. + */ +static int localio_enabled_set(const char *val, const struct kernel_param *kp) +{ + bool was = READ_ONCE(localio_enabled); + int ret; + + ret = param_set_bool(val, kp); + if (!ret && !was && READ_ONCE(localio_enabled)) + nfs_local_probe_all(); + return ret; +} + +static const struct kernel_param_ops localio_enabled_ops = { + .set = localio_enabled_set, + .get = param_get_bool, +}; +module_param_cb(localio_enabled, &localio_enabled_ops, &localio_enabled, 0644); static void nfs_local_do_read(struct nfs_local_kiocb *iocb, const struct rpc_call_ops *call_ops); @@ -222,6 +247,30 @@ void nfs_local_probe_async(struct nfs_client *clp) } EXPORT_SYMBOL_GPL(nfs_local_probe_async); +/* Probe every client, in every namespace, that does not use LOCALIO. */ +static void nfs_local_probe_all(void) +{ + struct nfs_client *clp; + struct net *net; + + /* Set on the module command line: there are no clients yet. */ + if (!nfs_net_id) + return; + + down_read(&net_rwsem); + for_each_net(net) { + struct nfs_net *nn = net_generic(net, nfs_net_id); + + spin_lock(&nn->nfs_client_lock); + list_for_each_entry(clp, &nn->nfs_client_list, cl_share_link) + if (clp->cl_cons_state == NFS_CS_READY && + !nfs_client_is_local(clp)) + nfs_local_probe_async(clp); + spin_unlock(&nn->nfs_client_lock); + } + up_read(&net_rwsem); +} + static inline void nfs_local_file_put(struct nfsd_file *localio) { /* nfs_to_nfsd_file_put_local() expects an __rcu pointer From 5e810a57a1cf2fc785d89e711ff40f1112f6404c Mon Sep 17 00:00:00 2001 From: Mike Snitzer Date: Mon, 2 Nov 2026 16:52:20 -0500 Subject: [PATCH 16/26] NFS/localio: give the LOCALIO probe a short timeout of its own The LOCALIO probe clones the NFS client with rpc_bind_new_program(), which NULL-pings the new program before the probe sends UUID_IS_LOCAL. Both calls use the mount's timeout. With timeo=600 and the default retrans, a server that does not answer the LOCALIO program holds each probe for about 170 s. The client then logs "nfslocalio: server ... not responding, timed out". On such a server, one mkdir took about 170 s of a 202 s run and ended when the probe gave up. Give the probe a soft timeout of its own, with no retries and no log message. The new nfs module parameter localio_timeout sets it in seconds, 10 by default and at most 600. Set it on a private clone of the NFS client before rpc_bind_new_program(), so that the ping uses it as well. Clear cl_noretranstimeo, which NFSv4 mounts set, because with it a sent call does not time out while the connection stays up. With localio_timeout set to 0, the probe uses the NFS client's timeouts, as before. The probe still shares the NFS client's transport and runs beside its RPCs. The NFS client's own timeouts do not change. Tested through a proxy that drops every call to the LOCALIO program: the probe gave up after 10.6 s and logged nothing. With localio_timeout=0 it took 182.9 s and logged the message above. The probe still succeeds against a server that supports LOCALIO: with no proxy, a 64 MiB O_DIRECT write through LOCALIO sent no WRITE RPCs. Signed-off-by: Mike Snitzer Signed-off-by: Anna Schumaker --- Documentation/filesystems/nfs/localio.rst | 9 ++++ fs/nfs/localio.c | 57 +++++++++++++++++++++-- 2 files changed, 63 insertions(+), 3 deletions(-) diff --git a/Documentation/filesystems/nfs/localio.rst b/Documentation/filesystems/nfs/localio.rst index e8f44332cc80..45c6981da0d8 100644 --- a/Documentation/filesystems/nfs/localio.rst +++ b/Documentation/filesystems/nfs/localio.rst @@ -316,6 +316,15 @@ It is checked on every I/O: setting it to N sends the next I/O of every client over the network, and setting it back to Y resumes LOCALIO. A client that probed while it was N is probed again when it is set to Y. +/sys/module/nfs/parameters/localio_timeout (uint) +sets how many seconds the LOCALIO probe waits for each reply from the +server, defaults to 10, at most 600. A server that does not support +LOCALIO may not answer the probe at all. If no reply comes in time, +the probe gives up without a log message. The client then does not use +LOCALIO. The mount's timeouts do not apply to the probe. Set to 0, the +probe uses the mount's timeouts instead, as an NFS RPC would. A change +applies to the next probe. + /sys/module/nfs/parameters/localio_O_DIRECT_semantics (bool) controls if O_DIRECT extends down to the underlying filesystem, defaults to N. Application IO must be logical blocksize aligned, otherwise diff --git a/fs/nfs/localio.c b/fs/nfs/localio.c index 80a496d6fe0d..7f8d794581a3 100644 --- a/fs/nfs/localio.c +++ b/fs/nfs/localio.c @@ -82,6 +82,25 @@ static const struct kernel_param_ops localio_enabled_ops = { }; module_param_cb(localio_enabled, &localio_enabled_ops, &localio_enabled, 0644); +/* + * Seconds the LOCALIO probe waits for each reply. With 0, the probe uses + * the NFS client's timeouts instead. + */ +static unsigned int localio_timeout __read_mostly = 10; + +static int localio_timeout_set(const char *val, const struct kernel_param *kp) +{ + return param_set_uint_minmax(val, kp, 0, 600); +} + +static const struct kernel_param_ops localio_timeout_ops = { + .set = localio_timeout_set, + .get = param_get_uint, +}; +module_param_cb(localio_timeout, &localio_timeout_ops, &localio_timeout, 0644); +MODULE_PARM_DESC(localio_timeout, + "LOCALIO probe timeout in seconds (default 10), 0 to use the mount's default"); + static void nfs_local_do_read(struct nfs_local_kiocb *iocb, const struct rpc_call_ops *call_ops); static void nfs_local_do_write(struct nfs_local_kiocb *iocb, @@ -156,13 +175,45 @@ const struct rpc_program nfslocalio_program = { /* * nfs_init_localioclient - Initialise an NFS localio client connection + * + * A server that does not support LOCALIO may not answer its RPCs at all. + * The probe shares the NFS client's transport and runs beside its RPCs. + * With localio_timeout set, it gets a quiet, soft timeout of that many + * seconds of its own, whatever timeouts the mount uses. With 0, it uses + * the NFS client's timeouts, as an NFS RPC would. */ static struct rpc_clnt *nfs_init_localioclient(struct nfs_client *clp) { - struct rpc_clnt *rpcclient_localio; + unsigned int timeout_secs = READ_ONCE(localio_timeout); + struct rpc_clnt *parent = clp->cl_rpcclient; + struct rpc_clnt *probe = NULL, *rpcclient_localio; - rpcclient_localio = rpc_bind_new_program(clp->cl_rpcclient, - &nfslocalio_program, 1); + /* + * rpc_bind_new_program() pings the new program with the timeout of + * the client it clones, so set the probe's timeout on a private + * clone first. The clone's cl_timeout points at the transport's + * timeout: give it its own copy rather than write through it. + */ + if (timeout_secs) { + probe = rpc_clone_client(parent); + if (IS_ERR(probe)) + return probe; + probe->cl_timeout_default = (struct rpc_timeout) { + .to_initval = timeout_secs * HZ, + .to_maxval = timeout_secs * HZ, + .to_retries = 0, + }; + probe->cl_timeout = &probe->cl_timeout_default; + probe->cl_softrtry = 1; + probe->cl_noretranstimeo = 0; + probe->cl_chatty = 0; + parent = probe; + } + + rpcclient_localio = rpc_bind_new_program(parent, &nfslocalio_program, 1); + /* The new client holds a reference to its parent, the probe clone */ + if (probe) + rpc_shutdown_client(probe); dprintk_rcu("%s: server (%s) %s NFS LOCALIO.\n", __func__, rpc_peeraddr2str(clp->cl_rpcclient, RPC_DISPLAY_ADDR), From 427d4cd8048c8e18e7d0c7d0637e7fbec9fba078 Mon Sep 17 00:00:00 2001 From: Jeff Layton Date: Mon, 9 Nov 2026 15:44:47 -0500 Subject: [PATCH 17/26] NFS: don't let delegated c/mtime stamps go backward With a timestamp delegation the client stamps c/mtime from its own coarse clock. That can land before the server's fine-grained ctime the client cached from the OPEN or SETATTR reply, e.g. on a WRITE right after an exclusive create. The server then either rejects the times (they don't pass dl_ctime) or can't apply the ctime, since inode_set_ctime_deleg() never moves ctime backward. Either way the c/mtime the client reported while holding the delegation changes after DELEGRETURN; git t1092 over NFSv4.2 sees a stat-dirty file. When a delegated stamp isn't after the cached ctime, use the cached ctime + 1ns instead. Compute the stamp before storing it, so lockless readers (getattr, CB_GETATTR) never see the trailing coarse time. Fixes: e12912d94137 ("NFSv4: Add support for delegated atime and mtime attributes") Signed-off-by: Jeff Layton Signed-off-by: Anna Schumaker --- fs/nfs/inode.c | 40 ++++++++++++++++++++++++++-------------- 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c index 9839087cf373..98d8a34fab9d 100644 --- a/fs/nfs/inode.c +++ b/fs/nfs/inode.c @@ -629,25 +629,33 @@ nfs_fattr_fixup_delegated(struct inode *inode, struct nfs_fattr *fattr) } } +/* + * With delegated timestamps, the client's coarse clock can trail the + * server's fine-grained ctime cached from an OPEN or SETATTR reply. The + * server won't move ctime backward when the times are returned, so a + * new stamp must land after the cached ctime. + */ +static struct timespec64 nfs_delegated_stamp(struct inode *inode) +{ + struct timespec64 ctime = inode_get_ctime(inode); + struct timespec64 now = current_time(inode); + + if (timespec64_compare(&now, &ctime) <= 0) { + now = ctime; + timespec64_add_ns(&now, 1); + } + return now; +} + static void nfs_set_timestamps_to_ts(struct inode *inode, struct iattr *attr) { unsigned int cache_flags = 0; if (attr->ia_valid & ATTR_MTIME_SET) { - struct timespec64 ctime = inode_get_ctime(inode); - struct timespec64 mtime = inode_get_mtime(inode); - struct timespec64 now; - bool updated = false; - - now = inode_set_ctime_current(inode); - if (!timespec64_equal(&now, &ctime)) - updated = true; - + /* The stamp is always after the old ctime */ + inode_set_ctime_to_ts(inode, nfs_delegated_stamp(inode)); inode_set_mtime_to_ts(inode, attr->ia_mtime); - if (!timespec64_equal(&now, &mtime)) - updated = true; - - inode_maybe_inc_iversion(inode, updated); + inode_maybe_inc_iversion(inode, true); cache_flags |= NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME; } if (attr->ia_valid & ATTR_ATIME) { @@ -665,7 +673,11 @@ static void nfs_update_atime(struct inode *inode) static void nfs_update_mtime(struct inode *inode) { - inode_update_time(inode, FS_UPD_CMTIME, 0); + struct timespec64 now = nfs_delegated_stamp(inode); + + /* Not inode_update_time(): lockless readers would see its stamp */ + inode_set_ctime_to_ts(inode, now); + inode_set_mtime_to_ts(inode, now); NFS_I(inode)->cache_validity &= ~(NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME); } From f13110f9cfb95a09d286eee561adc96c6a02664f Mon Sep 17 00:00:00 2001 From: Jeff Layton Date: Mon, 9 Nov 2026 15:44:48 -0500 Subject: [PATCH 18/26] NFS: stamp after the server's ctime when truncating under delegated times A size-changing SETATTR always has the server stamp c/mtime (__nfsd_setattr(), per RFC 5661 section 18.30.4). With delegated timestamps the client discards the reply's ctime and stamps its own in nfs_vmtruncate(), clamped only against its cached ctime. A coarse client stamp can still land before the server's, which then won't move ctime back when the holder returns its times, so ctime changes after DELEGRETURN. - Raise the cached ctime to the reply's before nfs_vmtruncate() stamps, so the delegated stamp lands after the server's. - Stamp before nfs_vmtruncate() drops i_lock for truncate_pagecache(), not after. A delegation return starting in that window made the stamp a no-op, and DELEGRETURN sent the pre-truncate mtime, which the server could apply over its own truncate stamp. Fixes: e12912d94137 ("NFSv4: Add support for delegated atime and mtime attributes") Signed-off-by: Jeff Layton Signed-off-by: Anna Schumaker --- fs/nfs/inode.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c index 98d8a34fab9d..270d8bb46110 100644 --- a/fs/nfs/inode.c +++ b/fs/nfs/inode.c @@ -647,6 +647,21 @@ static struct timespec64 nfs_delegated_stamp(struct inode *inode) return now; } +/* + * The server stamped c/mtime itself (e.g. for a size change). Raise the + * cached ctime to the server's so that the next delegated stamp lands + * after it. + */ +static void nfs_delegated_ctime_raise(struct inode *inode, + const struct nfs_fattr *fattr) +{ + struct timespec64 ctime = inode_get_ctime(inode); + + if ((fattr->valid & NFS_ATTR_FATTR_CTIME) && + timespec64_compare(&fattr->ctime, &ctime) > 0) + inode_set_ctime_to_ts(inode, fattr->ctime); +} + static void nfs_set_timestamps_to_ts(struct inode *inode, struct iattr *attr) { unsigned int cache_flags = 0; @@ -820,10 +835,14 @@ static int nfs_vmtruncate(struct inode * inode, loff_t offset) nfs_ooo_clear(NFS_I(inode)); } NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_SIZE; + /* + * Stamp in the same i_lock section that checked the delegation and + * raised ctime, before a delegation return can start. + */ + nfs_update_delegated_mtime_locked(inode); spin_unlock(&inode->i_lock); truncate_pagecache(inode, offset); - nfs_update_delegated_mtime_locked(inode); spin_lock(&inode->i_lock); out: return err; @@ -849,6 +868,8 @@ void nfs_setattr_update_inode(struct inode *inode, struct iattr *attr, if ((attr->ia_valid & ATTR_SIZE) != 0) { if (!nfs_have_delegated_mtime(inode)) nfs_set_cache_invalid(inode, NFS_INO_INVALID_MTIME); + else + nfs_delegated_ctime_raise(inode, fattr); nfs_set_cache_invalid(inode, NFS_INO_INVALID_BLOCKS); nfs_inc_stats(inode, NFSIOS_SETATTRTRUNC); nfs_vmtruncate(inode, attr->ia_size); From 23b9cf773211d536147aa53ade68a88d457838ac Mon Sep 17 00:00:00 2001 From: Jeff Layton Date: Mon, 9 Nov 2026 15:44:49 -0500 Subject: [PATCH 19/26] NFS: stamp after the server's ctime following a CLONE nfs42_copy_dest_done() stamps the client's own c/mtime when it holds delegated timestamps. A server that stamps c/mtime on CLONE (as nfsd did before "nfsd: mark the delegation written on COPY/CLONE instead of stamping c/mtime", and as btrfs did before "btrfs: honor FMODE_NOCMTIME when cloning") can end up with a ctime newer than the client's coarse stamp, which it won't move back when the holder returns its times. The stamp didn't survive anyway: nfs_post_op_update_inode() then marks ctime invalid, so nfs_fattr_fixup_delegated() keeps the reply's ctime and nfs_update_inode() writes it over the stamp. Against a server that no longer stamps on CLONE, the client's ctime goes back to its old value until DELEGRETURN. - Pass the CLONE reply's attributes to nfs42_copy_dest_done() and raise the cached ctime to the server's before stamping. COPY returns no attributes, so it passes NULL. - Stamp before dropping i_lock, so a concurrent refresh can't take the server's times while ctime and mtime are marked invalid. - Under delegated timestamps, apply the reply with nfs_refresh_inode(), which doesn't mark ctime invalid. Fixes: 6e7daa3dad29 ("NFSv4.2: fix CLONE/COPY attrs in presence of delegated attributes") Signed-off-by: Jeff Layton Signed-off-by: Anna Schumaker --- fs/nfs/delegation.h | 2 ++ fs/nfs/inode.c | 6 ++++-- fs/nfs/nfs42proc.c | 25 ++++++++++++++++++++----- 3 files changed, 26 insertions(+), 7 deletions(-) diff --git a/fs/nfs/delegation.h b/fs/nfs/delegation.h index fba4699952b8..cd6142939d34 100644 --- a/fs/nfs/delegation.h +++ b/fs/nfs/delegation.h @@ -95,6 +95,8 @@ void nfs4_inode_make_writeable(struct inode *inode); void nfs_update_delegated_atime(struct inode *inode); void nfs_update_delegated_mtime(struct inode *inode); void nfs_update_delegated_mtime_locked(struct inode *inode); +void nfs_delegated_ctime_raise(struct inode *inode, + const struct nfs_fattr *fattr); static inline int nfs_have_read_or_write_delegation(struct inode *inode) { diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c index 270d8bb46110..5a682f74d7d1 100644 --- a/fs/nfs/inode.c +++ b/fs/nfs/inode.c @@ -652,8 +652,8 @@ static struct timespec64 nfs_delegated_stamp(struct inode *inode) * cached ctime to the server's so that the next delegated stamp lands * after it. */ -static void nfs_delegated_ctime_raise(struct inode *inode, - const struct nfs_fattr *fattr) +void nfs_delegated_ctime_raise(struct inode *inode, + const struct nfs_fattr *fattr) { struct timespec64 ctime = inode_get_ctime(inode); @@ -661,6 +661,7 @@ static void nfs_delegated_ctime_raise(struct inode *inode, timespec64_compare(&fattr->ctime, &ctime) > 0) inode_set_ctime_to_ts(inode, fattr->ctime); } +EXPORT_SYMBOL_GPL(nfs_delegated_ctime_raise); static void nfs_set_timestamps_to_ts(struct inode *inode, struct iattr *attr) { @@ -711,6 +712,7 @@ void nfs_update_delegated_mtime_locked(struct inode *inode) nfs_have_directory_delegation(inode)) nfs_update_mtime(inode); } +EXPORT_SYMBOL_GPL(nfs_update_delegated_mtime_locked); void nfs_update_delegated_mtime(struct inode *inode) { diff --git a/fs/nfs/nfs42proc.c b/fs/nfs/nfs42proc.c index ab86246fc364..09b4a6ecad88 100644 --- a/fs/nfs/nfs42proc.c +++ b/fs/nfs/nfs42proc.c @@ -380,6 +380,7 @@ out: * @pos: destination offset * @len: copy length * @oldsize: length of the file prior to clone/copy + * @fattr: destination attributes from the reply, or NULL * * Punch a hole in the inode page cache, so that the NFS client will * know to retrieve new data. @@ -387,7 +388,7 @@ out: * invalid cached values for change attribute, ctime, mtime and space used. */ static void nfs42_copy_dest_done(struct file *file, loff_t pos, loff_t len, - loff_t oldsize) + loff_t oldsize, const struct nfs_fattr *fattr) { struct inode *inode = file_inode(file); struct address_space *mapping = file->f_mapping; @@ -405,8 +406,12 @@ static void nfs42_copy_dest_done(struct file *file, loff_t pos, loff_t len, NFS_INO_INVALID_CTIME | NFS_INO_INVALID_MTIME | NFS_INO_INVALID_BLOCKS); + /* The server may have stamped c/mtime; stamp after it */ + if (fattr && nfs_have_delegated_mtime(inode)) + nfs_delegated_ctime_raise(inode, fattr); + /* Before a refresh can see the times invalid and take the server's */ + nfs_update_delegated_mtime_locked(inode); spin_unlock(&inode->i_lock); - nfs_update_delegated_mtime(inode); } static ssize_t _nfs42_proc_copy(struct file *src, @@ -506,7 +511,8 @@ static ssize_t _nfs42_proc_copy(struct file *src, goto out; } - nfs42_copy_dest_done(dst, pos_dst, res->write_res.count, oldsize_dst); + nfs42_copy_dest_done(dst, pos_dst, res->write_res.count, oldsize_dst, + NULL); nfs_invalidate_atime(src_inode); status = res->write_res.count; out: @@ -1311,8 +1317,17 @@ static int _nfs42_proc_clone(struct rpc_message *msg, struct file *src_f, /* a zero-length count means clone to EOF in src */ if (count == 0 && res.dst_fattr->valid & NFS_ATTR_FATTR_SIZE) count = nfs_size_to_loff_t(res.dst_fattr->size) - dst_offset; - nfs42_copy_dest_done(dst_f, dst_offset, count, oldsize_dst); - status = nfs_post_op_update_inode(dst_inode, res.dst_fattr); + nfs42_copy_dest_done(dst_f, dst_offset, count, oldsize_dst, + res.dst_fattr); + /* + * nfs_post_op_update_inode() would mark ctime invalid, so the + * reply's ctime would overwrite the delegated stamp. + */ + if (nfs_have_delegated_mtime(dst_inode)) + status = nfs_refresh_inode(dst_inode, res.dst_fattr); + else + status = nfs_post_op_update_inode(dst_inode, + res.dst_fattr); } kfree(res.dst_fattr); From e8433473eca284bc561373d0a0fff632f76aa594 Mon Sep 17 00:00:00 2001 From: Olga Kornievskaia Date: Fri, 6 Nov 2026 22:12:31 -0500 Subject: [PATCH 20/26] NFSv4: wake blocking lock waiters when a delegated lock is released When the client holds a delegation, _nfs4_proc_setlk() caches POSIX locks locally and never sends a LOCK to the server. It has to strip FL_SLEEP to do so, since that branch runs holding sp->so_delegreturn_mutex and nfsi->rwsem and F_SETLKW can block indefinitely. A contending waiter is therefore never placed on the VFS blocked-lock list, and locks_wake_up_blocks() does not reach it when the holder releases. The waiter blocks in nfs4_retry_setlk() on clp->cl_lock_waitq instead, whose only waker is nfs4_callback_notify_lock(). The server never saw a LOCK for a cached lock, so it does not know anyone is waiting and will never send a CB_NOTIFY_LOCK; nfs4_proc_unlck() returns early without sending LOCKU for the same reason. The waiter sleeps out the full NFS4_LOCK_MAXTIMEOUT of 30s before retrying a lock that is long free. Wake cl_lock_waitq when releasing a cached lock. The releasing thread cannot tell which waiter wants the lock, so the wakeup names only the inode; make the owner comparison in nfs4_wake_lock_waiter() conditional on cbnl_valid so that a CB_NOTIFY_LOCK still wakes exactly the owner it names. The inode restriction matters: without it every blocked locker for the server is woken, and those on non-delegated state each retry with a LOCK RPC. Fixes: a1d617d8f134 ("nfs: allow blocking locks to be awoken by lock callbacks") Assisted-by: Claude Opus 5 Signed-off-by: Olga Kornievskaia Signed-off-by: Anna Schumaker --- fs/nfs/nfs4proc.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index e1018b03c1cb..50d36dc58fc6 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -7287,8 +7287,15 @@ static int nfs4_proc_unlck(struct nfs4_state *state, int cmd, struct file_lock * if (status != 0) goto out; /* Is this a delegated lock? */ - if (test_bit(NFS_LOCK_INITIALIZED, &lsp->ls_flags) == 0) + if (test_bit(NFS_LOCK_INITIALIZED, &lsp->ls_flags) == 0) { + struct nfs_client *clp = NFS_SERVER(inode)->nfs_client; + struct cb_notify_lock_args cbnl = { .cbnl_valid = false }; + + /* The server never saw it, so it will not notify waiters */ + nfs_copy_fh(&cbnl.cbnl_fh, NFS_FH(inode)); + __wake_up(&clp->cl_lock_waitq, TASK_NORMAL, 0, &cbnl); goto out; + } alloc_seqid = NFS_SERVER(inode)->nfs_client->cl_mvops->alloc_seqid; seqid = alloc_seqid(&lsp->ls_seqid, GFP_KERNEL); status = -ENOMEM; @@ -7708,8 +7715,14 @@ nfs4_wake_lock_waiter(wait_queue_entry_t *wait, unsigned int mode, int flags, vo struct nfs_lowner *lowner = &cbnl->cbnl_owner, *wowner = &waiter->owner; - /* Only wake if the callback was for the same owner. */ - if (lowner->id != wowner->id || lowner->s_dev != wowner->s_dev) + /* + * Only wake the owner a CB_NOTIFY_LOCK names. A lock cached + * under a delegation is released locally, and the releasing + * thread cannot tell which waiter wants it, so those wakeups + * (cbnl_valid false) name only the inode and wake them all. + */ + if (cbnl->cbnl_valid && + (lowner->id != wowner->id || lowner->s_dev != wowner->s_dev)) return 0; /* Make sure it's for the right inode */ From 4fdc29eb6730207b4a6f63f81575c7aa5c2a4ff1 Mon Sep 17 00:00:00 2001 From: Olga Kornievskaia Date: Fri, 6 Nov 2026 22:12:32 -0500 Subject: [PATCH 21/26] NFSv4: remember that the server may send CB_NOTIFY_LOCK Two processes contending for a POSIX lock on one file see occasional acquisitions of almost exactly 1, 3 or 7 seconds when the client holds a delegation, although the holder releases within microseconds. nfs4_retry_setlk() only parks a waiter on clp->cl_lock_waitq when NFS_STATE_MAY_NOTIFY_LOCK is set. Otherwise it uses nfs4_retry_setlk_simple(), which has no waitqueue at all and sleeps on a bare timeout starting at NFS4_LOCK_MINTIMEOUT and doubling, so a waiter that loses one poll waits a second, then two, then four. NFS_STATE_MAY_NOTIFY_LOCK is set in _nfs4_open_and_get_state() from the OPEN reply rflags. It describes the server, but is recorded per nfs4_state and can only be learned from a reply. Once a delegation is held, further opens are satisfied from it with no OPEN on the wire, so o_res.rflags is zero and the new state starts out believing no callback can arrive, even though the open that obtained the delegation was told otherwise. Record the capability on the nfs_client the first time an OPEN reply carries it, and apply it to every state built for that client. nfs_client is the right scope: the flag describes one server, CB_NOTIFY_LOCK arrives on that client's backchannel, and cl_lock_waitq is per nfs_client. This also covers a case the per-state flag misses. After the delegation is recalled the state has neither NFS_DELEGATED_STATE nor NFS_STATE_MAY_NOTIFY_LOCK, so locks go to the server and the server does send CB_NOTIFY_LOCK, but the client is in the backoff loop and ignores it. Fixes: a8ce377a5db8 ("nfs: track whether server sets MAY_NOTIFY_LOCK flag") Assisted-by: Claude Opus 5 Signed-off-by: Olga Kornievskaia Signed-off-by: Anna Schumaker --- fs/nfs/nfs4proc.c | 2 ++ include/linux/nfs_fs_sb.h | 1 + 2 files changed, 3 insertions(+) diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 50d36dc58fc6..2b01421edee7 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -3093,6 +3093,8 @@ static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata, if (server->caps & NFS_CAP_POSIX_LOCK) set_bit(NFS_STATE_POSIX_LOCKS, &state->flags); if (opendata->o_res.rflags & NFS4_OPEN_RESULT_MAY_NOTIFY_LOCK) + server->nfs_client->cl_may_notify_lock = true; + if (server->nfs_client->cl_may_notify_lock) set_bit(NFS_STATE_MAY_NOTIFY_LOCK, &state->flags); if (opendata->o_res.rflags & NFS4_OPEN_RESULT_PRESERVE_UNLINKED) set_bit(NFS_INO_PRESERVE_UNLINKED, &NFS_I(state->inode)->flags); diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h index 416c6f39f31d..6e1b0332f2de 100644 --- a/include/linux/nfs_fs_sb.h +++ b/include/linux/nfs_fs_sb.h @@ -120,6 +120,7 @@ struct nfs_client { #define NFS_SP4_MACH_CRED_COMMIT 6 /* COMMIT */ #define NFS_SP4_MACH_CRED_PNFS_CLEANUP 7 /* LAYOUTRETURN */ wait_queue_head_t cl_lock_waitq; + bool cl_may_notify_lock; #endif /* CONFIG_NFS_V4 */ /* Our own IP address, as a null-terminated string. From 1fa415883c37b02ff17acdd7977ed591d3d389ee Mon Sep 17 00:00:00 2001 From: Tim Menninger Date: Tue, 10 Nov 2026 18:27:27 +0000 Subject: [PATCH 22/26] xprtrdma: serialize unmap_sync with xprt_disconnect frwr_unmap_sync() posts LOCAL_INV Work Requests to the QP and waits for the matching CQE. This can race with rpcrdma_xprt_disconnect(): 1. frwr_unmap_sync() samples r_xprt->rx_ep 2. rpcrdma_xprt_disconnect() drains the QP 3. frwr_unmap_sync() posts LOCAL_INV WRs after the drain 4. No CQE arrives, so wait_for_completion() blocks indefinitely The hung task appears in the rpciod workqueue: Workqueue: rpciod rpc_async_schedule [sunrpc] wait_for_completion() frwr_unmap_sync() xprt_rdma_free() xprt_release() Serialize synchronous MR invalidation against QP teardown with rx_unmap_rwsem. frwr_unmap_sync() holds the read side while it builds and submits the LOCAL_INV chain. Disconnect holds the write side across QP drain and resource teardown. Drop the read side before waiting so disconnect can drain the submitted Work Requests and deliver their CQEs. Disconnect can reset a request while frwr_unmap_sync() waits for the read side. Recheck rl_registered after acquiring the rwsem and return without posting if the request has already been reset. Move the synchronous invalidation completion from the final MR to the request, whose lifetime covers xprt_rdma_free(). This allows disconnect to destroy MRs after draining the QP while frwr_unmap_sync() is still waiting. On a post failure, call rpcrdma_force_disconnect() before dropping the read side so the endpoint is not dereferenced after disconnect can run. frwr_unmap_async() runs inline from Receive completion processing. rpcrdma_xprt_drain() drains the Receive Queue before the Send Queue, so asynchronous LOCAL_INV Work Requests are submitted before Send Queue drain. frwr_unmap_sync(), however, runs from xprt_rdma_free() after the transport send lock has been released, so it is not ordered against disconnect. Tested on v7.4-rc1 under pNFS load with repeated data server restarts to provoke rapid QP disconnect/reconnect cycling. Without the fix, a hung task in frwr_unmap_sync() reproduced twice in six attempts. With the fix, no hang occurred across 200 cycles. Also observed on v6.8 and v6.18.34. Fixes: e28ce90083f0 ("xprtrdma: kmalloc rpcrdma_ep separate from rpcrdma_xprt") Cc: stable@vger.kernel.org Assisted-by: Claude:opus-4-7 Reviewed-by: Chuck Lever Signed-off-by: Tim Menninger Signed-off-by: Anna Schumaker --- net/sunrpc/xprtrdma/frwr_ops.c | 89 ++++++++++++++++++++------------- net/sunrpc/xprtrdma/transport.c | 2 + net/sunrpc/xprtrdma/verbs.c | 16 ++++-- net/sunrpc/xprtrdma/xprt_rdma.h | 10 +++- 4 files changed, 78 insertions(+), 39 deletions(-) diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c index e83cef19e656..97034a3b5141 100644 --- a/net/sunrpc/xprtrdma/frwr_ops.c +++ b/net/sunrpc/xprtrdma/frwr_ops.c @@ -27,13 +27,17 @@ /* Transport recovery * - * frwr_map and frwr_unmap_* cannot run at the same time the transport - * connect worker is running. The connect worker holds the transport - * send lock, just as ->send_request does. This prevents frwr_map and - * the connect worker from running concurrently. When a connection is - * closed, the Receive completion queue is drained before the allowing - * the connect worker to get control. This prevents frwr_unmap and the - * connect worker from running concurrently. + * frwr_map() is serialized with the connect worker by the transport + * send lock. + * + * frwr_unmap_async() runs from Receive completion processing. + * rpcrdma_xprt_drain() drains the Receive Queue before the Send Queue, + * so asynchronous LOCAL_INV Work Requests are submitted before Send + * Queue drain. + * + * frwr_unmap_sync() runs after the transport send lock is released. + * rx_unmap_rwsem orders its LOCAL_INV submission before disconnect + * drains the QP. * * When the underlying transport disconnects, MRs that are in flight * are flushed and are likely unusable. Thus all MRs are destroyed. @@ -139,7 +143,6 @@ int frwr_mr_init(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr *mr) mr->mr_ibmr = frmr; mr->mr_device = NULL; INIT_LIST_HEAD(&mr->mr_list); - init_completion(&mr->mr_linv_done); frwr_cid_init(ep, mr); sg_init_table(sg, depth); @@ -539,40 +542,60 @@ static void frwr_wc_localinv(struct ib_cq *cq, struct ib_wc *wc) * @cq: completion queue * @wc: WCE for a completed LocalInv WR * - * Awaken anyone waiting for an MR to finish being fenced. + * Wake frwr_unmap_sync() after the final LOCAL_INV completion. */ static void frwr_wc_localinv_wake(struct ib_cq *cq, struct ib_wc *wc) { struct ib_cqe *cqe = wc->wr_cqe; struct rpcrdma_mr *mr = container_of(cqe, struct rpcrdma_mr, mr_cqe); + struct rpcrdma_req *req = mr->mr_req; /* WARNING: Only wr_cqe and status are reliable at this point */ trace_xprtrdma_wc_li_wake(wc, &mr->mr_cid); frwr_mr_done(wc, mr); - complete(&mr->mr_linv_done); + complete(&req->rl_linv_done); rpcrdma_flush_disconnect(cq->cq_context, wc); } /** - * frwr_unmap_sync - invalidate memory regions that were registered for @req + * frwr_unmap_sync - synchronously invalidate MRs registered for @req * @r_xprt: controlling transport instance - * @req: rpcrdma_req with a non-empty list of MRs to process + * @req: request whose registered MRs are to be invalidated * - * Sleeps until it is safe for the host CPU to access the previously mapped - * memory regions. This guarantees that registered MRs are properly fenced - * from the server before the RPC consumer accesses the data in them. It - * also ensures proper Send flow control: waking the next RPC waits until - * this RPC has relinquished all its Send Queue entries. + * If @req still owns registered MRs after synchronizing with transport + * disconnect, post a chain of LOCAL_INV Work Requests and wait for the + * final completion. On successful completion, this fences the mapped + * regions from remote access and preserves Send Queue flow control. + * + * A concurrent disconnect can reset @req while this function waits for + * the read side. In that case the MRs have already been released and no + * LOCAL_INV Work Requests are posted. + * + * Context: Process context. Takes and releases + * @r_xprt->rx_unmap_rwsem for read. May sleep. */ void frwr_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req) { struct ib_send_wr *first, **prev, *last; - struct rpcrdma_ep *ep = r_xprt->rx_ep; const struct ib_send_wr *bad_wr; + struct rpcrdma_ep *ep; struct rpcrdma_mr *mr; int rc; + /* serialize against rpcrdma_xprt_disconnect() */ + down_read(&r_xprt->rx_unmap_rwsem); + + /* + * Disconnect can reset this request while we wait for the read side. + * Recheck rl_registered in case rx_ep now refers to a new connection. + */ + ep = r_xprt->rx_ep; + if (!ep || list_empty(&req->rl_registered)) { + up_read(&r_xprt->rx_unmap_rwsem); + return; + } + /* ORDER: Invalidate all of the MRs first * * Chain the LOCAL_INV Work Requests and post them with @@ -599,37 +622,35 @@ void frwr_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req) prev = &last->next; } while ((mr = rpcrdma_mr_pop(&req->rl_registered))); - mr = container_of(last, struct rpcrdma_mr, mr_invwr); - /* Strong send queue ordering guarantees that when the * last WR in the chain completes, all WRs in the chain * are complete. */ last->wr_cqe->done = frwr_wc_localinv_wake; - reinit_completion(&mr->mr_linv_done); + reinit_completion(&req->rl_linv_done); - /* Transport disconnect drains the receive CQ before it - * replaces the QP. The RPC reply handler won't call us - * unless re_id->qp is a valid pointer. + /* + * The read side prevents disconnect from draining this QP until the + * LOCAL_INV chain has been submitted. */ bad_wr = NULL; rc = ib_post_send(ep->re_id->qp, first, &bad_wr); + if (rc) { + /* On error, the MRs get destroyed once the QP has drained. */ + trace_xprtrdma_post_linv_err(req, rc); + + /* Force a connection loss to ensure complete recovery. + */ + rpcrdma_force_disconnect(ep); + } + up_read(&r_xprt->rx_unmap_rwsem); /* The final LOCAL_INV WR in the chain is supposed to * do the wake. If it was never posted, the wake will * not happen, so don't wait in that case. */ if (bad_wr != first) - wait_for_completion(&mr->mr_linv_done); - if (!rc) - return; - - /* On error, the MRs get destroyed once the QP has drained. */ - trace_xprtrdma_post_linv_err(req, rc); - - /* Force a connection loss to ensure complete recovery. - */ - rpcrdma_force_disconnect(ep); + wait_for_completion(&req->rl_linv_done); } /** diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index d4e6746d8ecd..e38637c2ec78 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -364,6 +364,8 @@ xprt_setup_rdma(struct xprt_create *args) INIT_DELAYED_WORK(&new_xprt->rx_connect_worker, xprt_rdma_connect_worker); + init_rwsem(&new_xprt->rx_unmap_rwsem); + xprt->max_payload = RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT; return xprt; diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 54957295a631..620afff79eff 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -572,20 +572,24 @@ out: * rpcrdma_xprt_disconnect - Disconnect underlying transport * @r_xprt: controlling transport instance * - * Caller serializes. Either the transport send lock is held, - * or we're being called to destroy the transport. + * Context: Caller serializes. Either the transport send lock is held, + * or the transport is being destroyed. Takes and releases + * @r_xprt->rx_unmap_rwsem for write. May sleep. * * On return, @r_xprt is completely divested of all hardware * resources and prepared for the next ->connect operation. */ void rpcrdma_xprt_disconnect(struct rpcrdma_xprt *r_xprt) { - struct rpcrdma_ep *ep = r_xprt->rx_ep; + struct rpcrdma_ep *ep; struct rdma_cm_id *id; int rc; + down_write(&r_xprt->rx_unmap_rwsem); + + ep = r_xprt->rx_ep; if (!ep) - return; + goto out_unlock; /* Prevent teardown-generated events from requesting another close. */ atomic_xchg(&ep->re_force_disconnect, 1); @@ -604,6 +608,9 @@ void rpcrdma_xprt_disconnect(struct rpcrdma_xprt *r_xprt) rdma_destroy_id(id); r_xprt->rx_ep = NULL; + +out_unlock: + up_write(&r_xprt->rx_unmap_rwsem); } /* Fixed-size circular FIFO queue. This implementation is wait-free and @@ -927,6 +934,7 @@ struct rpcrdma_req *rpcrdma_req_create(struct rpcrdma_xprt *r_xprt, INIT_LIST_HEAD(&req->rl_free_mrs); INIT_LIST_HEAD(&req->rl_registered); + init_completion(&req->rl_linv_done); spin_lock(&buffer->rb_lock); list_add(&req->rl_all, &buffer->rb_allreqs); spin_unlock(&buffer->rb_lock); diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 4cbc941e4a3e..b3016e1eb172 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -46,6 +46,7 @@ #include /* spinlock_t, etc */ #include /* atomic_t, etc */ #include /* struct kref */ +#include /* struct rw_semaphore */ #include /* struct work_struct */ #include @@ -249,7 +250,6 @@ struct rpcrdma_mr { int mr_nents; enum dma_data_direction mr_dir; struct ib_cqe mr_cqe; - struct completion mr_linv_done; union { struct ib_reg_wr mr_regwr; struct ib_send_wr mr_invwr; @@ -348,6 +348,7 @@ struct rpcrdma_req { struct list_head rl_free_mrs; struct list_head rl_registered; + struct completion rl_linv_done; }; static inline struct rpcrdma_req * @@ -449,6 +450,13 @@ struct rpcrdma_xprt { struct delayed_work rx_connect_worker; struct rpc_timeout rx_timeout; struct rpcrdma_stats rx_stats; + + /* + * Orders frwr_unmap_sync()'s LOCAL_INV submission ahead of the + * QP drain. Disconnect holds the write side across the drain + * and resource teardown. + */ + struct rw_semaphore rx_unmap_rwsem; }; #define rpcx_to_rdmax(x) container_of(x, struct rpcrdma_xprt, rx_xprt) From 9e28836b480c66cca4c3d391177f5b3a50927f7d Mon Sep 17 00:00:00 2001 From: Tim Menninger Date: Thu, 5 Nov 2026 00:01:31 +0000 Subject: [PATCH 23/26] pNFS: avoid repeatedly cancelling I/O for the same lseg Commit b739a5bd9d9f ("NFSv4/flexfiles: Cancel I/O if the layout is recalled or revoked") added I/O cancellation when a layout segment is invalidated while outstanding references remain. The invalid segment stays on the layout's segment list until those references drain. Subsequent scans can find it again and repeatedly invoke the layout driver's cancel_io() callback. For flexfiles, this cancels matching RPC tasks and disconnects data server RPC clients. During RPC/RDMA data server recovery, repeated cancellation can cause a reconnect loop that consumes essentially all client CPU and prevents forward progress. Record cancellation requests per mirror stripe in the flexfiles layout segment. Each stripe is cancelled at most once, regardless of how many times the invalid segment is encountered. A per-stripe bitmap rather than a single per-segment bit preserves the ability to cancel I/O to a different device after an earlier device-specific cancellation. Cancellation is best-effort. An RPC that has already passed the layout-validity check may not yet be visible to the cancellation scan. Such I/O retains its lseg reference and remains subject to the normal RPC completion and recovery paths. Suppress repeated cancellation and disconnect requests without changing the existing lseg lifetime and deferred layout-return accounting. This reproduced on 6.8.0, 7.0.0, and 7.4.0-rc1. It was tested on 7.4.0-rc1 with sustained pNFS/flexfiles I/O over RPC/RDMA while restarting the NFS service on a data server. Without it, repeated cancellation led to a reconnect loop and no workload recovery. With it, the client recovered after the data server returned. Fixes: b739a5bd9d9f ("NFSv4/flexfiles: Cancel I/O if the layout is recalled or revoked") Cc: stable@vger.kernel.org Signed-off-by: Tim Menninger Signed-off-by: Anna Schumaker --- fs/nfs/flexfilelayout/flexfilelayout.c | 12 ++++++++++++ fs/nfs/flexfilelayout/flexfilelayout.h | 1 + 2 files changed, 13 insertions(+) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 9492d6bc1d1a..4419ded8fe58 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -340,6 +341,7 @@ static void _ff_layout_free_lseg(struct nfs4_ff_layout_segment *fls) { if (fls) { ff_layout_free_mirror_array(fls); + bitmap_free(fls->cancelled_stripes); kfree(fls); } } @@ -653,6 +655,12 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh, fls->mirror_array[i]->report_interval = be32_to_cpup(p); out_sort_mirrors: + fls->cancelled_stripes = bitmap_zalloc( + fls->mirror_array_cnt * dss_count, gfp_flags); + if (!fls->cancelled_stripes) { + rc = -ENOMEM; + goto out_err_free; + } ff_layout_sort_mirrors(fls); ret = &fls->generic_hdr; dprintk("<-- %s (success)\n", __func__); @@ -2582,6 +2590,10 @@ static void ff_layout_cancel_io(struct pnfs_layout_segment *lseg, if (devid && memcmp(&mirror->dss[dss_id].devid, devid, sizeof(*devid)) != 0) continue; + /* Do not cancel a stripe again while this lseg drains. */ + if (test_and_set_bit(idx * mirror->dss_count + dss_id, + flseg->cancelled_stripes)) + continue; rcu_read_lock(); mirror_ds = rcu_dereference(mirror->dss[dss_id].mirror_ds); if (IS_ERR_OR_NULL(mirror_ds) || diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h index 09c3cd6964fd..5d5587fded76 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.h +++ b/fs/nfs/flexfilelayout/flexfilelayout.h @@ -109,6 +109,7 @@ struct nfs4_ff_layout_segment { u64 stripe_unit; u32 flags; u32 mirror_array_cnt; + unsigned long *cancelled_stripes; struct nfs4_ff_layout_mirror *mirror_array[] __counted_by(mirror_array_cnt); }; From 9d84719d34cb98a2f584e38679f95216c1a15d0e Mon Sep 17 00:00:00 2001 From: Tim Menninger Date: Fri, 6 Nov 2026 23:28:42 +0000 Subject: [PATCH 24/26] SUNRPC: add request-scoped disconnects when cancelling tasks rpc_cancel_tasks() allows callers to cancel a selected set of RPC tasks, but callers that also need to tear down the connection currently have to disconnect at the rpc_clnt level. That is broader than necessary for clients with multiple transports. A cancelled request is associated with a specific rpc_xprt and records the connection generation on which it was transmitted in rq_connect_cookie. Add rpc_cancel_tasks_and_disconnect() to cancel matching tasks and mark matching requests that remain on the transport transmit or receive queues for conditional transport disconnect when the request is released. Store the disconnect indication in struct rpc_rqst rather than struct rpc_task. An RPC task can release one request and later acquire another, so task-scoped state could otherwise be consumed by a successor request and disconnect the wrong transport generation. After cancelling matching tasks, walk the client's transports and mark matching requests belonging to that client that remain on the transmit or receive queues. An rpc_xprt may be shared by multiple rpc_clnt instances, so requests belonging to other clients are not marked. When a marked request is released, consume the marker under the existing transport_lock critical section. If the request was transmitted, call xprt_conditional_disconnect_locked() with its rq_connect_cookie. The disconnect is skipped if that cookie no longer matches the transport's current connection generation. This adds no lock acquisition to the normal request-release path. Requests that were never transmitted do not trigger this additional disconnect. Signed-off-by: Tim Menninger Signed-off-by: Anna Schumaker --- include/linux/sunrpc/sched.h | 4 ++ include/linux/sunrpc/xprt.h | 5 ++ net/sunrpc/clnt.c | 98 +++++++++++++++++++++++++------- net/sunrpc/sunrpc.h | 8 +++ net/sunrpc/xprt.c | 107 +++++++++++++++++++++++++++++++++-- 5 files changed, 197 insertions(+), 25 deletions(-) diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h index 0dbdf3722537..dbaad8b5f181 100644 --- a/include/linux/sunrpc/sched.h +++ b/include/linux/sunrpc/sched.h @@ -230,6 +230,10 @@ unsigned long rpc_cancel_tasks(struct rpc_clnt *clnt, int error, bool (*fnmatch)(const struct rpc_task *, const void *), const void *data); +unsigned long rpc_cancel_tasks_and_disconnect(struct rpc_clnt *clnt, int error, + bool (*fnmatch)(const struct rpc_task *, + const void *), + const void *data); void rpc_execute(struct rpc_task *); void rpc_init_priority_wait_queue(struct rpc_wait_queue *, const char *); void rpc_init_wait_queue(struct rpc_wait_queue *, const char *); diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 005730623f3b..780eaa157a9f 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -112,6 +112,11 @@ struct rpc_rqst { ktime_t rq_xtime; /* transmit time stamp */ int rq_ntrans; + bool rq_disconnect_on_release; + /* + * protected by rq_xprt->transport_lock + * while the request is active + */ #if defined(CONFIG_SUNRPC_BACKCHANNEL) struct lwq_node rq_bc_list; /* Callback service list */ diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 33e64c0225f3..2f076b4bfd54 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -904,6 +904,59 @@ void rpc_killall_tasks(struct rpc_clnt *clnt) } EXPORT_SYMBOL_GPL(rpc_killall_tasks); +struct rpc_cancel_disconnect_ctx { + bool (*fnmatch)(const struct rpc_task *task, const void *data); + const void *data; +}; + +static int rpc_mark_xprt_disconnect_on_release(struct rpc_clnt *clnt, + struct rpc_xprt *xprt, + void *arg) +{ + struct rpc_cancel_disconnect_ctx *ctx = arg; + + xprt_mark_matching_reqs_disconnect_on_release(xprt, clnt, + ctx->fnmatch, + ctx->data); + return 0; +} + +static unsigned long +__rpc_cancel_tasks(struct rpc_clnt *clnt, int error, + bool (*fnmatch)(const struct rpc_task *, + const void *), + const void *data, + bool disconnect) +{ + struct rpc_task *task; + unsigned long count = 0; + + if (list_empty(&clnt->cl_tasks)) + return 0; + + spin_lock(&clnt->cl_lock); + list_for_each_entry(task, &clnt->cl_tasks, tk_task) { + if (!RPC_IS_ACTIVATED(task)) + continue; + if (!fnmatch(task, data)) + continue; + rpc_task_try_cancel(task, error); + count++; + } + spin_unlock(&clnt->cl_lock); + + if (disconnect && count) { + struct rpc_cancel_disconnect_ctx ctx = { + .fnmatch = fnmatch, + .data = data, + }; + rpc_clnt_iterate_for_each_xprt(clnt, + rpc_mark_xprt_disconnect_on_release, + &ctx); + } + return count; +} + /** * rpc_cancel_tasks - try to cancel a set of RPC tasks * @clnt: Pointer to RPC client @@ -919,28 +972,35 @@ unsigned long rpc_cancel_tasks(struct rpc_clnt *clnt, int error, const void *), const void *data) { - struct rpc_task *task; - unsigned long count = 0; - - if (list_empty(&clnt->cl_tasks)) - return 0; - /* - * Spin lock all_tasks to prevent changes... - */ - spin_lock(&clnt->cl_lock); - list_for_each_entry(task, &clnt->cl_tasks, tk_task) { - if (!RPC_IS_ACTIVATED(task)) - continue; - if (!fnmatch(task, data)) - continue; - rpc_task_try_cancel(task, error); - count++; - } - spin_unlock(&clnt->cl_lock); - return count; + return __rpc_cancel_tasks(clnt, error, fnmatch, data, false); } EXPORT_SYMBOL_GPL(rpc_cancel_tasks); +/** + * rpc_cancel_tasks_and_disconnect - cancel matching RPC tasks + * @clnt: Pointer to RPC client + * @error: RPC task error value to set + * @fnmatch: Pointer to selector function + * @data: User data + * + * Like rpc_cancel_tasks(), but after cancelling matching tasks, marks + * matching in-flight requests so that xprt_release() conditionally + * disconnects the connection generation on which each request was sent. + * + * @fnmatch is called while holding spinlocks and must not sleep. + * + * Returns the number of active tasks matched by @fnmatch. + */ +unsigned long +rpc_cancel_tasks_and_disconnect(struct rpc_clnt *clnt, int error, + bool (*fnmatch)(const struct rpc_task *, + const void *), + const void *data) +{ + return __rpc_cancel_tasks(clnt, error, fnmatch, data, true); +} +EXPORT_SYMBOL_GPL(rpc_cancel_tasks_and_disconnect); + static int rpc_clnt_disconnect_xprt(struct rpc_clnt *clnt, struct rpc_xprt *xprt, void *dummy) { diff --git a/net/sunrpc/sunrpc.h b/net/sunrpc/sunrpc.h index e3c6e3b63f0b..11b9e5cead8e 100644 --- a/net/sunrpc/sunrpc.h +++ b/net/sunrpc/sunrpc.h @@ -38,9 +38,17 @@ static inline int sock_is_loopback(struct sock *sk) struct svc_serv; struct svc_rqst; +struct rpc_clnt; +struct rpc_task; +struct rpc_xprt; int rpc_clients_notifier_register(void); void rpc_clients_notifier_unregister(void); void auth_domain_cleanup(void); void svc_sock_update_bufs(struct svc_serv *serv); enum svc_auth_status svc_authenticate(struct svc_rqst *rqstp); +void xprt_mark_matching_reqs_disconnect_on_release(struct rpc_xprt *xprt, + const struct rpc_clnt *clnt, + bool (*fnmatch)(const struct rpc_task *, + const void *), + const void *data); #endif /* _NET_SUNRPC_SUNRPC_H */ diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 48a3618cbb29..c110806989c5 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -812,6 +812,17 @@ xprt_request_retransmit_after_disconnect(struct rpc_task *task) !xprt_connected(xprt); } +static void xprt_conditional_disconnect_locked(struct rpc_xprt *xprt, + unsigned int cookie) + __must_hold(&xprt->transport_lock) +{ + if (cookie != xprt->connect_cookie) + return; + if (test_bit(XPRT_CLOSING, &xprt->state)) + return; + xprt_schedule_autoclose_locked(xprt); +} + /** * xprt_conditional_disconnect - force a transport to disconnect * @xprt: transport to disconnect @@ -827,12 +838,7 @@ void xprt_conditional_disconnect(struct rpc_xprt *xprt, unsigned int cookie) { /* Don't race with the test_bit() in xprt_clear_locked() */ spin_lock(&xprt->transport_lock); - if (cookie != xprt->connect_cookie) - goto out; - if (test_bit(XPRT_CLOSING, &xprt->state)) - goto out; - xprt_schedule_autoclose_locked(xprt); -out: + xprt_conditional_disconnect_locked(xprt, cookie); spin_unlock(&xprt->transport_lock); } @@ -1434,6 +1440,79 @@ xprt_request_dequeue_transmit(struct rpc_task *task) spin_unlock(&xprt->queue_lock); } +static bool +xprt_request_matches(const struct rpc_rqst *req, + const struct rpc_clnt *clnt, + bool (*fnmatch)(const struct rpc_task *, const void *), + const void *data) +{ + const struct rpc_task *task = req->rq_task; + + return task && task->tk_client == clnt && fnmatch(task, data); +} + +static void +xprt_request_mark_disconnect_on_release(struct rpc_xprt *xprt, + struct rpc_rqst *req) + __must_hold(&xprt->queue_lock) +{ + spin_lock(&xprt->transport_lock); + req->rq_disconnect_on_release = true; + spin_unlock(&xprt->transport_lock); +} + +/** + * xprt_mark_matching_reqs_disconnect_on_release - mark matching in-flight + * requests for connection-generation + * teardown at release + * @xprt: transport whose queues to walk + * @clnt: RPC client whose requests are eligible + * @fnmatch: selector called with the request's owning rpc_task + * @data: caller data forwarded to @fnmatch + * + * @fnmatch must not sleep and must be safe to call while xprt->queue_lock + * is held. + * + * Walks the transmit queue (including per-owner rq_xmit2 chains) and the + * receive tree under xprt->queue_lock and marks each matching rpc_rqst so + * that xprt_release() will conditionally disconnect the transport against + * that request's rq_connect_cookie. Requests remain eligible for marking + * while they are present on a transport queue. The marker is set under + * xprt->transport_lock, nested inside xprt->queue_lock, and xprt_release() + * consumes it with xprt->transport_lock held after removing queue membership. + * + * The @clnt qualification preserves the selection domain of + * rpc_cancel_tasks(). An rpc_xprt may be shared by multiple rpc_clnt + * instances, so requests belonging to other clients must not be marked. + */ +void +xprt_mark_matching_reqs_disconnect_on_release(struct rpc_xprt *xprt, + const struct rpc_clnt *clnt, + bool (*fnmatch)(const struct rpc_task *, + const void *), + const void *data) +{ + struct rpc_rqst *req, *pos; + struct rb_node *n; + + spin_lock(&xprt->queue_lock); + list_for_each_entry(req, &xprt->xmit_queue, rq_xmit) { + if (xprt_request_matches(req, clnt, fnmatch, data)) + xprt_request_mark_disconnect_on_release(xprt, req); + list_for_each_entry(pos, &req->rq_xmit2, rq_xmit2) { + if (xprt_request_matches(pos, clnt, fnmatch, data)) + xprt_request_mark_disconnect_on_release(xprt, + pos); + } + } + for (n = rb_first(&xprt->recv_queue); n; n = rb_next(n)) { + req = rb_entry(n, struct rpc_rqst, rq_recv); + if (xprt_request_matches(req, clnt, fnmatch, data)) + xprt_request_mark_disconnect_on_release(xprt, req); + } + spin_unlock(&xprt->queue_lock); +} + /** * xprt_request_dequeue_xprt - remove a task from the transmit+receive queue * @task: pointer to rpc_task @@ -1915,6 +1994,7 @@ xprt_request_init(struct rpc_task *task) req->rq_rcv_buf.bvec = NULL; req->rq_release_snd_buf = NULL; req->rq_seqno_count = 0; + req->rq_disconnect_on_release = false; xprt_init_majortimeo(task, req, task->tk_client->cl_timeout); trace_xprt_reserve(req); @@ -1991,6 +2071,20 @@ void xprt_release(struct rpc_task *task) xprt = req->rq_xprt; xprt_request_dequeue_xprt(task); spin_lock(&xprt->transport_lock); + if (unlikely(req->rq_disconnect_on_release)) { + req->rq_disconnect_on_release = false; + /* + * Dequeue may skip queue_lock. Pair with the fully ordered + * test_and_clear_bit(RPC_TASK_NEED_XMIT) in + * xprt_request_dequeue_transmit_locked() to order the runstate + * check before reading the sender's rq_ntrans and + * rq_connect_cookie updates. + */ + smp_rmb(); + if (req->rq_ntrans > 0) + xprt_conditional_disconnect_locked(xprt, + req->rq_connect_cookie); + } xprt->ops->release_xprt(xprt, task); if (xprt->ops->release_request) xprt->ops->release_request(task); @@ -2020,6 +2114,7 @@ xprt_init_bc_request(struct rpc_rqst *req, struct rpc_task *task, task->tk_rqstp = req; req->rq_task = task; xprt_init_connect_cookie(req, req->rq_xprt); + req->rq_disconnect_on_release = false; /* * Set up the xdr_buf length. * This also indicates that the buffer is XDR encoded already. From 788c4f628565897e7090cdf5bb2e049e74aeca96 Mon Sep 17 00:00:00 2001 From: Tim Menninger Date: Fri, 6 Nov 2026 23:28:43 +0000 Subject: [PATCH 25/26] NFSv4/flexfiles: limit transport disconnects when cancelling I/O ff_layout_cancel_io() cancels RPC tasks associated with a recalled or revoked layout segment. If any matching tasks are found, it then calls rpc_clnt_disconnect(). rpc_clnt_disconnect() disconnects every transport attached to the RPC client. A data server client can have multiple transports, while the cancelled requests may have used only a subset of them. Cancelling I/O for one layout segment can therefore disrupt unrelated I/O using other transports belonging to the same data server client. This is particularly disruptive with RPC/RDMA during data server recovery, where unnecessary transport disconnects can result in repeated reconnect activity on otherwise unaffected connections. Use rpc_cancel_tasks_and_disconnect() instead. Matching tasks are still cancelled, but only matching requests belonging to that client that remain on the transport transmit or receive queues are marked for disconnect. When such a request is released, SUNRPC conditionally disconnects the connection generation on which that request was sent. This preserves cancellation and draining of I/O associated with the layout segment without disconnecting unrelated transports attached to the same RPC client. Fixes: b739a5bd9d9f ("NFSv4/flexfiles: Cancel I/O if the layout is recalled or revoked") Signed-off-by: Tim Menninger Signed-off-by: Anna Schumaker --- fs/nfs/flexfilelayout/flexfilelayout.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 4419ded8fe58..5c289737366f 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -2611,10 +2611,9 @@ static void ff_layout_cancel_io(struct pnfs_layout_segment *lseg, clnt = ds_clp->cl_rpcclient; if (!clnt) goto next; - if (!rpc_cancel_tasks(clnt, -ECANCELED, - ff_layout_match_io, lseg)) - goto next; - rpc_clnt_disconnect(clnt); + rpc_cancel_tasks_and_disconnect(clnt, -ECANCELED, + ff_layout_match_io, + lseg); next: nfs4_ff_layout_put_deviceid(mirror_ds); } From 151d95a1e696979a7d54b23c96338754be2fcc1f Mon Sep 17 00:00:00 2001 From: Tim Menninger Date: Fri, 30 Oct 2026 23:44:22 +0000 Subject: [PATCH 26/26] NFSv4: Prevent indefinite waits for out-of-order OPEN stateids When processing an out-of-order OPEN stateid on an NFSv4.1 or later mount, nfs_set_open_stateid_locked() waits for stateid updates to be processed in order. A five-second timeout is intended to prevent indefinite waits when preceding updates never arrive. However, each iteration calls schedule_timeout() with a fresh five-second timeout. An update to the shared open state can wake the caller without satisfying its ordering check. The remaining timeout is then discarded, and the next iteration starts another full wait. Under sustained concurrent OPEN/CLOSE activity on the same open state, repeated wakeups can prevent the timeout from ever expiring. Some tasks can remain stuck in update_open_stateid() while others continue making progress. Compute a single deadline on entry and pass only the remaining time to schedule_timeout(). Once the deadline expires, take the existing timeout path rather than starting another wait. This prevents wakeups from extending the wait indefinitely. Observed on Linux 6.8.0 and reproduced on 7.3.0 with 96 concurrent O_RDWR|O_TRUNC open/close loops on one NFSv4.1/TCP file (nconnect=16). Ten workers made no progress for over 300 seconds while others ran. With the fix, all workers resumed after transient stalls. Fixes: c9399f21c215 ("NFSv4: Fix OPEN / CLOSE race") Cc: stable@vger.kernel.org Signed-off-by: Tim Menninger Signed-off-by: Anna Schumaker --- fs/nfs/nfs4proc.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 2b01421edee7..6759fa61b9b4 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -1722,6 +1722,8 @@ static void nfs_set_open_stateid_locked(struct nfs4_state *state, { DEFINE_WAIT(wait); int status = 0; + unsigned long deadline = jiffies + 5 * HZ; + signed long remaining; for (;;) { if (nfs_stateid_is_sequential(state, stateid)) @@ -1756,7 +1758,8 @@ static void nfs_set_open_stateid_locked(struct nfs4_state *state, if (!fatal_signal_pending(current) && !nfs_current_task_exiting()) { - if (schedule_timeout(5*HZ) == 0) + remaining = (long)(deadline - jiffies); + if (remaining <= 0 || schedule_timeout(remaining) == 0) status = -EAGAIN; else status = 0;