mirror of
https://github.com/torvalds/linux.git
synced 2026-11-20 13:03:48 +00:00
master
54670
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c45d9aaa0b |
Merge tag 'dma-mapping-7.4-2026-11-18' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping fixes from Marek Szyprowski:
"Various fixes for the issues identified by sashiko.dev review system
in the dma-mapping helpers:
- added serialization to reserved memory region initialization
- fixed initialization of the shared memory for confidential
computing memory pools
- fixed error and exit code paths"
* tag 'dma-mapping-7.4-2026-11-18' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
dma-pool: decrypt pages before creating the shared remap
dma: swiotlb: Properly tear down the default pool on exit
dma: swiotlb: Round restricted DMA pool size down to IO_TLB_SEGSIZE
dma-pool: release CMA pages on atomic_pool_expand() error path
dma: swiotlb: Serialize restricted DMA pool initialization
dma-coherent: serialize reserved memory pool initialization
|
||
|
|
14866e632f |
dma-pool: decrypt pages before creating the shared remap
atomic_pool_expand() creates the pgprot_decrypted() remap of the pool
pages before calling set_memory_decrypted() on them. The error path
when gen_pool_add_virt() fails does the reverse: it re-encrypts the
pages and only then tears down the decrypted remap. In both cases there
is a window where a decrypted (shared) kernel mapping points at a page
that is still private, i.e. encrypted, to the guest.
dma_direct_alloc() gets this right: it converts the pages first, then
creates the remap, and on error removes the remap before re-encrypting.
Do the same in atomic_pool_expand(). The pages are now converted before
arch_dma_prep_coherent() and the remap, and the error path unwinds in
reverse order.
If set_memory_decrypted() fails, the encryption state of the pages is
unknown (the conversion may have partially succeeded), so they must not
be returned to the page allocator or CMA and are intentionally leaked,
as before. No remap exists at that point any more, so nothing else needs
to be undone. Add a comment explaining this.
Closes: https://sashiko.dev/#/bug/linux-9212d9b5-4a56-43e2-bc00-6e69b724078b
Fixes:
|
||
|
|
7d4a177d0d |
dma: swiotlb: Properly tear down the default pool on exit
With CONFIG_SWIOTLB_DYNAMIC enabled, the default memory pool is added to the io_tlb_default_mem.pools RCU list during initialization. When an IOMMU takes over DMA translation, swiotlb_exit() tears the default pool down and clears it with memset(), but leaves its node on the pools list. The list then contains a zeroed entry with a NULL next pointer, and any later walk of the list, for example mem_used() when reading the io_tlb_used debugfs file, dereferences an invalid pointer and crashes. swiotlb_exit() also leaves the io_tlb_default_mem fields that describe the pool. io_tlb_default_mem.nslabs and can_grow stay set, so is_swiotlb_allocated() and is_swiotlb_active() keep returning true, and swiotlb_find_slots() can schedule dynamic pool allocation after teardown. Without CONFIG_SWIOTLB_DYNAMIC, a later mapping can reach the zeroed defpool and hit BUG_ON(area_index >= pool->nareas). Before the pool data was split out of struct io_tlb_mem, swiotlb_exit() cleared the whole structure, including nslabs. Clear can_grow and cancel any pending dyn_alloc work, then remove the default pool from the list and subtract its slots from nslabs, and wait for an RCU grace period before clearing it, so that no reader can still see it. Without CONFIG_SWIOTLB_DYNAMIC, reset nslabs directly. As a result, with CONFIG_SWIOTLB_DYNAMIC swiotlb no longer silently keeps working through dynamic or transient pools after an IOMMU has turned it off, so for example dma-iommu bouncing for untrusted devices reports that bounce buffers are inactive, as it did before the dynamic pools support. Fixes: |
||
|
|
c3f98ff630 |
rcu/nocb: Avoid unsequenced accesses to jiffies in show_rcu_nocb_state()
After a recent improvement in clang to warn on unsequenced accesses to
volatile variables [1], an instance appears in show_rcu_nocb_state(),
which can break the build with CONFIG_WERROR=y / W=e:
In file included from kernel/rcu/tree.c:5123:
kernel/rcu/tree_nocb.h:1651:3: error: unsequenced accesses to volatile qualified 'jiffies' [-Werror,-Wunsequenced]
1650 | jiffies - rdp->nocb_bypass_first,
| ~~~~~~~
1651 | jiffies - rdp->nocb_nobypass_last,
| ^
Per C11 5.1.2.3.2, accessing a volatile object is a side effect. Per C11
6.5.2, unsequenced side effects on the same scalar object are undefined
behavior. Per C11 6.5.2.2.10/12, the evaluation of arguments in a
function call is indeterminately sequenced, meaning any side effects in
the arguments are unsequenced.
Add a local variable to hold the value of jiffies before the function
call so that there are no unsequenced side effects, clearing up the
warning.
Cc: [email protected]
Fixes:
|
||
|
|
38828d44f3 |
rcu-tasks: Use pr_debug() for callback contend/collapse messages
Commit
|
||
|
|
a9daf327f3 |
Merge tag 'sched_ext-for-7.4-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo: - With proxy execution or core scheduling, a CPU could run a task that had been requeued on another CPU while dispatch had the rq lock dropped. - Unregistering a scheduler could return while its teardown was still running, letting the teardown call into BPF code that was already freed. - A CPU coming online could call into the BPF scheduler before the scheduler was told that the CPU is online. - A parent scheduler could be asked to attach a sub-scheduler to a cgroup that was already removed. - Hardening: more kfuncs that take a queue lock are rejected in tracing programs, where they could deadlock the CPU, and a queue iterator no longer restarts after reaching the end, which the BPF verifier assumes can't happen. - Smaller fixes: tasks needlessly sent back to the BPF scheduler on idle CPUs, stale scheduling state carried into a task's next scheduler, and uneven CPU shards with partially populated SMT cores. * tag 'sched_ext-for-7.4-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: sched_ext: Reject DSQ iterator kfuncs in tracing programs sched_ext: make DSQ iterator stay drained after NULL sched_ext: Fix partial-SMT core counting in calc_shard_layout() sched_ext: Reset task fields on exit of an SCX_TASK_READY task sched_ext: Read exit_kind before flushing in scx_flush_disable_work() sched_ext: Reject scx_bpf_destroy_dsq() in tracing programs sched_ext: Fix ops.enqueue/dispatch() calls before ops.cpu_online() sched_ext: Factor out drain_offline_rq() sched_ext: Fix stale pick when the donor changes during dispatch sched_ext: Call ops.sub_attach() only while the cgroup is online sched_ext: Fix scx_bpf_dsq_reenq() kerneldoc claiming NMI calls are allowed sched_ext: Avoid unnecessary IMMED reenqueues on idle CPUs |
||
|
|
4b15991b3b |
Merge tag 'cgroup-for-7.4-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo: - A file notification racing the file's removal could re-arm the notification timer after it was shut down, corrupting the timer wheel or firing on freed memory. - The v1 freezer could leave an exiting task frozen forever after its cgroup was thawed or removed. - With memcg built without the block cgroup controller, memcg reclaim waited on writeback and could deadlock against a filesystem freeze. memcg now selects the controller. - cpuset partition state could go wrong when CPUs move between nested partitions or go offline: isolated CPUs leaked or went missing, invalid partitions kept exclusive CPUs or came back with no usable CPU, and expanding a remote partition could fail. Separately, a forked child could inherit a corrupted memory node mask. * tag 'cgroup-for-7.4-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: cgroup/cpuset: Don't revive invalid partition without active CPU on hotplug cgroup/cpuset: Remove redundant WARN_ON_ONCE() in remote_cpus_update() memcg: make MEMCG select BLK_CGROUP cpuset: generalize the validate_partition() interface cpuset: settle isolation state when CPUs are given back to parent cgroup/cpuset: Correctly set isolated_cpus when switching partition state cgroup/cpuset: Protect current->mems_allowed read in cpuset_fork() selftests/cgroup: Fix "test_cpuset_prs.sh -v" failures cpuset: add early empty cpumask check in partition_xcpus_add/del cgroup/cpuset: Reset partition data when an invalid partition stays invalid cgroup/freezer: Fix exiting tasks left frozen forever after rmdir cgroup/freezer: walk exiting tasks in the v1 freezer cgroup: arm the notify timer only while the file exists cgroup/cpuset: Do housekeeping check when isolated child with root parent may get more xcpus cgroup/cpuset: Recompute effective cpumask in update_cpumasks_hier() after calling update_parent_effective_cpumask() cgroup/cpuset: Fix incorrect CPU accounting with parent's effective_xcpus expansion |
||
|
|
acec9637e0 |
cgroup/cpuset: Don't revive invalid partition without active CPU on hotplug
When a CPU hotplug leaves no CPU outside of the partitions for the top
cpuset, cpuset_handle_hotplug() clears subpartitions_cpus and gives all
the active CPUs back to the top cpuset. cpuset_hotplug_update_tasks()
then walks the hierarchy and invalidates the partitions, as
subpartitions_cpus is empty.
However, an invalid partition visited in the same walk may be revived
if its effective_xcpus is not empty. tasks_nocpu_error() doesn't catch
it when its CPUs are all offline but it has no task. Its CPUs are put
back into subpartitions_cpus, and the valid partitions visited after it
are no longer invalidated, as subpartitions_cpus is not empty any more.
The top cpuset is left with CPUs overlapping with these partitions.
It can be reproduced on a system with 8 CPUs:
# cd /sys/fs/cgroup
# echo +cpuset > cgroup.subtree_control
# mkdir A1 B1
# echo 1 > A1/cpuset.cpus
# echo 1 > A1/cpuset.cpus.exclusive
# echo 0,2-7 > B1/cpuset.cpus
# echo root > B1/cpuset.cpus.partition
# echo root > A1/cpuset.cpus.partition
# cat A1/cpuset.cpus.partition
root invalid (Parent unable to distribute cpu downstream)
# echo 0 > /sys/devices/system/cpu/cpu1/online
# cat A1/cpuset.cpus.partition B1/cpuset.cpus.partition
root
root
# cat cpuset.cpus.effective
0,2-7
A1 can't take CPU 1, the last CPU of the root cgroup which has tasks.
When CPU 1 is offlined, A1 is revived without any active CPU, and B1
stays valid while its CPUs are also in the effective CPUs of the root
cgroup, which triggers
WARNING: kernel/cgroup/cpuset.c:919 at rebuild_sched_domains_locked
Only revive an invalid partition in hotplug if its effective_xcpus has
an active CPU, the same as what is required to create a partition with
tasks. With that, A1 and B1 are both invalidated in the above case.
Fixes:
|
||
|
|
42ef815ff6 |
cgroup/cpuset: Remove redundant WARN_ON_ONCE() in remote_cpus_update()
remote_cpus_update() warns if the CPUs to be added to a remote partition
intersect subpartitions_cpus, and then checks the very same condition
right below to set PERR_NOCPUS and invalidate the partition:
WARN_ON_ONCE(cpumask_intersects(tmp->addmask, subpartitions_cpus));
if (!capable(CAP_SYS_ADMIN))
WRITE_ONCE(cs->prs_err, PERR_ACCESS);
else if (cpumask_intersects(tmp->addmask, subpartitions_cpus) ||
cpumask_subset(top_cpuset.effective_cpus, tmp->addmask))
WRITE_ONCE(cs->prs_err, PERR_NOCPUS);
Sashiko reported that the warning could be triggered by a write to a
cpuset control file, before the CAP_SYS_ADMIN check, which would panic a
system with panic_on_warn set.
The warning was added by commit
|
||
|
|
0feaf5349e |
Merge tag 'timers-urgent-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fixes from Ingo Molnar:: - Add VDSO_UNALIGNED_PACKED config option for targets without efficient unaligned memory accesses (such as ARMv5), and turn it on for !CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS platforms. Propagate these changes into tooling headers as well (Ian Rogers) - Read getboottime64() offsets under the tk_core.seq sequence counter (Hungyu Lin) - Keep tsacct task begin times stable across suspend (Hungyu Lin) - Reject cross timestamps for invalid AUX clocks (Avi Weiss) - Refresh auxiliary timekeeper base before offset validation (Avi Weiss) * tag 'timers-urgent-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: timekeeping: Refresh auxiliary base before offset validation timekeeping: Reject cross timestamps for invalid AUX clocks tsacct: Keep task begin times stable across suspend timekeeping: Read getboottime64() offsets under the sequence counter tools headers: Update the vdso/unaligned.h copy with the kernel sources vdso: Add VDSO_UNALIGNED_PACKED config option |
||
|
|
fc28723fae |
Merge tag 'sched-urgent-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fixes from Ingo Molnar: - Fix spurious SYS_sched_setattr() -EPERM for migration-disabled tasks (Andrea Parri) - Reject clamp values above SCHED_CAPACITY_SCALE (Andrea Parri) - Fix cpuset_mutex double unlock in __sched_setscheduler() (Andrea Parri) - Don't use fits_capacity() to compare thread and CPU counts in in invalid_llc_nr() (Zhan Xusheng) - Do not touch SMT siblings that have not joined the core yet during CPU onlining (Seiji Nishikawa) * tag 'sched-urgent-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: sched/core: Do not touch SMT siblings that have not joined the core yet sched/cache: Don't use fits_capacity() to compare thread and CPU counts sched/core: Fix cpuset_mutex double unlock in __sched_setscheduler() sched/uclamp: Reject clamp values above SCHED_CAPACITY_SCALE sched/deadline: Fix spurious -EPERM for migration-disabled tasks |
||
|
|
ca6709295a |
Merge tag 'perf-urgent-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar: - Avoid AMD uncore context migration from non-owners (Sandipan Das) - Clear stop state in perf_trace_add(), to fix self-profilers such as async-profiler. Also fix a similar bug in hw_breakpoint_add() (Paolo Bertinetti) - Fix racy update of cpc->hrtimer_interval (Benjamin Segall) - Do not resurrect cpuctx->cgrp after last cgroup event exit, to fix UAF bug (Phat Dinh) * tag 'perf-urgent-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit perf: Fix racy update of cpc->hrtimer_interval perf/hw_breakpoint: Clear stop state in hw_breakpoint_add() tracing/perf: Clear stop state in perf_trace_add() perf/x86/amd/uncore: Avoid context migration from non-owners |
||
|
|
04da1a14b9 |
Merge tag 'fixes-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux
Pull liveupdate fixes from Mike Rapoport: - avoid address translation in is_kho_boot() before the linear map is initialized, fixing early boot BUG_ONs on arm64 - check for duplicate session names before growing the sessions preservations block set to avoid preserving an empty block that causes restoration to panic() * tag 'fixes-2026-11-15' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux: kho: check fdt_phys in is_kho_boot() liveupdate: check for duplicate session before growing block set |
||
|
|
e234ed1bb8 |
cpuset: generalize the validate_partition() interface
Refactor validate_partition() to take the parent partition, the new
partition root state, the new effective exclusive CPUs and the CPUs to
be deleted, instead of a trial cpuset. A NULL parent stands for a
remote partition, whose CPUs come from top_cpuset. This makes the
interface flexible enough to handle both local and remote partition
validation.
This also fixes a false PERR_NOCPUS error when expanding a remote
partition. The no CPU left for tasks check in partition_cpus_change()
is done against the cgroup parent, but the CPUs of a remote partition
come from top_cpuset, not from its parent. For example, on a 4-CPU
system:
echo 2-3 > P/cpuset.cpus
echo 2-3 > P/cpuset.cpus.exclusive
echo 2-3 > P/C/cpuset.cpus
echo 2 > P/C/cpuset.cpus.exclusive
echo root > P/C/cpuset.cpus.partition
(a task in P)
echo 2-3 > P/C/cpuset.cpus.exclusive
cat P/C/cpuset.cpus.partition
root invalid (Parent is not a partition root)
The effective_cpus of P is 3, a subset of the new exclusive CPUs 2-3 of
C, and P has a task, so the expansion is rejected. However, P is not a
partition root and does not give any CPU to C, and top_cpuset still has
CPUs 0-1 for the task. Enabling C directly with exclusive CPUs 2-3
succeeds, as remote_partition_enable() checks top_cpuset instead.
The no CPU left for tasks check now uses the given parent instead of
parent_cs(cs), and top_cpuset for a remote partition, as that is where
its CPUs are taken from.
The housekeeping conflict check is folded into validate_partition(). It
is skipped when the parent is not a valid partition root, as @cs can't
be a valid local partition in that case.
Use the new interface for local partition enablement in
update_parent_effective_cpumask() and in partition_cpus_change(),
replacing the open-coded checks there. The order of the checks is now
the same for both paths: empty CPUs, housekeeping conflict, then no CPU
left for tasks.
Add a test case to test_cpuset_prs.sh for the remote partition
expansion above.
Fixes:
|
||
|
|
c1ad011735 |
cpuset: settle isolation state when CPUs are given back to parent
partition_xcpus_del() only updates isolated_cpus when the partition that
gives back the CPUs has a different state than its parent. CPUs that were
further distributed to descendant isolated partitions are not taken into
account, so isolated_cpus_release() has to be called afterwards in
several places to walk up the ancestors and fix isolated_cpus.
This misses some cases and leaks isolated CPUs, as reported by Guopeng.
An isolated partition can be invalidated after its parent has already
become invalid, and its CPUs are left in isolated_cpus. For example, on a
32-CPU system with all CPUs online and without isolcpus= or nohz_full=
boot parameters:
cd /sys/fs/cgroup
echo +cpuset > cgroup.subtree_control
mkdir A
echo 0,2-31 > A/cpuset.cpus
echo root > A/cpuset.cpus.partition
echo +cpuset > A/cgroup.subtree_control
mkdir A/B
echo 2-3 > A/B/cpuset.cpus
echo isolated > A/B/cpuset.cpus.partition
cat cpuset.cpus.isolated
2-3
echo 0 > /sys/devices/system/cpu/cpu1/online
cat A/B/cpuset.cpus.partition
isolated invalid
cat cpuset.cpus.isolated
2-3
When A is invalidated, partition_xcpus_del() gives CPUs 0,2-31 back to
top cpuset. Both A and top cpuset are root partitions, so isolated_cpus
is left untouched although CPUs 2-3 were distributed to the isolated B.
B is then invalidated with an invalid parent, and nothing releases CPUs
2-3 either.
Make partition_xcpus_del() always set the isolation state of the given
back CPUs to the state of the parent that receives them. As the
effective_xcpus of a partition covers the CPUs distributed to its
descendants, the isolation state of those CPUs is settled at the same
time. Boot-time isolated CPUs are still preserved by isolated_cpu_update().
compute_partition_effective_cpumask() invalidates a child partition
without going through partition_xcpus_del(), so do the same there: the
CPUs of the invalidated child take the isolation state of its parent.
As the parent may be shrunk and its effective_xcpus not updated yet,
only do so for the CPUs the parent still owns. This is to keep the
isolation state correct for now, the in-place invalidation here will be
removed later in the partition rework.
isolated_cpus_update() assumes the old and new states are different,
which no longer holds now that partition_xcpus_del() proceeds regardless
of the states. Drop its @old_prs argument and the WARN_ON_ONCE() so that
it simply sets the isolation state of the given CPUs, and use it
everywhere. The state check in partition_xcpus_add() is dropped as well,
as setting the state is a no-op when the states are the same.
isolated_cpus_release() is no longer needed and is removed.
Fixes:
|
||
|
|
c15120853e |
tracing/uprobes: Fix memory leak of tail in __trace_uprobe_create()
When generating a default event name in __trace_uprobe_create(), tail is
allocated with kstrdup() without __free(kfree). If the subsequent
kmalloc() for buf fails, the function returns -ENOMEM immediately and
leaks tail.
Declare tail with __free(kfree) so it is automatically freed on all
exit paths.
Link: https://lore.kernel.org/all/179455856197.403392.16569431894958507645.stgit@devnote2/
Reported-by: Sashiko <[email protected]>
Closes: https://lore.kernel.org/all/[email protected]/
Fixes:
|
||
|
|
0941144ebe |
tracing/probes: Reject array count on non-dereference string sources
The runtime array loop (FETCH_OP_LP_ARRAY) for string types requires a
preceding memory dereference instruction so it can advance the pointer
address on each iteration. Non-dereference string sources such as $comm,
immediate strings, and eprobe event fields cannot be indexed as pointer
arrays, but finalize_fetch_insn() still accepts an array count on them
and emits an invalid FETCH_OP_LP_ARRAY sequence.
Reject string array counts when the preceding operation is not a memory
dereference.
Link: https://lore.kernel.org/all/179455855309.403392.4197028577158545202.stgit@devnote2/
Fixes:
|
||
|
|
2721573e6c |
tracing/probes: Fix user-space dereference string array store type
When finalizing a string array argument (e.g. +0u(%reg):string[N]),
finalize_fetch_insn() advances the instruction pointer to a new slot
for the store operation before checking whether the preceding operation
was FETCH_OP_UDEREF. As a result, user-space dereference string arrays
with type "string" emit FETCH_OP_ST_STRING (kernel memory read) instead
of FETCH_OP_ST_USTRING.
Check for FETCH_OP_UDEREF before advancing the instruction pointer.
Link: https://lore.kernel.org/all/179455854396.403392.18073771811100192405.stgit@devnote2/
Fixes:
|
||
|
|
92d9fa0ed6 |
tracing/eprobe: Disallow attaching eprobes to fprobe events
Event probes (eprobes) are not allowed to attach to other dynamic probe
events (kprobes, uprobes, and eprobes) to avoid recursive probe chains
and lifecycle dependencies. However, when fprobe events were introduced,
TRACE_EVENT_FL_FPROBE was omitted from the exclusion mask in
find_and_get_event(), allowing eprobes to attach to fprobe and tprobe
events.
Exclude TRACE_EVENT_FL_FPROBE events in find_and_get_event().
Link: https://lore.kernel.org/all/179455853457.403392.14549390017750529531.stgit@devnote2/
Fixes:
|
||
|
|
a0f836eea4 |
tracing/eprobe: Fix 1-byte signed field fetch in get_event_field()
Because the kernel is built with -funsigned-char, plain 'char' is
unsigned on all architectures. Reading 1-byte signed trace event fields
via *(char *)addr in get_event_field() therefore zero-extends instead of
sign-extending negative s8 values.
Use explicit fixed-width integer types (s8/u8, s16/u16, s32/u32) when
reading event fields.
Link: https://lore.kernel.org/all/179455852542.403392.15666310371702309464.stgit@devnote2/
Fixes:
|
||
|
|
c1e2c63e65 |
tracing/probes: Fix missing BTF kind checks in char pointer/array helpers
In struct btf_type, the type ID and byte size share a union, and only
BTF_KIND_INT types are followed by a 32-bit integer encoding word.
Because btf_type_is_char_ptr() and btf_type_is_char_array() do not
verify that the outer type is BTF_KIND_PTR or that the resolved element
type is a non-NULL BTF_KIND_INT before inspecting it, non-pointer BTF
types can have their byte size misinterpreted as a BTF type ID, and
arrays of non-integer types read out-of-bounds or misinterpret trailing
BTF metadata as BTF_INT_* encoding bits.
Validate the BTF type kinds and NULL pointers before reading inner type
metadata in both helpers.
Link: https://lore.kernel.org/all/179455851632.403392.258264246782960594.stgit@devnote2/
Fixes:
|
||
|
|
9b6e9d6a11 |
tracing/probes: Fix BTF bitfield parsing and out-of-bounds buffer write
BTF bitfield parsing in trace_probe.c assumes a 64-bit (8-byte) container
when computing FETCH_OP_MOD_BF shift amounts and basesize, whereas the
preceding store instruction only allocates and writes the field's fetch
type size (which may be 1, 2, or 4 bytes when kflag=1). At runtime,
fetch_apply_bitfield() performs an 8-byte read-modify-write on the
smaller ring buffer slot, corrupting adjacent ring buffer data or writing
past the end of the event record.
In addition, signed bitfields with bit widths other than 8/16/32/64 fail
to resolve a fetch type, bitfields whose width is a multiple of 8 inside
a larger integer or spanning a byte boundary fail to emit FETCH_OP_MOD_BF
or select a container wide enough for bitoffs + bitsize, kflag=0 bitfield
offsets >= 8 bits are not folded into the byte offset, and big-endian
architectures use the little-endian left-shift formula in
parse_btf_bitfield().
Compute the bitfield byte and bit offsets consistently during field
lookup, select the smallest unsigned fetch type that covers the bit
offset and bit width, and calculate the FETCH_OP_MOD_BF parameters from
the selected fetch type size for both big- and little-endian bitfields.
Link: https://lore.kernel.org/all/179455850711.403392.10512730891220678403.stgit@devnote2/
Fixes:
|
||
|
|
72d3a5cb0c |
tracing/probes: Enable newly appended probe when target event is enabled
Unlike kprobe events, fprobe and uprobe events only register their underlying probes when the trace event is enabled. When a new probe is appended to an existing multiprobe fprobe or uprobe event that is already enabled, the appended probe is linked into the event's probe list without being registered. As a result, the newly appended probe never fires until the event is disabled and re-enabled. Moreover, disabling an fprobe event calls unregister_fprobe() on the unregistered fprobe, and opening or closing a perf event on an enabled uprobe event dereferences a NULL tu->uprobe in uprobe_apply(). Register and enable the newly appended fprobe or uprobe immediately when the target event is already enabled. Link: https://lore.kernel.org/all/179455849798.403392.135678786198324285.stgit@devnote2/ Fixes: |
||
|
|
f1d9b7a3fe |
tracing/probes: Check enabled and busy state before unlinking sibling probes
When a multiprobe event is enabled or referenced (for example, by an eprobe, trigger, or open tracefs file) and a user attempts to delete the event or one of its probes via dynamic_events, checking trace_probe_has_sibling() before the enabled and busy checks allows the first probe to be unlinked and freed while the event is still active, only returning -EBUSY on the last sibling probe. For uprobe events, unlinking and freeing an enabled sibling probe without unregistering its uprobe_consumer also causes a use-after-free. Ensure all four probe types (kprobe, fprobe, uprobe, and eprobe) verify that the event is neither enabled nor busy before unlinking any sibling probe. Link: https://lore.kernel.org/all/179455848898.403392.17675270675769076596.stgit@devnote2/ Fixes: |
||
|
|
c55722233c |
kprobes: Re-optimize blocked sibling probe when cooling finishes
When an optimized kprobe is disabled (for example, to arm a new kprobe
in its jump-optimized instruction range), its jump is replaced with a
breakpoint (or disarmed) and the probe is placed on the cooling list
until the Tasks RCU grace period completes. While the disabled probe is
still on the cooling list, any sibling probe whose jump range covers
this probe's address cannot be optimized because can_optimize() sees the
not-yet-disarmed probe via get_kprobe().
In optprobe_finalize_generation(), when a disabled probe on cooling_list
finishes cooling and transitions to safely disarmed (list_del_init()),
check whether an optimized sibling probe covers op->kp.addr via
get_optimized_kprobe() and re-queue it for optimization, matching the
freeing_list path in do_free_cleaned_kprobes().
Link: https://lore.kernel.org/all/179440643954.328088.8524265330405527243.stgit@devnote2/
Fixes:
|
||
|
|
081f8fba97 |
kprobes: Do not overwrite jump instruction when re-arming unoptimizing probe
When an optimized probe is temporarily disarmed or disabled,
unoptimize_kprobe() places it on the unoptimizing list, but leaves
the relative jump in text until the optimizer thread runs.
__disarm_kprobe() explicitly skips arch_disarm_kprobe() when
kprobe_queued_unopt(p) is true.
However, if __arm_kprobe() is called before the unoptimization pass
executes, it unconditionally calls arch_arm_kprobe(p), writing a
breakpoint instruction over the first byte of the jump while leaving the
trailing bytes intact. optimize_kprobe(p) then dequeues the probe from
the unoptimizing queue, leaving the jump opcode overwritten with a
breakpoint in text while KPROBE_FLAG_OPTIMIZED remains set.
Skip calling arch_arm_kprobe() in __arm_kprobe() when
kprobe_queued_unopt(p) is true, keeping the existing relative jump intact
while unoptimization is cancelled. If optimize_kprobe() cannot optimize
the probe (e.g. due to a post_handler or disabled optimization), force
unoptimization immediately.
Link: https://lore.kernel.org/all/179440642973.328088.10282643330198602134.stgit@devnote2/
Fixes:
|
||
|
|
c8c1e40968 |
kprobes: Fix UAF and queue cooling probes on forced unoptimization
When unoptimize_kprobe() is called with force = true on an actively
optimized probe (whose op->list is empty), force_unoptimize_kprobe()
replaces the relative jump with a breakpoint, but leaves op->list empty
without queueing op to cur->cooling_list.
Because op->list remains empty, kprobe_disarmed() returns true
prematurely as soon as the probe is disabled. When unregister_kprobes()
runs, __unregister_kprobe_top() unhashes the probe immediately, and
unregister_kprobes() frees op and the optinsn slot after synchronize_rcu()
without waiting for Tasks RCU. Any task preempted inside the optinsn slot
then encounters a use-after-free when resuming execution.
Queue forcibly unoptimized probes onto cur->cooling_list and include
gen->cooling_list in optprobe_has_queued_probes() so that the generation
is dispatched and flushed properly. In do_unoptimize_kprobes(), collect
newly unoptimized probes into a separate local list so that probes
already placed on freeing_list are not processed a second time.
Link: https://lore.kernel.org/all/179440642050.328088.13914468408589377897.stgit@devnote2/
Fixes:
|
||
|
|
f60f98881b |
kprobes: Switch ftrace ops when adding probe with post_handler
When an ftrace-based kprobe is registered without a post_handler, it is
armed using kprobe_ftrace_ops. If a second probe with a post_handler is
subsequently registered at the same address, register_aggr_kprobe() sets
ap->post_handler to aggr_post_handler, but leaves the probe armed using
kprobe_ftrace_ops.
When the probe is later disarmed or disabled, disarm_kprobe_ftrace()
checks whether p->post_handler is non-NULL and attempts to disarm from
kprobe_ipmodify_ops. This fails with -ENOENT, decrements
kprobe_ipmodify_enabled below zero, leaves the probe armed on
kprobe_ftrace_ops, and aborts unregistration, leading to a use-after-free
if the probe module is unloaded.
Switch the armed probe from kprobe_ftrace_ops to kprobe_ipmodify_ops in
register_aggr_kprobe() outside of text_mutex and cpus_read_lock to avoid
lockdep and deadlock issues when modifying ftrace code. Defer adding the
new probe onto ap->list until the ftrace ops switch completes so that the
new probe's handlers are not published prematurely while ap is still on
kprobe_ftrace_ops, and mark ap disabled if rollback re-arming fails.
Also restore ftrace ops registration and counter if ftrace_set_filter_ip()
fails in __disarm_kprobe_ftrace().
Link: https://lore.kernel.org/all/179440641120.328088.15542754095040659094.stgit@devnote2/
Fixes:
|
||
|
|
2388a673e9 |
tracing: skip boot kprobe setup when no events were requested
setup_boot_kprobe_events() calls enable_boot_kprobe_events() even when
the kprobe_event= boot parameter was not supplied. That call takes
event_mutex without any command-line probes to enable.
Asynchronous eval-map updates also hold event_mutex. A normal boot can
therefore wait in init_kprobe_trace() for this background work despite
having no boot probes to set up.
Return early when the boot-event buffer is empty. Keep the mutex
protection in trace_event_update_all() and retain setup for explicit
kprobe_event= parameters. Bootconfig handles its own probe enablement,
and the existing trace_eval_sync() still waits for background work
before init memory is freed.
Link: https://lore.kernel.org/all/[email protected]/
Fixes:
|
||
|
|
700a332c3f |
sched_ext: Reject DSQ iterator kfuncs in tracing programs
bpf_iter_scx_dsq_next() and bpf_iter_scx_dsq_destroy() take the target
user DSQ's raw spinlock, but they are in scx_kfunc_ids_any, which is
also registered for BPF_PROG_TYPE_TRACING.
For an active root scheduler without ops.sub_attach, the compatibility
path in scx_prog_sched() lets an unassociated tracing program resolve to
that scheduler. The tracing program can then call the iterator kfuncs
while the target DSQ lock is already held.
For example, a scheduler which mixes FIFO and priority-queue insertions
into a user DSQ triggers scx_error() from scx_dispatch_enqueue() while
holding that DSQ's lock. scx_claim_exit() emits sched_ext_exit before
the lock is released:
scx_dispatch_enqueue()
raw_spin_lock(&dsq->lock)
scx_error()
scx_claim_exit()
trace_sched_ext_exit()
tracing BPF program
bpf_iter_scx_dsq_new(dsq_id)
bpf_for_each(scx_dsq, p, dsq_id, 0)
bpf_iter_scx_dsq_next(it)
raw_spin_lock_irqsave(&dsq->lock)
The tracing program recursively acquires the same raw spinlock and locks
up the CPU. Because the tracepoint runs before scx_finish_exit(),
disable_irq_work is not queued and the scheduler cannot enter its normal
disable path.
Reject the two iterator kfuncs from tracing programs at verification
time. The kfuncs remain available to SCX struct_ops and syscall
programs. bpf_iter_scx_dsq_destroy() has no independent live
reproduction: its cursor only links into the DSQ after at least one
successful next(), which deadlocks in the same window first. It is
included for symmetry.
This extends the coverage of commit
|
||
|
|
9d66f26fef |
Merge tag 'vfs-7.4-rc3.fixes.2' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull more vfs fixes from Christian Brauner:
- inode: Let the shrinker reclaim inodes with I_METADATA_WRITEBACK like
any other clean inode and have evict() write out what the bit tracks
unless the inode is being deleted.
Since v7.3 ext2, ext4 without a journal, fat, udf and minix set
I_METADATA_WRITEBACK when they copy an inode into its buffer and only
fsync() and friends clear it again. inode_lru_isolate() took such
inodes off the LRU expecting something to put them back once the bit
was gone but nothing ever did. So they couldn't be reclaimed until
something fsynced them or the filesystem was unmounted.
On ext4 without a journal an atime update was enough to pin an inode,
and creating 60000 small files in a memory cgroup limited to 96M got
the task OOM killed after about 43000 of them. Now all 60000 get
created.
- dcache:
- Give d_set_mounted() a seqlock of its own that d_invalidate()
waits on after unhashing the dentry.
Since v6.17 d_set_mounted() doesn't bump the seqcount of
rename_lock anymore, so a mount racing d_invalidate() could end
up below the unhashed directory. Nobody can reach it there and it
pins the subtree until the mount namespace goes away. With a FUSE
directory 3000 levels deep 151 of 200 bind mounts racing
FUSE_NOTIFY_INVAL_ENTRY ended up there.
- Only take busy secondary roots off ->s_roots in do_one_tree() so
that umount waits for a racing eviction of an idle one. Since
v7.3 it didn't, which ended in busy inodes after unmount and an
iput() on a dead superblock.
- Make a disconnected dentry a secondary root when the mount on it
goes away so that umount finds it and waits for its eviction.
With CAP_DAC_READ_SEARCH such a dentry can be mounted on via
open_by_handle_at() and move_mount() and umount racing its
eviction ended the same way.
- splice:
- Remove pipe buffer stealing as you and David suggested.
fuse and virtio_console were the only users of ->try_steal() and
now always copy, as they already did when a steal failed.
Stealing took page cache folios out of their files and turned
pages into page cache folios and back. SPLICE_F_GIFT and
SPLICE_F_MOVE are still accepted and fuse still advertises
FUSE_SPLICE_MOVE so servers that ask for it mount as before.
- Clamp a ->write_iter() result that is larger than the iterator in
iter_file_splice_write() instead of only warning about it.
Otherwise the pipe releases buffers past its head.
- ns:
-Allocate the namespace id before calling the namespace_init LSM
hook that was added this merge window. Landlock now logs the
allocated id instead of zero when it denies the creation of a
namespace.
-Don't mark the arguments of the namespace_init and namespace_free
hooks as trusted for bpf LSM programs. Both see namespaces that
are only partially set up or on their way out.
- proc: Don't let proc_sys_call_handler() report more bytes written
than it was passed when a cgroup/sysctl bpf program replaced the
value with a longer one. A splice() into the sysctl file then walked
past the pipe head.
- coda: Pass the append state of each write on to the container file
with RWF_APPEND or RWF_NOAPPEND. coda_open() sets O_APPEND on the
container file after its iocb flags were cached, so an O_APPEND write
overwrote existing data instead of appending.
- adfs: Reject big directory entries in adfs_fplus_getnext() whose name
doesn't fit into the name buffer. A crafted image overflowed it on
the stack.
* tag 'vfs-7.4-rc3.fixes.2' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (28 commits)
nstree: drop ns_tree_add_raw()
ns: allocate the namespace id before the namespace_init hook
bpf: don't mark namespace_init and namespace_free arguments as trusted
splice: replace nosteal_pipe_buf_ops with default_pipe_buf_ops
splice: remove pipe buffer stealing
fuse: don't move pipe buffers into the page cache
virtio_console: copy pipe buffers in splice_write()
fs/adfs: bound directory entry name length in adfs_fplus_getnext()
dcache: let umount find a disconnected ex-mountpoint
dcache: wait for a racing eviction of a secondary root
dcache: let d_invalidate() wait for a d_set_mounted() in flight
fat: drop mmb_sync() from fat_evict_inode()
udf: drop mmb_sync() from udf_evict_inode()
minix: drop mmb_sync() from minix_evict_inode()
ext4: drop mmb_sync() from ext4_evict_inode()
ext2: drop mmb_sync() from ext2_evict_inode()
fs: don't strand I_METADATA_WRITEBACK inodes outside the LRU
fs: set I_METADATA_WRITEBACK in mmb_mark_buffer_dirty()
udf: don't allocate the inode block in udf_sync_inode_metadata()
minix: don't read the inode block in minix_sync_inode_metadata()
...
|
||
|
|
4c8b94e59b |
nstree: drop ns_tree_add_raw()
Now that ns_tree_add() doesn't allocate an id anymore it does the same
thing as ns_tree_add_raw(). Switch the remaining ns_tree_add_raw() users
over to ns_tree_add() and drop it. Rename __ns_tree_add_raw() to
__ns_tree_add() while at it.
No functional changes.
Fixes:
|
||
|
|
08471914d6 |
ns: allocate the namespace id before the namespace_init hook
Currently the namespace id is allocated only after __ns_common_init()
has called the namespace_init LSM hook. Most namespace types allocate it
in ns_tree_add() right before the namespace is added to the namespace
trees. So the namespace_init hook always sees an id of zero.
The namespace id is the only identifier of a namespace that is never
recycled. A bpf LSM wants to use it to track a namespace from
namespace_init until namespace_free, e.g., by using it as the key into a
map that associates data with the namespace. The inode number doesn't
work for that as it is quickly recycled.
So allocate the id in __ns_common_init() before calling the
namespace_init hook. Every namespace then has its id by the time it is
added to the namespace trees. The initial namespaces still get their
fixed ids, either statically from NS_COMMON_INIT() or for init_net from
ns_common_init(). So ns_tree_add() doesn't need to allocate an id
anymore. Ids are still handed out in creation order so mnt_ns_loop()
keeps working. Mount and ipc namespaces already allocated their id right
after ns_common_init() anyway.
If creating the namespace fails after the id has been allocated the id
is simply never used.
Landlock logs the namespace id when it denies the use of a namespace.
For a denied namespace creation that used to be zero. Now it's the id
that was allocated for the new namespace. That namespace never becomes
visible to userspace but it's the same id that other LSMs see in
namespace_init. Update the Landlock documentation and selftests
accordingly. The selftests can't know the id of a namespace whose
creation was denied so they only check that it isn't zero.
While at it, drop ns_tree_gen_id() as it has no users left.
Reproducer, 100 user and uts namespaces:
bpftrace -e 'fentry:bpf_lsm_namespace_init {
@[args.ns->ns_id ? "set" : "zero"] = count(); }' &
for i in $(seq 100); do unshare -Uu true; done
before: @[zero]: 200
after: @[set]: 200
Fixes:
|
||
|
|
ea8c4b37b8 |
bpf: don't mark namespace_init and namespace_free arguments as trusted
The bpf LSM marks the arguments of all hooks as trusted unless the hook is listed in untrusted_lsm_hooks. That list was added in commit |
||
|
|
0f76da93e5 |
splice: remove pipe buffer stealing
Nothing calls pipe_buf_try_steal() any more. Remove ->try_steal() from struct pipe_buf_operations, its implementations and the exported generic_pipe_buf_try_steal(). PIPE_BUF_FLAG_LRU and PIPE_BUF_FLAG_GIFT were set only for stealing, and go too. page_cache_pipe_buf_release() did nothing but drop the page and the LRU flag, so page cache and user page buffers use generic_pipe_buf_release(). vmsplice() still accepts SPLICE_F_GIFT, and splice() SPLICE_F_MOVE. Both were hints, and nothing acts on them now. Suggested-by: Linus Torvalds <[email protected]> Signed-off-by: Kiryl Shutsemau (Meta) <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Christian Brauner (Amutable) <[email protected]> |
||
|
|
fcbff7b56a |
sched_ext: make DSQ iterator stay drained after NULL
nldsq_cursor_next_task() unlinks the iteration cursor when it reaches the
end of the DSQ. A later call sees list_empty(&cursor->node), starts
from the DSQ head again, and returns the first task of the same
iteration.
The verifier relies on the iterator contract that once next() returns
NULL it keeps returning NULL: process_iter_next_call() marks the
iterator DRAINED and treats the result of later next() calls as the
constant zero. A restarted DSQ iteration therefore hands the BPF
program a task pointer while the verifier believes R0 == 0.
Remember the drained state in the cursor flags and return NULL on every
later next() call. The cursor has spare flag bits and the BPF-visible
iterator layout is unchanged.
Fixes:
|
||
|
|
56681bc448 |
cgroup/cpuset: Correctly set isolated_cpus when switching partition state
When update_prstate() is called to switch the partition state from
root to isolated or vice versa, the whole effective_xcpus is passed to
isolated_cpus_update(). However, if there are child partitions present,
some of the effective_xcpus are dedicated to those child partitions
and their partition states will remain the same or get invalidated in
the case of child root partition with isolated parent in the latter
update_cpumasks_hier() call. In the latter case, their exclusive CPUs
will be returned to their parent.
Therefore isolated_cpus may be set incorrectly with child partitions
underneath. Fix that by excluding their exclusive CPUs in computing
the correct cpumask of exclusive CPUs used by that partition alone.
Add new test cases into test_cpuset_prs.sh to test for such cases.
Fixes:
|
||
|
|
a25cc4c75f |
cgroup/cpuset: Protect current->mems_allowed read in cpuset_fork()
Reading mems_allowed of the current task outside of cpuset_mutex critical
section should be protected by using mems_allowed_seq seqlock as the
nodemask can be updated concurrently leading to a torn read. Such
protection is currently not implemented in cpuset_fork() for the
!CLONE_INTO_CGROUP case. As a result, the child task may inherit an empty
or corrupted mems_allowed nodemask. This violates NUMA memory policy
constraints and causes subsequent page allocations to unconditionally
fail, potentially leading to crashes or spurious OOM killer invocations
for the child task.
Fix that by using a read_mems_allowed_begin/read_mems_allowed_retry()
loop to protect the mems_allowed read.
Fixes:
|
||
|
|
8ef0c92310 |
selftests/cgroup: Fix "test_cpuset_prs.sh -v" failures
When the -v option is used to run test_cpuset_prs.sh, it activates the use of sched-debug to determine the set of isolated CPUs which will also consider 1-CPU partition as being isolated though they will not be reflected in "cpuset.cpus.isolated". As a result, some of the expected isolated CPUs values have 2 different set of values to correspond to these two ways of determining isolated CPUs. Commit |
||
|
|
3b1ce3e0cc |
sched_ext: Fix partial-SMT core counting in calc_shard_layout()
calc_shard_layout() counts the cores of an LLC to spread them evenly
across shards. A core is counted when its cpu is the first cpu of
topology_sibling_cpumask(), but the cores scx_cid_init() builds are
the siblings intersected with @llc_cpus.
The sibling mask may contain cpus outside @llc_cpus - siblings which
landed in another node or LLC unit, or offlined siblings on archs
that keep them in the mask (mips, sparc64). A core whose first
sibling is such a cpu is never counted, so cores_per_shard and
nr_large come out too small and the shards end up uneven.
Count a core at its first sibling in @llc_cpus, matching the cores the
build loop constructs.
Fixes:
|
||
|
|
e4fb7e5e8e |
Merge tag 'probes-fixes-v7.4-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull probe fixes from Masami Hiramatsu:
"Fix registration and unregistration error handling:
- Release the instruction slot via arch_remove_kprobe() and propagate
the error code when arm_kprobe() fails in
__register_kprobe() instead of leaking the slot and returning 0.
- Skip rethook_free() and kretprobe holder cleanup in
unregister_kretprobes() when __unregister_kprobe_top() fails, and
clear rp->rh after synchronize_rcu() to avoid double free and NULL
pointer dereferences.
- Resolve the target address up-front in register_kretprobe() and
check for re-registration before allocating resources so
symbol_name-based re-registration does not corrupt the active
kretprobe on error.
Fix optprobe state transitions during forced unoptimization:
- Disarm a disabled optprobe in unoptimize_kprobe() when forced
unoptimization moves it directly to a cooling or freeing list, for
preventing breakpoint callback of freed probes.
- Skip disarming unhashed optprobes in the optimizer dispatch path so
a newly registered probe at the same address does not get disarmed
(or have freed module text written to).
- Keep a reused cooling optprobe marked disabled in
reuse_unused_kprobe() so register_aggr_kprobe() or enable_kprobe()
properly arms it, and skip redundant force-unoptimization cleanup
for already-unoptimized cooling probes.
Fix symstr buffer overflow and eprobe triggers:
- Format symstr symbols into a temporary buffer and copy only up to
the reserved field length (updating __data_loc accordingly) to
prevent overwriting the trace record if a probed pointer changes
between sizing and storing passes.
- Install and remove the eprobe event trigger for each trace instance
file that enables the eprobe rather than only on first enable and
last disable, so secondary instances record events and disabled
triggers do not reference freed eprobes.
Replace printk() with trace_printk() in probe handlers:
- Replace printk() and stack_trace_print() with trace_printk() and
trace_dump_stack() in kprobe and fprobe sample modules, and depend
on CONFIG_EVENT_TRACING.
- Document in kprobes.rst and fprobe.rst that calling printk()
directly from probe handlers can deadlock on console or scheduler
locks and recommend trace_printk() instead"
* tag 'probes-fixes-v7.4-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
kprobes: Arm reused optprobe on cooling list
kprobes: Do not disarm unhashed optprobe on dispatch
kprobes: Disarm disabled optprobe on forced unoptimization
tracing/eprobe: Install the trigger for every instance that enables the eprobe
tracing/probes: bound symstr stores to reserved length
docs: trace: Warn against using printk() in probe handlers
samples/fprobe: Use trace_printk() and trace_dump_stack()
samples/kprobes: Use trace_printk() in probe handlers
kprobes: Validate re-registration earlier in register_kretprobe()
kprobes: Fix double free and NULL deref in unregister_kretprobes()
kprobes: Fix error return and slot leak in __register_kprobe()
|
||
|
|
c1b96ba87e |
timekeeping: Refresh auxiliary base before offset validation
timekeeping_forward_now() can advance an auxiliary timekeeper across a
second boundary without refreshing tkr_mono.base. The negative-time
check in __timekeeping_inject_offset() then uses the stale base and
rejects valid ADJ_SETOFFSET adjustments with -EINVAL.
Refresh the derived timekeeper data before calculating the current
auxiliary time, as aux_clock_set() already does.
On 7.4-rc1, a valid negative adjustment fails at the first second
rollover. The patched kernel accepts all 12 tested rollover adjustments
and continues to reject invalid negative offsets.
Fixes:
|
||
|
|
67f1969e29 |
timekeeping: Reject cross timestamps for invalid AUX clocks
get_device_system_crosststamp() accepts AUX clock IDs even when the
corresponding timekeeper is disabled. A never-enabled AUX timekeeper
has no initialized clocksource, and a previously enabled one can
retain stale timekeeping data.
Return -ENODEV before calling the device timestamp callback when the
selected timekeeper is invalid.
Fixes:
|
||
|
|
c49b3c20a2 |
tsacct: Keep task begin times stable across suspend
bacct_add_tsk() derives ac_btime and ac_btime64 by subtracting a
CLOCK_MONOTONIC elapsed duration from CLOCK_REALTIME. MONOTONIC stops
during suspend while REALTIME continues, so querying an existing task
after resume moves its reported begin time forward by the time spent
suspended.
Calculate the begin time from the wall-clock boot timestamp and the
task's CLOCK_BOOTTIME start timestamp instead. This accounts for suspend
and adds the nanoseconds before truncating to seconds. Keep ac_etime and
ac_tgetime on CLOCK_MONOTONIC and retain the existing rebasing when the
wall clock is adjusted.
The new begin time can be negative: setting CLOCK_REALTIME is bounded
by CLOCK_MONOTONIC, which excludes suspend, rather than CLOCK_BOOTTIME.
A permitted backward clock adjustment after suspend can therefore put
the task's begin time before the epoch. Clamp ac_btime64 at zero too,
instead of wrapping in the unsigned field, and retain its positive range.
Fixes:
|
||
|
|
385b5ec1f3 |
timekeeping: Read getboottime64() offsets under the sequence counter
getboottime64() computes the wall-clock boot timestamp from offs_real
minus offs_boot without checking tk_core.seq. A concurrent timekeeping
update can leave the reader with an inconsistent pair of offsets or,
on 32-bit systems, a torn 64-bit offset, producing an incorrect boot
timestamp.
Read both offsets under tk_core.seq and retry if an update overlaps
the sample. Convert the resulting local value to a timespec64 afterward.
Fixes:
|
||
|
|
93eb230146 |
cpuset: add early empty cpumask check in partition_xcpus_add/del
Add a check for an empty cpumask at the start of partition_xcpus_add() and partition_xcpus_del(). This allows the functions to return early, avoiding unnecessary computation when there is no work to be done. With these changes, partition_xcpus_add() and partition_xcpus_del() can be called even if xcpus is empty. The caller no longer needs to check whether xcpus is empty, eliminating many conditional statements. Signed-off-by: Chen Ridong <[email protected]> Reviewed-by: Guopeng Zhang <[email protected]> Reviewed-by: Waiman Long <[email protected]> Signed-off-by: Ridong Chen <[email protected]> Signed-off-by: Tejun Heo <[email protected]> |
||
|
|
ba4834fc81 |
cgroup/cpuset: Reset partition data when an invalid partition stays invalid
When cpuset.cpus or cpuset.cpus.exclusive of a partition root is changed, update_cpumask() and update_exclusive_cpumask() copy the new effective_xcpus computed for the trial cpuset into the cpuset, and call reset_partition_data() only if the partition was valid before and is invalid now: if ((old_prs > 0) && !is_partition_valid(cs)) reset_partition_data(cs); An invalid partition root (old_prs < 0) that stays invalid is not covered. Its effective_xcpus is then left with the trial value, which is computed from user_xcpus() and so falls back to cpuset.cpus when cpuset.cpus.exclusive is cleared. As an invalid partition root does not own any exclusive CPUs, and rm_siblings_excl_cpus() treats a non-empty effective_xcpus of a sibling without cpuset.cpus.exclusive as owned, those CPUs can then no longer be claimed by a sibling: # A1: cpuset.cpus=2-3, cpuset.cpus.exclusive=2-3, root, with tasks # offline CPUs 2-3 -> A1 becomes invalid echo > A1/cpuset.cpus.exclusive cat A1/cpuset.cpus.exclusive.effective # 2-3, expected empty echo 3 > B1/cpuset.cpus.exclusive # -EINVAL Reset the partition data whenever the cpuset is a partition root, valid or not, and ends up invalid. As effective_xcpus may be cleared by that, also force update_cpumasks_hier() to update the subtree, so that the effective_xcpus of the descendants doesn't keep the released CPUs. Add test cases for it. Fixes: |
||
|
|
ec6f673153 |
sched/core: Do not touch SMT siblings that have not joined the core yet
When a CPU comes online, it is added to cpu_smt_mask() of its siblings
before sched_core_cpu_starting() sets its rq->core to the core leader.
For example, on x86, ap_starting() calls set_cpu_sibling_map() first,
and calls notify_cpu_starting() after that. This order is intended.
Before sched_core_cpu_starting() runs, rq->core of the new CPU still
points to its own rq. But rq->core_enabled of the new CPU is already
set. __sched_core_flip() sets it for all possible CPUs, also for
offline CPUs, and CPU offline does not clear it. So __rq_lockp() of the
new CPU returns its own rq->__lock. It does not return the core wide
lock.
If a sibling CPU runs pick_next_task() for SCHED_CORE at this time, it
checks all CPUs in cpu_smt_mask(). It calls update_rq_clock() and
pick_task() for the rq of the new CPU. But it holds only its own core
lock, not the lock of the new CPU. Then lockdep shows this warning in
update_rq_clock().
debug_locks && !(lock_is_held(&(__rq_lockp(rq))->dep_map) != 0)
WARNING: kernel/sched/sched.h:1666 at update_rq_clock+0x40a/0xd20 kernel/sched/core.c:875, CPU#0: syz-executor251/5712
Call Trace:
<TASK>
pick_next_task kernel/sched/core.c:6375 [inline]
__schedule+0x1c69/0x6920 kernel/sched/core.c:7199
preempt_schedule_irq+0x4e/0x90 kernel/sched/core.c:7606
The next loops in pick_next_task() also change rq_i->core_pick of this
rq, and can call resched_curr() for it, without its lock.
resched_curr() has the same lockdep_assert_rq_held() check.
__sched_core_account_forceidle() also checks all CPUs in the same mask.
CPU offline does not have this problem. remove_siblinginfo() and
sched_core_cpu_dying() both run in take_cpu_down() under stop_machine.
So no sibling CPU can be in pick_next_task() at that time.
Fix this. In these loops, skip a sibling CPU if its rq->core is
different from our rq->core. At run time, rq->core of a CPU is changed
by sched_core_cpu_starting(), sched_core_cpu_deactivate() and
sched_core_cpu_dying(). The first two hold sched_core_lock(), and
sched_core_lock() takes the rq locks of all CPUs in the SMT mask. The
last one runs under stop_machine, as written above. So while we hold
our core lock, the result of this check does not change. A CPU that is
not in the core yet will pick its own tasks. The comment in the
reschedule loop of pick_next_task() already expects this case.
I tested this with the syzbot C reproducer in QEMU. The two vCPUs were
SMT siblings (-smp 2,sockets=1,cores=1,threads=2). Without this patch,
the warning came after about 100 seconds. Just before the warning, a
debug print showed that the loop on CPU 0 saw CPU 1 with rq->core
pointing to CPU 1 itself, and cpu_online() was false for CPU 1. With
this patch, the same state still happened, but there was no warning
in more than 2 hours.
Fixes:
|
||
|
|
78b47fb995 |
sched/cache: Don't use fits_capacity() to compare thread and CPU counts
invalid_llc_nr() uses fits_capacity() to compare a thread count with a CPU count, only for its 20% margin. Qais's series changes fits_capacity() to take a CPU number as its second argument, and this call would then pass the CPU count as a CPU number without a warning. Give the check its own helper. x * 5 < y * 4 is the same comparison as x * 1280 < y * 1024, so there is no functional change. Signed-off-by: Zhan Xusheng <[email protected]> Signed-off-by: Peter Zijlstra (Intel) <[email protected]> Signed-off-by: Ingo Molnar <[email protected]> Reviewed-by: Qais Yousef <[email protected]> Reviewed-by: Chen Yu <[email protected]> Reviewed-by: Tim Chen <[email protected]> Link: https://lore.kernel.org/all/[email protected]/ Link: https://patch.msgid.link/[email protected] |
||
|
|
f7141257dd |
sched/core: Fix cpuset_mutex double unlock in __sched_setscheduler()
A sched_setattr() with SCHED_FLAG_KEEP_POLICY on a SCHED_DEADLINE task
unlocks cpuset_mutex twice if the task's policy changes concurrently:
WARNING: bad unlock balance detected!
7.4.0-rc1 #1 Not tainted
-------------------------------------
repro/126 is trying to release lock (cpuset_mutex) at:
[<ffffffff9a17cacc>] __sched_setscheduler+0x9cc/0xc40
but there are no more locks to release!
...
Call Trace:
print_unlock_imbalance_bug.part.0+0xfb/0x110
lock_release+0x286/0x2e0
__mutex_unlock_slowpath+0x1a/0x400
__sched_setscheduler+0x9cc/0xc40
__x64_sys_sched_setattr+0xa3/0x1b0
do_syscall_64+0x106/0x5f0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
__sched_setscheduler() samples p->policy before taking any lock, and
restarts from the recheck label if p->policy differs under the rq lock.
The restart drops cpuset_mutex but leaves cpuset_locked set, so a pass
that no longer needs the mutex still unlocks it on its way out:
CPU 1 CPU 2
__sched_setscheduler(KEEP_POLICY)
policy = p->policy /* DEADLINE */
__sched_setscheduler(SCHED_OTHER)
p->policy = SCHED_OTHER
cpuset_locked = true
cpuset_lock()
rq = task_rq_lock(p)
oldpolicy != p->policy
cpuset_unlock()
goto recheck
/* no cpuset_lock() this time */
...
if (cpuset_locked)
cpuset_unlock()
Without CONFIG_DEBUG_MUTEXES, or CONFIG_DEBUG_RT_MUTEXES on PREEMPT_RT,
nothing warns. The second unlock still clears the owner field regardless
of who holds the mutex, so another task can lose cpuset_mutex inside its
critical section. Entering SCHED_DEADLINE needs CAP_SYS_NICE, but a
racing caller that owns the task does not need it if it samples a policy
other than SCHED_DEADLINE.
Clear cpuset_locked when dropping the mutex for the restart. This should be
safe because the flag is only tested on the paths leaving the next pass,
after that pass has set it again if it took the mutex.
The following program, run as root, reproduces the splat, which no longer
appears with this applied. On PREEMPT_RT it crashes the kernel within
seconds, which this also fixes:
#define _GNU_SOURCE
#include <pthread.h>
#include <stdint.h>
#include <sys/syscall.h>
#include <unistd.h>
struct attr {
uint32_t size, policy;
uint64_t flags;
int32_t nice;
uint32_t prio;
uint64_t runtime, deadline, period;
};
static volatile int stop;
static volatile pid_t target;
static void set(struct attr a)
{
a.size = sizeof(a);
syscall(SYS_sched_setattr, target, &a, 0);
}
static void *sleeper(void *arg)
{
target = syscall(SYS_gettid);
while (!stop)
usleep(1000);
return arg;
}
static void *switcher(void *arg)
{
while (!stop) {
/* SCHED_DEADLINE, 2us every 1s */
set((struct attr){ .policy = 6, .runtime = 2000,
.deadline = 1000000000,
.period = 1000000000 });
set((struct attr){ .policy = 0 }); /* SCHED_OTHER */
}
return arg;
}
static void *racer(void *arg)
{
/* KEEP_POLICY | KEEP_PARAMS */
while (!stop)
set((struct attr){ .flags = 0x18 });
return arg;
}
int main(void)
{
pthread_t t[5];
int i;
pthread_create(&t[0], NULL, sleeper, NULL);
while (!target)
usleep(100);
pthread_create(&t[1], NULL, switcher, NULL);
for (i = 2; i < 5; i++)
pthread_create(&t[i], NULL, racer, NULL);
sleep(30);
stop = 1;
for (i = 0; i < 5; i++)
pthread_join(t[i], NULL);
return 0;
}
An LLM helped find the PREEMPT_RT crash and write this patch and its test.
Fixes:
|