Files
linux/fs/bpf_fs_kfuncs.c
Linus Torvalds 55fbedcec6 Merge tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Alexei Starovoitov:
 "Major changes:

   - Introduce scalar evolution (SCEV) analysis in the verifier.

     It infers how registers and stack slots change in a loop and lets
     the verifier check bounded loops with induction variables as ranges
     instead of walking every iteration. The number of processed insns
     drops ten fold in programs with loops (Eduard Zingerman)

   - Add 'callx' instruction: indirect call of a static BPF subprogram.

     The address is taken with ld_imm64 or loaded from read-only data.
     Supported by x86-64, arm64 and riscv64 JITs (Alexei Starovoitov, Pu
     Lehui)

   - Add unwind_goto and unwind_exit instructions, so that Rust panic or
     C++ exception can run destructors of every frame it passes through.
     The normal path is free of unwind code. Supported by x86-64 and
     arm64 JITs (Alexei Starovoitov)

   - Support programs compiled by rust-bpf: treat load and store through
     a number as arena access with BPF_F_ARENA_SCALAR, accept rust names
     in BTF (Alexei Starovoitov)

   - Raise BPF program stack size from 512 bytes to 2 KiB on x86-64 and
     arm64 (Kumar Kartikeya Dwivedi)

   - Support struct, union and __int128 up to 16 bytes passed and
     returned by value in BPF subprograms and kfuncs, and fix x86-64 and
     arm64 trampolines for such arguments (Yonghong Song)

   - Introduce a common way to attach struct_ops to a cgroup via
     bpf_link. bpf_tcp_ops is the first user and mirrors TCP sock_ops
     callbacks (Martin KaFai Lau, Amery Hung)

   - Describe inlined functions in BTF. New kinds tell where the inline
     sites are and how to fetch the arguments there.

     CONFIG_DEBUG_INFO_BTF_INLINE keeps it apart from the main BTF in
     .BTF.inline section or in btf_vmlinux_inline.ko (Alan Maguire)

   - Fix many memory safety, lifetime and locking bugs in BPF core,
     verifier, iterators, local storage, dynptr, devmap, cpumap and
     seg6local (Josef Bacik, Alexei Starovoitov)

   - Diff stat is dominated by selftests and by removal of
     lib/test_bpf.c. Its 16k lines of instruction tests were converted
     to BPF selftests that BPF CI runs on every architecture it covers
     (Daniel Borkmann)

  Other features and fixes:

   - Add KASAN checks to memory accesses of JITed programs on x86-64
     (Alexis Lothoré)

   - Complete unification of helper, kfunc and subprog argument checks
     in the verifier (Amery Hung)

   - Add bpf_copy_from_user_mm[_str]() kfuncs (Anastasios Papagiannis)

   - Various features and fixes (Daniel Borkmann):
       - Add .bpf keyring for program signature verification, modelled
         after dm-verity keyring, and support ML-DSA in signing tools
       - Emit shorter and faster code in x86-64 JIT for patterns that
         LLVM generates
       - Fix various issues in gotox handling

   - Let BPF LSM set xattrs at inode creation with bpf_inode_init_xattr()
     (David Windsor, Daniel Borkmann)

   - Use sleepable page allocation in sleepable arena paths. Fix races
     between arena free, allocation and user page fault (Emil Tsalapatis)

   - Switch BPF crypto from lskcipher to AES-CBC and AES-ECB libraries
     (Eric Biggers)

   - Add bpf_proactive_reclaim() kfunc (Hui Zhu)

   - Render decl_tags in libbpf btf_dump and add tests for bpftool C
     dump (Ihor Solodrai)

   - Speed up tracing_multi attach on x86 (Jiri Olsa)

   - Various features and fixes (Kumar Kartikeya Dwivedi):
       - Add BPF_PROG_STREAM_OPEN command to read BPF streams through a
         file descriptor with poll support
       - Let kernel structs publish arena pointers with __arena type tag
       - Fix __uninit kfunc output buffers and fastcall rewrite with
         indirect stack accesses
       - Refuse CO-RE relocations for signed programs

   - Add enqueue_rcvq and dequeue_rcvq hooks to bpf_tcp_ops to adjust
     sk_rcvlowat and avoid unnecessary wakeups (Kuniyuki Iwashima)

   - Add bpf_ksock_sendto() kfunc and allow ksock connect and send from
     bpf_wq (Mahe Tardy)

   - Fix racy plt target update in arm64 bpf_arch_text_poke()
     (Matthew Wood)

   - Restructure BPF selftests Makefile as a layered build
     (Mykola Lysenko)

   - Speed up hash map lookups for u32 and u64 keys (Mykyta Yatsenko)

   - Add signed div/mod, movsx, bswap and gotol to MIPS JITs, signed
     div/mod to parisc JITs (Nicholas Dudar, Johan Almbladh)

   - Fix stack overflow in build ID parser on 32-bit (Nickolai
     Zeldovich)

   - Add stack arguments, signed arena loads and gotox to riscv64 JIT.
     Pu Lehui is now co-maintainer of riscv BPF JITs (Pu Lehui, Chen
     Pei, Feng Jiang)

   - Add bpf_call_rcu() and bpf_call_rcu_tasks_trace() kfuncs. Support
     fetching and/or/xor atomics in arena on x86-64 (Puranjay Mohan)

   - Fix flaky selftests, selftests with clang 23 and bpf-gcc
     (Qiliang Yuan, KaFai Wan, Vineet Gupta)

   - Fix builds broken by fatal resolve_btfids warnings, BTF generation
     on ARC, and define arch_cmpxchg_local() on arc, hexagon and
     openrisc (Sasha Levin)

   - Reduce memory use of hash maps by up to 50% per element
     (T.J. Mercier)

   - Mark pointer fields of sched_ext ops arguments as trusted (Tejun
     Heo)

   - Remove legacy bpf_asm, bpf_dbg and bpf_jit_disasm tools (Xia Yan)

   - Fix index overflow in queue and stack maps, arena range tree after
     allocation failure, tail call loop after freplace detach, lost
     signal in bpf_send_signal() (Yuan Chen)

   - Many more fixes and cleanups"

* tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (660 commits)
  selftests/bpf: Time tc_edt from its first packet with data
  selftests/bpf: Test unwind_exit of an exception callback
  selftests/bpf: Tests for 32-bit scalar evolution bounds computation
  bpf: Reject unwind_goto and unwind_exit in offloaded progs
  bpf: Support 32-bit linear scalar evolution
  bpf: Make unwind_exit of an exception callback exit in JITs
  selftests/bpf: Test forbidden bpf_ksock_sendto() LSM attach
  selftests/bpf: Test ksock sendto()
  bpf: Add bpf_ksock_sendto() kfunc
  selftests/bpf: Test unwinding from R0: R2 subprog
  selftests/bpf: Add a C++ prog with exceptions
  selftests/bpf: Add tests for unwind_goto and unwind_exit
  bpf, arm64: Support unwind_goto and unwind_exit
  bpf, x86: Support unwind_goto and unwind_exit
  bpf: Add unwind_goto and unwind_exit insns
  tools/bpf: Remove legacy BPF tools
  bpf: Annotate data races on bpf_lru_node type
  selftests/bpf: Add arena allocation race tests
  bpf: Fix arena race between page free and alloc leading to incoherency
  bpf: Revalidate faulted arena pages before installing user mappings
  ...
2026-10-23 13:08:49 -10:00

599 lines
18 KiB
C

// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2024 Google LLC. */
#include <linux/binfmt_misc.h>
#include <linux/bpf.h>
#include <linux/bpf_lsm.h>
#include <linux/btf.h>
#include <linux/btf_ids.h>
#include <linux/dcache.h>
#include <linux/fs.h>
#include <linux/fsnotify.h>
#include <linux/file.h>
#include <linux/kernfs.h>
#include <linux/lsm_hooks.h>
#include <linux/mm.h>
#include <linux/net.h>
#include <linux/xattr.h>
__bpf_kfunc_start_defs();
/**
* bpf_get_task_exe_file - get a reference on the exe_file struct file member of
* the mm_struct that is nested within the supplied
* task_struct
* @task: task_struct of which the nested mm_struct exe_file member to get a
* reference on
*
* Get a reference on the exe_file struct file member field of the mm_struct
* nested within the supplied *task*. The referenced file pointer acquired by
* this BPF kfunc must be released using bpf_put_file(). Failing to call
* bpf_put_file() on the returned referenced struct file pointer that has been
* acquired by this BPF kfunc will result in the BPF program being rejected by
* the BPF verifier.
*
* This BPF kfunc may only be called from BPF LSM programs.
*
* Internally, this BPF kfunc leans on get_task_exe_file(), such that calling
* bpf_get_task_exe_file() would be analogous to calling get_task_exe_file()
* directly in kernel context.
*
* Return: A referenced struct file pointer to the exe_file member of the
* mm_struct that is nested within the supplied *task*. On error, NULL is
* returned.
*/
__bpf_kfunc struct file *bpf_get_task_exe_file(struct task_struct *task)
{
return get_task_exe_file(task);
}
/**
* bpf_put_file - put a reference on the supplied file
* @file: file to put a reference on
*
* Put a reference on the supplied *file*. Only referenced file pointers may be
* passed to this BPF kfunc. Attempting to pass an unreferenced file pointer, or
* any other arbitrary pointer for that matter, will result in the BPF program
* being rejected by the BPF verifier.
*
* This BPF kfunc may only be called from BPF LSM programs.
*/
__bpf_kfunc void bpf_put_file(struct file *file)
{
fput(file);
}
/**
* bpf_path_d_path - resolve the pathname for the supplied path
* @path: path to resolve the pathname for
* @buf: buffer to return the resolved pathname in
* @buf__sz: length of the supplied buffer
*
* Resolve the pathname for the supplied *path* and store it in *buf*. This BPF
* kfunc is the safer variant of the legacy bpf_d_path() helper and should be
* used in place of bpf_d_path() whenever possible.
*
* This BPF kfunc may only be called from BPF LSM programs.
*
* Return: A positive integer corresponding to the length of the resolved
* pathname in *buf*, including the NUL termination character. On error, a
* negative integer is returned.
*/
__bpf_kfunc int bpf_path_d_path(const struct path *path, char *buf, size_t buf__sz)
{
int len;
char *ret;
if (!buf__sz)
return -EINVAL;
ret = d_path(path, buf, buf__sz);
if (IS_ERR(ret))
return PTR_ERR(ret);
len = buf + buf__sz - ret;
memmove(buf, ret, len);
return len;
}
static bool match_security_bpf_prefix(const char *name__str)
{
return !strncmp(name__str, XATTR_NAME_BPF_LSM, XATTR_NAME_BPF_LSM_LEN);
}
static int bpf_xattr_read_permission(const char *name, struct inode *inode)
{
if (!inode)
return -EINVAL;
/* Allow reading xattr with user. and security.bpf. prefix */
if (strncmp(name, XATTR_USER_PREFIX, XATTR_USER_PREFIX_LEN) &&
!match_security_bpf_prefix(name))
return -EPERM;
return inode_permission(&nop_mnt_idmap, inode, MAY_READ);
}
/**
* bpf_get_dentry_xattr - get xattr of a dentry
* @dentry: dentry to get xattr from
* @name__str: name of the xattr
* @value_p: output buffer of the xattr value
*
* Get xattr *name__str* of *dentry* and store the output in *value_ptr*.
*
* For security reasons, only *name__str* with prefixes "user." or
* "security.bpf." are allowed.
*
* Return: length of the xattr value on success, a negative value on error.
*/
__bpf_kfunc int bpf_get_dentry_xattr(struct dentry *dentry, const char *name__str,
struct bpf_dynptr *value_p)
{
struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p;
struct inode *inode = d_inode(dentry);
u32 value_len;
void *value;
int ret;
value_len = __bpf_dynptr_size(value_ptr);
value = __bpf_dynptr_data_rw(value_ptr, value_len);
if (!value)
return -EINVAL;
ret = bpf_xattr_read_permission(name__str, inode);
if (ret)
return ret;
return __vfs_getxattr(dentry, inode, name__str, value, value_len);
}
/**
* bpf_get_file_xattr - get xattr of a file
* @file: file to get xattr from
* @name__str: name of the xattr
* @value_p: output buffer of the xattr value
*
* Get xattr *name__str* of *file* and store the output in *value_ptr*.
*
* For security reasons, only *name__str* with prefixes "user." or
* "security.bpf." are allowed.
*
* Return: length of the xattr value on success, a negative value on error.
*/
__bpf_kfunc int bpf_get_file_xattr(struct file *file, const char *name__str,
struct bpf_dynptr *value_p)
{
struct dentry *dentry;
dentry = file_dentry(file);
return bpf_get_dentry_xattr(dentry, name__str, value_p);
}
__bpf_kfunc_end_defs();
static int bpf_xattr_write_permission(const char *name, struct inode *inode)
{
if (!inode)
return -EINVAL;
/* Only allow setting and removing security.bpf. xattrs */
if (!match_security_bpf_prefix(name))
return -EPERM;
return inode_permission(&nop_mnt_idmap, inode, MAY_WRITE);
}
/**
* bpf_set_dentry_xattr_locked - set a xattr of a dentry
* @dentry: dentry to get xattr from
* @name__str: name of the xattr
* @value_p: xattr value
* @flags: flags to pass into filesystem operations
*
* Set xattr *name__str* of *dentry* to the value in *value_ptr*.
*
* For security reasons, only *name__str* with prefix "security.bpf."
* is allowed.
*
* The caller already locked dentry->d_inode.
*
* Return: 0 on success, a negative value on error.
*/
int bpf_set_dentry_xattr_locked(struct dentry *dentry, const char *name__str,
const struct bpf_dynptr *value_p, int flags)
{
const struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p;
struct inode *inode = d_inode(dentry);
const void *value;
u32 value_len;
int ret;
value_len = __bpf_dynptr_size(value_ptr);
value = __bpf_dynptr_data(value_ptr, value_len);
if (!value)
return -EINVAL;
ret = bpf_xattr_write_permission(name__str, inode);
if (ret)
return ret;
ret = __vfs_setxattr(&nop_mnt_idmap, dentry, inode, name__str,
value, value_len, flags);
if (!ret) {
fsnotify_xattr(dentry);
/* This xattr is set by BPF LSM, so we do not call
* security_inode_post_setxattr. Otherwise, we would
* risk deadlocks by calling back to the same kfunc.
*
* This is the same as security_inode_setsecurity().
*/
}
return ret;
}
/**
* bpf_remove_dentry_xattr_locked - remove a xattr of a dentry
* @dentry: dentry to get xattr from
* @name__str: name of the xattr
*
* Remove xattr *name__str* of *dentry*.
*
* For security reasons, only *name__str* with prefix "security.bpf."
* is allowed.
*
* The caller already locked dentry->d_inode.
*
* Return: 0 on success, a negative value on error.
*/
int bpf_remove_dentry_xattr_locked(struct dentry *dentry, const char *name__str)
{
struct inode *inode = d_inode(dentry);
int ret;
ret = bpf_xattr_write_permission(name__str, inode);
if (ret)
return ret;
ret = __vfs_removexattr(&nop_mnt_idmap, dentry, name__str);
if (!ret) {
fsnotify_xattr(dentry);
/* This xattr is removed by BPF LSM, so we do not call
* security_inode_post_removexattr. Otherwise, we would
* risk deadlocks by calling back to the same kfunc.
*/
}
return ret;
}
__bpf_kfunc_start_defs();
/**
* bpf_set_dentry_xattr - set a xattr of a dentry
* @dentry: dentry to get xattr from
* @name__str: name of the xattr
* @value_p: xattr value
* @flags: flags to pass into filesystem operations
*
* Set xattr *name__str* of *dentry* to the value in *value_ptr*.
*
* For security reasons, only *name__str* with prefix "security.bpf."
* is allowed.
*
* The caller has not locked dentry->d_inode.
*
* Return: 0 on success, a negative value on error.
*/
__bpf_kfunc int bpf_set_dentry_xattr(struct dentry *dentry, const char *name__str,
const struct bpf_dynptr *value_p, int flags)
{
struct inode *inode = d_inode(dentry);
int ret;
if (!inode)
return -EINVAL;
inode_lock(inode);
ret = bpf_set_dentry_xattr_locked(dentry, name__str, value_p, flags);
inode_unlock(inode);
return ret;
}
/**
* bpf_remove_dentry_xattr - remove a xattr of a dentry
* @dentry: dentry to get xattr from
* @name__str: name of the xattr
*
* Remove xattr *name__str* of *dentry*.
*
* For security reasons, only *name__str* with prefix "security.bpf."
* is allowed.
*
* The caller has not locked dentry->d_inode.
*
* Return: 0 on success, a negative value on error.
*/
__bpf_kfunc int bpf_remove_dentry_xattr(struct dentry *dentry, const char *name__str)
{
struct inode *inode = d_inode(dentry);
int ret;
if (!inode)
return -EINVAL;
inode_lock(inode);
ret = bpf_remove_dentry_xattr_locked(dentry, name__str);
inode_unlock(inode);
return ret;
}
static int bpf_inode_init_xattrs_claimed(const struct xattr *xattrs, int xattr_count)
{
const size_t suffix_len = sizeof(XATTR_BPF_LSM_SUFFIX) - 1;
int i, claimed = 0;
for (i = 0; i < xattr_count; i++) {
const char *name = xattrs[i].name;
if (name && !strncmp(name, XATTR_BPF_LSM_SUFFIX, suffix_len))
claimed++;
}
return claimed;
}
/**
* bpf_inode_init_xattr - attach a xattr to an inode that is being created
* @xattrs: xattr array the inode_init_security hook was handed
* @xattr_count__ctx_out: xattr count the inode_init_security hook was handed
* @name__str: name of the xattr
* @value_p: xattr value
*
* Claim one of the slots the BPF LSM reserved in the xattr array, so that
* the xattr is written out as part of the transaction creating the inode.
*
* For security reasons, only *name__str* with prefix "security.bpf." is
* allowed. The slot stores the name without that "security." prefix, which
* the filesystem's initxattrs() callback puts back.
*
* At most BPF_LSM_INODE_INIT_XATTRS xattrs can be attached to one inode,
* matching the number of slots the BPF LSM reserves.
*
* Return: 0 on success, a negative value on error.
*/
__bpf_kfunc int bpf_inode_init_xattr(struct xattr *xattrs,
int *xattr_count__ctx_out,
const char *name__str,
const struct bpf_dynptr *value_p)
{
const struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p;
int *xattr_count = xattr_count__ctx_out;
const char *suffix;
struct xattr *slot;
const void *value;
size_t name_len;
u32 value_len;
char *buf;
if (!match_security_bpf_prefix(name__str))
return -EPERM;
if (bpf_inode_init_xattrs_claimed(xattrs, *xattr_count) >=
BPF_LSM_INODE_INIT_XATTRS)
return -ENOSPC;
suffix = name__str + XATTR_SECURITY_PREFIX_LEN;
name_len = strlen(suffix);
if (name_len <= sizeof(XATTR_BPF_LSM_SUFFIX) - 1 ||
name_len > XATTR_NAME_MAX - XATTR_SECURITY_PREFIX_LEN)
return -EINVAL;
value_len = __bpf_dynptr_size(value_ptr);
value = __bpf_dynptr_data(value_ptr, value_len);
if (!value)
return -EINVAL;
if (value_len > XATTR_SIZE_MAX)
return -E2BIG;
buf = kmalloc(value_len + name_len + 1, GFP_NOWAIT | __GFP_NOWARN);
if (!buf)
return -ENOMEM;
memcpy(buf, value, value_len);
memcpy(buf + value_len, suffix, name_len + 1);
slot = lsm_get_xattr_slot(xattrs, xattr_count);
if (!slot) {
kfree(buf);
return -ENOSPC;
}
slot->value = buf;
slot->value_len = value_len;
slot->name = buf + value_len;
return 0;
}
#ifdef CONFIG_CGROUPS
/**
* bpf_cgroup_read_xattr - read xattr of a cgroup's node in cgroupfs
* @cgroup: cgroup to get xattr from
* @name__str: name of the xattr
* @value_p: output buffer of the xattr value
*
* Get xattr *name__str* of *cgroup* and store the output in *value_ptr*.
*
* For security reasons, only *name__str* with prefix "user." is allowed.
*
* Return: length of the xattr value on success, a negative value on error.
*/
__bpf_kfunc int bpf_cgroup_read_xattr(struct cgroup *cgroup, const char *name__str,
struct bpf_dynptr *value_p)
{
struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p;
u32 value_len;
void *value;
/* Only allow reading "user.*" xattrs */
if (strncmp(name__str, XATTR_USER_PREFIX, XATTR_USER_PREFIX_LEN))
return -EPERM;
value_len = __bpf_dynptr_size(value_ptr);
value = __bpf_dynptr_data_rw(value_ptr, value_len);
if (!value)
return -EINVAL;
return kernfs_xattr_get(cgroup->kn, name__str, value, value_len);
}
#endif /* CONFIG_CGROUPS */
#ifdef CONFIG_NET
/**
* bpf_sock_read_xattr - read xattr of a socket's inode in sockfs
* @sock: socket to get xattr from
* @name__str: name of the xattr
* @value_p: output buffer of the xattr value
*
* Get xattr *name__str* of *sock* and store the output in *value_p*.
*
* For security reasons, only *name__str* with prefix "user." is allowed.
*
* Return: length of the xattr value on success, a negative value on error.
*/
__bpf_kfunc int bpf_sock_read_xattr(struct socket *sock, const char *name__str,
struct bpf_dynptr *value_p)
{
struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p;
u32 value_len;
void *value;
/* Only allow reading "user.*" xattrs */
if (strncmp(name__str, XATTR_USER_PREFIX, XATTR_USER_PREFIX_LEN))
return -EPERM;
value_len = __bpf_dynptr_size(value_ptr);
value = __bpf_dynptr_data_rw(value_ptr, value_len);
if (!value)
return -EINVAL;
return sock_read_xattr(sock, name__str, value, value_len);
}
#endif /* CONFIG_NET */
/**
* bpf_real_data_inode - get the real inode hosting a file's data
* @file: file to resolve
*
* Resolve @file to the inode that hosts its data. For a regular file on a
* union/overlay filesystem this is the underlying (upper or lower) inode that
* stores the data, not the overlay inode.
*
* Data resolution only applies to regular files. For a non-regular file (e.g.
* a device node, fifo or socket) on a union/overlay filesystem the overlay
* inode itself is returned; for any file on a non-union filesystem the inode
* attached to @file is returned.
*
* Return: The inode hosting @file's data, or NULL.
*/
__bpf_kfunc struct inode *bpf_real_data_inode(struct file *file)
{
return d_real_inode(file_dentry(file));
}
__bpf_kfunc_end_defs();
BTF_KFUNCS_START(bpf_fs_kfunc_set_ids)
BTF_ID_FLAGS(func, bpf_get_task_exe_file, KF_ACQUIRE | KF_RET_NULL)
BTF_ID_FLAGS(func, bpf_put_file, KF_RELEASE)
BTF_ID_FLAGS(func, bpf_path_d_path)
BTF_ID_FLAGS(func, bpf_get_dentry_xattr, KF_SLEEPABLE)
BTF_ID_FLAGS(func, bpf_get_file_xattr, KF_SLEEPABLE)
BTF_ID_FLAGS(func, bpf_set_dentry_xattr, KF_SLEEPABLE)
BTF_ID_FLAGS(func, bpf_remove_dentry_xattr, KF_SLEEPABLE)
BTF_ID_FLAGS(func, bpf_real_data_inode, KF_SLEEPABLE | KF_RET_NULL)
BTF_ID_FLAGS(func, bpf_inode_init_xattr)
#ifdef CONFIG_NET
BTF_ID_FLAGS(func, bpf_sock_read_xattr, KF_RCU)
#endif
BTF_KFUNCS_END(bpf_fs_kfunc_set_ids)
/* Side-effecting kfuncs that stay exclusive to LSM programs. */
BTF_SET_START(bpf_fs_kfunc_lsm_only_ids)
BTF_ID(func, bpf_set_dentry_xattr)
BTF_ID(func, bpf_remove_dentry_xattr)
BTF_SET_END(bpf_fs_kfunc_lsm_only_ids)
BTF_ID_LIST_SINGLE(bpf_inode_init_xattr_ids, func, bpf_inode_init_xattr)
BTF_SET_START(bpf_inode_init_xattr_hooks)
BTF_ID(func, bpf_lsm_inode_init_security)
BTF_SET_END(bpf_inode_init_xattr_hooks)
static int bpf_fs_kfuncs_filter(const struct bpf_prog *prog, u32 kfunc_id)
{
if (!btf_id_set8_contains(&bpf_fs_kfunc_set_ids, kfunc_id))
return 0;
if (kfunc_id == bpf_inode_init_xattr_ids[0]) {
if (prog->type != BPF_PROG_TYPE_LSM ||
prog->expected_attach_type != BPF_LSM_MAC ||
!btf_id_set_contains(&bpf_inode_init_xattr_hooks,
prog->aux->attach_btf_id))
return -EACCES;
return 0;
}
if (prog->type == BPF_PROG_TYPE_LSM)
return 0;
if (prog->type != BPF_PROG_TYPE_STRUCT_OPS)
return -EACCES;
/* ->st_ops is unset during the cfg pass; enforced once it is set. */
if (!prog->aux->st_ops)
return 0;
if (bpf_prog_is_binfmt_misc_ops(prog) &&
!btf_id_set_contains(&bpf_fs_kfunc_lsm_only_ids, kfunc_id))
return 0;
return -EACCES;
}
/* bpf_[set|remove]_dentry_xattr.* hooks have KF_SLEEPABLE, so they are only
* available to sleepable hooks with dentry arguments.
*
* Setting and removing xattr requires exclusive lock on dentry->d_inode.
* Some hooks already locked d_inode, while some hooks have not locked
* d_inode. Therefore, we need different kfuncs for different hooks.
* Specifically, hooks in the following list (d_inode_locked_hooks)
* should call bpf_[set|remove]_dentry_xattr_locked; while other hooks
* should call bpf_[set|remove]_dentry_xattr.
*/
BTF_SET_START(d_inode_locked_hooks)
BTF_ID(func, bpf_lsm_inode_post_removexattr)
BTF_ID(func, bpf_lsm_inode_post_setattr)
BTF_ID(func, bpf_lsm_inode_post_setxattr)
BTF_ID(func, bpf_lsm_inode_removexattr)
BTF_ID(func, bpf_lsm_inode_rmdir)
BTF_ID(func, bpf_lsm_inode_setattr)
BTF_ID(func, bpf_lsm_inode_setxattr)
BTF_ID(func, bpf_lsm_inode_unlink)
BTF_SET_END(d_inode_locked_hooks)
bool bpf_lsm_has_d_inode_locked(const struct bpf_prog *prog)
{
return btf_id_set_contains(&d_inode_locked_hooks, prog->aux->attach_btf_id);
}
static const struct btf_kfunc_id_set bpf_fs_kfunc_set = {
.owner = THIS_MODULE,
.set = &bpf_fs_kfunc_set_ids,
.filter = bpf_fs_kfuncs_filter,
};
static int __init bpf_fs_kfuncs_init(void)
{
int ret;
ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_LSM, &bpf_fs_kfunc_set);
if (ret || !IS_ENABLED(CONFIG_BINFMT_MISC_BPF))
return ret;
return register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS,
&bpf_fs_kfunc_set);
}
late_initcall(bpf_fs_kfuncs_init);