* [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
@ 2020-11-23 7:36 Antony Yu
2020-11-23 18:16 ` Nathan Chancellor
` (2 more replies)
0 siblings, 3 replies; 17+ messages in thread
From: Antony Yu @ 2020-11-23 7:36 UTC (permalink / raw)
Cc: swpenim, Russell King, Nathan Chancellor, Nick Desaulniers,
linux-arm-kernel, linux-kernel, clang-built-linux
__do_div64 clobbers the input register r0 in little endian system.
According to the inline assembly document, if an input operand is
modified, it should be tied to a output operand. This patch can
prevent compilers from reusing r0 register after asm statements.
Signed-off-by: Antony Yu <swpenim@gmail.com>
---
arch/arm/include/asm/div64.h | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
index 898e9c78a7e7..809efc51e90f 100644
--- a/arch/arm/include/asm/div64.h
+++ b/arch/arm/include/asm/div64.h
@@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
asm( __asmeq("%0", __xh)
__asmeq("%1", "r2")
__asmeq("%2", "r0")
- __asmeq("%3", "r4")
+ __asmeq("%3", "r0")
+ __asmeq("%4", "r4")
"bl __do_div64"
- : "=r" (__rem), "=r" (__res)
+ : "=r" (__rem), "=r" (__res), "=r" (__n)
: "r" (__n), "r" (__base)
: "ip", "lr", "cc");
*n = __res;
--
2.23.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-23 7:36 [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang Antony Yu
@ 2020-11-23 18:16 ` Nathan Chancellor
2020-11-24 7:42 ` Antony Yu
2020-11-24 23:16 ` [RESEND,PATCH] " kernel test robot
2020-11-30 10:11 ` Ard Biesheuvel
2 siblings, 1 reply; 17+ messages in thread
From: Nathan Chancellor @ 2020-11-23 18:16 UTC (permalink / raw)
To: Antony Yu
Cc: Russell King, Nick Desaulniers, linux-arm-kernel, linux-kernel,
clang-built-linux
On Mon, Nov 23, 2020 at 03:36:32PM +0800, Antony Yu wrote:
> __do_div64 clobbers the input register r0 in little endian system.
> According to the inline assembly document, if an input operand is
> modified, it should be tied to a output operand. This patch can
> prevent compilers from reusing r0 register after asm statements.
>
> Signed-off-by: Antony Yu <swpenim@gmail.com>
> ---
> arch/arm/include/asm/div64.h | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> index 898e9c78a7e7..809efc51e90f 100644
> --- a/arch/arm/include/asm/div64.h
> +++ b/arch/arm/include/asm/div64.h
> @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> asm( __asmeq("%0", __xh)
> __asmeq("%1", "r2")
> __asmeq("%2", "r0")
> - __asmeq("%3", "r4")
> + __asmeq("%3", "r0")
> + __asmeq("%4", "r4")
> "bl __do_div64"
> - : "=r" (__rem), "=r" (__res)
> + : "=r" (__rem), "=r" (__res), "=r" (__n)
> : "r" (__n), "r" (__base)
> : "ip", "lr", "cc");
> *n = __res;
> --
> 2.23.0
>
I am not sure that I am qualified to review this (my assembly knowledge
is not the best) but your commit title mentions an error when compiling
with clang. What is the exact error, what configuration generates it,
and what version of clang? We have done fairly decent testing for
32-bit ARM, I would like to know what we are missing.
Cheers,
Nathan
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-23 18:16 ` Nathan Chancellor
@ 2020-11-24 7:42 ` Antony Yu
2020-11-24 10:14 ` Antony Yu
0 siblings, 1 reply; 17+ messages in thread
From: Antony Yu @ 2020-11-24 7:42 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Russell King, Nick Desaulniers, linux-arm-kernel, linux-kernel,
clang-built-linux
[-- Attachment #1: Type: text/plain, Size: 2242 bytes --]
On Mon, Nov 23, 2020 at 11:16:02AM -0700, Nathan Chancellor wrote:
> On Mon, Nov 23, 2020 at 03:36:32PM +0800, Antony Yu wrote:
> > __do_div64 clobbers the input register r0 in little endian system.
> > According to the inline assembly document, if an input operand is
> > modified, it should be tied to a output operand. This patch can
> > prevent compilers from reusing r0 register after asm statements.
> >
> > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > ---
> > arch/arm/include/asm/div64.h | 5 +++--
> > 1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > index 898e9c78a7e7..809efc51e90f 100644
> > --- a/arch/arm/include/asm/div64.h
> > +++ b/arch/arm/include/asm/div64.h
> > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > asm( __asmeq("%0", __xh)
> > __asmeq("%1", "r2")
> > __asmeq("%2", "r0")
> > - __asmeq("%3", "r4")
> > + __asmeq("%3", "r0")
> > + __asmeq("%4", "r4")
> > "bl __do_div64"
> > - : "=r" (__rem), "=r" (__res)
> > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > : "r" (__n), "r" (__base)
> > : "ip", "lr", "cc");
> > *n = __res;
> > --
> > 2.23.0
> >
>
> I am not sure that I am qualified to review this (my assembly knowledge
> is not the best) but your commit title mentions an error when compiling
> with clang. What is the exact error, what configuration generates it,
> and what version of clang? We have done fairly decent testing for
> 32-bit ARM, I would like to know what we are missing.
>
> Cheers,
> Nathan
We have run fail on android R vts vts_libsnapshot_test with kernel 4.14.
This bug is triggered accidently by a workaround patch in our code base.
It is fine on a pure clean 4.14 branch since __do_div64 may not be
executed in skip_metadata.
The attachment are .i and generated .s file. .s file can be reproduced
with clang -target arm-linux-eabi -march=armv8.2-a -O2.
In function skip_metadata, it loads some value to r0, calls __do_div64,
adds 1 to r0 and stores it to [r5]. It gets wrong value since __do_div64
clobbers r0 register.
We have tried clang-10, clang-11 and android prebuilt clang-r383902b. All
of them have the same problem.
[-- Attachment #2: dm-snap-persistent_clang.trim.i --]
[-- Type: text/plain, Size: 1412151 bytes --]
# 1 "drivers/md/dm-snap-persistent.c"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 366 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "././include/linux/kconfig.h" 1
# 1 "./include/generated/autoconf.h" 1
# 6 "././include/linux/kconfig.h" 2
# 74 "././include/linux/kconfig.h"
# 1 "./include/linux/compiler_types.h" 1
# 58 "./include/linux/compiler_types.h"
# 1 "./include/linux/compiler-gcc.h" 1
# 59 "./include/linux/compiler_types.h" 2
# 78 "./include/linux/compiler_types.h"
# 1 "./include/linux/compiler-clang.h" 1
# 79 "./include/linux/compiler_types.h" 2
# 99 "./include/linux/compiler_types.h"
struct ftrace_branch_data {
const char *func;
const char *file;
unsigned line;
union {
struct {
unsigned long correct;
unsigned long incorrect;
};
struct {
unsigned long miss;
unsigned long hit;
};
unsigned long miss_hit[2];
};
};
struct ftrace_likely_data {
struct ftrace_branch_data data;
unsigned long constant;
};
# 75 "././include/linux/kconfig.h" 2
# 2 "<built-in>" 2
# 1 "drivers/md/dm-snap-persistent.c" 2
# 1 "drivers/md/dm-exception-store.h" 1
# 13 "drivers/md/dm-exception-store.h"
# 1 "./include/linux/blkdev.h" 1
# 1 "./include/linux/sched.h" 1
# 10 "./include/linux/sched.h"
# 1 "./include/uapi/linux/sched.h" 1
# 11 "./include/linux/sched.h" 2
# 1 "./arch/arm/include/generated/asm/current.h" 1
# 1 "./include/asm-generic/current.h" 1
# 1 "./include/linux/thread_info.h" 1
# 11 "./include/linux/thread_info.h"
# 1 "./include/linux/types.h" 1
# 1 "./include/uapi/linux/types.h" 1
# 1 "./arch/arm/include/uapi/asm/types.h" 1
# 1 "./include/asm-generic/int-ll64.h" 1
# 11 "./include/asm-generic/int-ll64.h"
# 1 "./include/uapi/asm-generic/int-ll64.h" 1
# 12 "./include/uapi/asm-generic/int-ll64.h"
# 1 "./arch/arm/include/generated/uapi/asm/bitsperlong.h" 1
# 1 "./include/asm-generic/bitsperlong.h" 1
# 1 "./include/uapi/asm-generic/bitsperlong.h" 1
# 6 "./include/asm-generic/bitsperlong.h" 2
# 2 "./arch/arm/include/generated/uapi/asm/bitsperlong.h" 2
# 13 "./include/uapi/asm-generic/int-ll64.h" 2
typedef __signed__ char __s8;
typedef unsigned char __u8;
typedef __signed__ short __s16;
typedef unsigned short __u16;
typedef __signed__ int __s32;
typedef unsigned int __u32;
__extension__ typedef __signed__ long long __s64;
__extension__ typedef unsigned long long __u64;
# 12 "./include/asm-generic/int-ll64.h" 2
typedef signed char s8;
typedef unsigned char u8;
typedef signed short s16;
typedef unsigned short u16;
typedef signed int s32;
typedef unsigned int u32;
typedef signed long long s64;
typedef unsigned long long u64;
# 6 "./arch/arm/include/uapi/asm/types.h" 2
# 6 "./include/uapi/linux/types.h" 2
# 1 "./include/uapi/linux/posix_types.h" 1
# 1 "./include/linux/stddef.h" 1
# 1 "./include/uapi/linux/stddef.h" 1
# 6 "./include/linux/stddef.h" 2
enum {
false = 0,
true = 1
};
# 6 "./include/uapi/linux/posix_types.h" 2
# 25 "./include/uapi/linux/posix_types.h"
typedef struct {
unsigned long fds_bits[1024 / (8 * sizeof(long))];
} __kernel_fd_set;
typedef void (*__kernel_sighandler_t)(int);
typedef int __kernel_key_t;
typedef int __kernel_mqd_t;
# 1 "./arch/arm/include/uapi/asm/posix_types.h" 1
# 23 "./arch/arm/include/uapi/asm/posix_types.h"
typedef unsigned short __kernel_mode_t;
typedef unsigned short __kernel_ipc_pid_t;
typedef unsigned short __kernel_uid_t;
typedef unsigned short __kernel_gid_t;
typedef unsigned short __kernel_old_dev_t;
# 1 "./include/uapi/asm-generic/posix_types.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/bitsperlong.h" 1
# 6 "./include/uapi/asm-generic/posix_types.h" 2
# 15 "./include/uapi/asm-generic/posix_types.h"
typedef long __kernel_long_t;
typedef unsigned long __kernel_ulong_t;
typedef __kernel_ulong_t __kernel_ino_t;
typedef int __kernel_pid_t;
# 41 "./include/uapi/asm-generic/posix_types.h"
typedef __kernel_long_t __kernel_suseconds_t;
typedef int __kernel_daddr_t;
typedef unsigned int __kernel_uid32_t;
typedef unsigned int __kernel_gid32_t;
typedef __kernel_uid_t __kernel_old_uid_t;
typedef __kernel_gid_t __kernel_old_gid_t;
# 68 "./include/uapi/asm-generic/posix_types.h"
typedef unsigned int __kernel_size_t;
typedef int __kernel_ssize_t;
typedef int __kernel_ptrdiff_t;
# 79 "./include/uapi/asm-generic/posix_types.h"
typedef struct {
int val[2];
} __kernel_fsid_t;
typedef __kernel_long_t __kernel_off_t;
typedef long long __kernel_loff_t;
typedef __kernel_long_t __kernel_time_t;
typedef __kernel_long_t __kernel_clock_t;
typedef int __kernel_timer_t;
typedef int __kernel_clockid_t;
typedef char * __kernel_caddr_t;
typedef unsigned short __kernel_uid16_t;
typedef unsigned short __kernel_gid16_t;
# 37 "./arch/arm/include/uapi/asm/posix_types.h" 2
# 37 "./include/uapi/linux/posix_types.h" 2
# 15 "./include/uapi/linux/types.h" 2
# 29 "./include/uapi/linux/types.h"
typedef __u16 __le16;
typedef __u16 __be16;
typedef __u32 __le32;
typedef __u32 __be32;
typedef __u64 __le64;
typedef __u64 __be64;
typedef __u16 __sum16;
typedef __u32 __wsum;
# 7 "./include/linux/types.h" 2
typedef __u32 __kernel_dev_t;
typedef __kernel_fd_set fd_set;
typedef __kernel_dev_t dev_t;
typedef __kernel_ino_t ino_t;
typedef __kernel_mode_t mode_t;
typedef unsigned short umode_t;
typedef __u32 nlink_t;
typedef __kernel_off_t off_t;
typedef __kernel_pid_t pid_t;
typedef __kernel_daddr_t daddr_t;
typedef __kernel_key_t key_t;
typedef __kernel_suseconds_t suseconds_t;
typedef __kernel_timer_t timer_t;
typedef __kernel_clockid_t clockid_t;
typedef __kernel_mqd_t mqd_t;
typedef _Bool bool;
typedef __kernel_uid32_t uid_t;
typedef __kernel_gid32_t gid_t;
typedef __kernel_uid16_t uid16_t;
typedef __kernel_gid16_t gid16_t;
typedef unsigned long uintptr_t;
typedef __kernel_old_uid_t old_uid_t;
typedef __kernel_old_gid_t old_gid_t;
typedef __kernel_loff_t loff_t;
# 55 "./include/linux/types.h"
typedef __kernel_size_t size_t;
typedef __kernel_ssize_t ssize_t;
typedef __kernel_ptrdiff_t ptrdiff_t;
typedef __kernel_time_t time_t;
typedef __kernel_clock_t clock_t;
typedef __kernel_caddr_t caddr_t;
typedef unsigned char u_char;
typedef unsigned short u_short;
typedef unsigned int u_int;
typedef unsigned long u_long;
typedef unsigned char unchar;
typedef unsigned short ushort;
typedef unsigned int uint;
typedef unsigned long ulong;
typedef __u8 u_int8_t;
typedef __s8 int8_t;
typedef __u16 u_int16_t;
typedef __s16 int16_t;
typedef __u32 u_int32_t;
typedef __s32 int32_t;
typedef __u8 uint8_t;
typedef __u16 uint16_t;
typedef __u32 uint32_t;
typedef __u64 uint64_t;
typedef __u64 u_int64_t;
typedef __s64 int64_t;
# 131 "./include/linux/types.h"
typedef u64 sector_t;
typedef u64 blkcnt_t;
# 155 "./include/linux/types.h"
typedef u32 dma_addr_t;
typedef unsigned gfp_t;
typedef unsigned fmode_t;
typedef u32 phys_addr_t;
typedef phys_addr_t resource_size_t;
typedef unsigned long irq_hw_number_t;
typedef struct {
int counter;
} atomic_t;
struct list_head {
struct list_head *next, *prev;
};
struct hlist_head {
struct hlist_node *first;
};
struct hlist_node {
struct hlist_node *next, **pprev;
};
struct ustat {
__kernel_daddr_t f_tfree;
__kernel_ino_t f_tinode;
char f_fname[6];
char f_fpack[6];
};
# 223 "./include/linux/types.h"
struct callback_head {
struct callback_head *next;
void (*func)(struct callback_head *head);
} __attribute__((aligned(sizeof(void *))));
typedef void (*rcu_callback_t)(struct callback_head *head);
typedef void (*call_rcu_func_t)(struct callback_head *head, rcu_callback_t func);
# 12 "./include/linux/thread_info.h" 2
# 1 "./include/linux/bug.h" 1
# 1 "./arch/arm/include/asm/bug.h" 1
# 1 "./include/linux/linkage.h" 1
# 1 "./include/linux/stringify.h" 1
# 7 "./include/linux/linkage.h" 2
# 1 "./include/linux/export.h" 1
# 27 "./include/linux/export.h"
struct kernel_symbol
{
unsigned long value;
const char *name;
};
# 8 "./include/linux/linkage.h" 2
# 1 "./arch/arm/include/asm/linkage.h" 1
# 9 "./include/linux/linkage.h" 2
# 6 "./arch/arm/include/asm/bug.h" 2
# 1 "./arch/arm/include/asm/opcodes.h" 1
# 14 "./arch/arm/include/asm/opcodes.h"
extern unsigned int arm_check_condition(u32 opcode, u32 psr);
# 89 "./arch/arm/include/asm/opcodes.h"
# 1 "./include/linux/swab.h" 1
# 1 "./include/uapi/linux/swab.h" 1
# 1 "./include/linux/compiler.h" 1
# 180 "./include/linux/compiler.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline))
void __read_once_size(const volatile void *p, void *res, int size)
{
({ switch (size) { case 1: *(__u8 *)res = *(volatile __u8 *)p; break; case 2: *(__u16 *)res = *(volatile __u16 *)p; break; case 4: *(__u32 *)res = *(volatile __u32 *)p; break; case 8: *(__u64 *)res = *(volatile __u64 *)p; break; default: __asm__ __volatile__("": : :"memory"); __builtin_memcpy((void *)res, (const void *)p, size); __asm__ __volatile__("": : :"memory"); } });
}
# 198 "./include/linux/compiler.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline))
void __read_once_size_nocheck(const volatile void *p, void *res, int size)
{
({ switch (size) { case 1: *(__u8 *)res = *(volatile __u8 *)p; break; case 2: *(__u16 *)res = *(volatile __u16 *)p; break; case 4: *(__u32 *)res = *(volatile __u32 *)p; break; case 8: *(__u64 *)res = *(volatile __u64 *)p; break; default: __asm__ __volatile__("": : :"memory"); __builtin_memcpy((void *)res, (const void *)p, size); __asm__ __volatile__("": : :"memory"); } });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __write_once_size(volatile void *p, void *res, int size)
{
switch (size) {
case 1: *(volatile __u8 *)p = *(__u8 *)res; break;
case 2: *(volatile __u16 *)p = *(__u16 *)res; break;
case 4: *(volatile __u32 *)p = *(__u32 *)res; break;
case 8: *(volatile __u64 *)p = *(__u64 *)res; break;
default:
__asm__ __volatile__("": : :"memory");
__builtin_memcpy((void *)p, (const void *)res, size);
__asm__ __volatile__("": : :"memory");
}
}
# 240 "./include/linux/compiler.h"
# 1 "./arch/arm/include/asm/barrier.h" 1
# 82 "./arch/arm/include/asm/barrier.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long array_index_mask_nospec(unsigned long idx,
unsigned long sz)
{
unsigned long mask;
asm volatile(
"cmp %1, %2\n"
" sbc %0, %1, %1\n"
".inst 0xe320f014"
: "=r" (mask)
: "r" (idx), "Ir" (sz)
: "cc");
return mask;
}
# 1 "./include/asm-generic/barrier.h" 1
# 20 "./include/asm-generic/barrier.h"
# 1 "./include/linux/compiler.h" 1
# 21 "./include/asm-generic/barrier.h" 2
# 101 "./arch/arm/include/asm/barrier.h" 2
# 241 "./include/linux/compiler.h" 2
# 1 "./include/linux/kasan-checks.h" 1
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_check_read(const volatile void *p, unsigned int size)
{ }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_check_write(const volatile void *p, unsigned int size)
{ }
# 242 "./include/linux/compiler.h" 2
# 261 "./include/linux/compiler.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline))
unsigned long read_word_at_a_time(const void *addr)
{
kasan_check_read(addr, 1);
return *(unsigned long *)addr;
}
# 7 "./include/uapi/linux/swab.h" 2
# 1 "./arch/arm/include/generated/uapi/asm/bitsperlong.h" 1
# 8 "./include/uapi/linux/swab.h" 2
# 1 "./arch/arm/include/asm/swab.h" 1
# 19 "./arch/arm/include/asm/swab.h"
# 1 "./arch/arm/include/uapi/asm/swab.h" 1
# 20 "./arch/arm/include/asm/swab.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) __u32 __arch_swahb32(__u32 x)
{
__asm__ ("rev16 %0, %1" : "=r" (x) : "r" (x));
return x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) __u32 __arch_swab32(__u32 x)
{
__asm__ ("rev %0, %1" : "=r" (x) : "r" (x));
return x;
}
# 9 "./include/uapi/linux/swab.h" 2
# 48 "./include/uapi/linux/swab.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) __u16 __fswab16(__u16 val)
{
return ((__u16)__arch_swahb32(val));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) __u32 __fswab32(__u32 val)
{
return __arch_swab32(val);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) __u64 __fswab64(__u64 val)
{
__u32 h = val >> 32;
__u32 l = val & ((1ULL << 32) - 1);
return (((__u64)__fswab32(l)) << 32) | ((__u64)(__fswab32(h)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) __u32 __fswahw32(__u32 val)
{
return ((__u32)( (((__u32)(val) & (__u32)0x0000ffffUL) << 16) | (((__u32)(val) & (__u32)0xffff0000UL) >> 16)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) __u32 __fswahb32(__u32 val)
{
return __arch_swahb32(val);
}
# 136 "./include/uapi/linux/swab.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long __swab(const unsigned long y)
{
return (__builtin_constant_p((__u32)(y)) ? ((__u32)( (((__u32)(y) & (__u32)0x000000ffUL) << 24) | (((__u32)(y) & (__u32)0x0000ff00UL) << 8) | (((__u32)(y) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(y) & (__u32)0xff000000UL) >> 24))) : __fswab32(y));
}
# 171 "./include/uapi/linux/swab.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u16 __swab16p(const __u16 *p)
{
return (__builtin_constant_p((__u16)(*p)) ? ((__u16)( (((__u16)(*p) & (__u16)0x00ffU) << 8) | (((__u16)(*p) & (__u16)0xff00U) >> 8))) : __fswab16(*p));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u32 __swab32p(const __u32 *p)
{
return (__builtin_constant_p((__u32)(*p)) ? ((__u32)( (((__u32)(*p) & (__u32)0x000000ffUL) << 24) | (((__u32)(*p) & (__u32)0x0000ff00UL) << 8) | (((__u32)(*p) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(*p) & (__u32)0xff000000UL) >> 24))) : __fswab32(*p));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u64 __swab64p(const __u64 *p)
{
return (__builtin_constant_p((__u64)(*p)) ? ((__u64)( (((__u64)(*p) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(*p) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(*p) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(*p) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(*p) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(*p) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(*p) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(*p) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(*p));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u32 __swahw32p(const __u32 *p)
{
return (__builtin_constant_p((__u32)(*p)) ? ((__u32)( (((__u32)(*p) & (__u32)0x0000ffffUL) << 16) | (((__u32)(*p) & (__u32)0xffff0000UL) >> 16))) : __fswahw32(*p));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u32 __swahb32p(const __u32 *p)
{
return (__builtin_constant_p((__u32)(*p)) ? ((__u32)( (((__u32)(*p) & (__u32)0x00ff00ffUL) << 8) | (((__u32)(*p) & (__u32)0xff00ff00UL) >> 8))) : __fswahb32(*p));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __swab16s(__u16 *p)
{
*p = __swab16p(p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __swab32s(__u32 *p)
{
*p = __swab32p(p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __swab64s(__u64 *p)
{
*p = __swab64p(p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __swahw32s(__u32 *p)
{
*p = __swahw32p(p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __swahb32s(__u32 *p)
{
*p = __swahb32p(p);
}
# 6 "./include/linux/swab.h" 2
# 90 "./arch/arm/include/asm/opcodes.h" 2
# 8 "./arch/arm/include/asm/bug.h" 2
# 60 "./arch/arm/include/asm/bug.h"
# 1 "./include/asm-generic/bug.h" 1
# 16 "./include/asm-generic/bug.h"
# 1 "./include/linux/kernel.h" 1
# 1 "/home/fann/src/DB1101/kernel/android/R/prebuilts/clang/host/linux-x86/clang-r383902b/lib64/clang/11.0.2/include/stdarg.h" 1 3
# 14 "/home/fann/src/DB1101/kernel/android/R/prebuilts/clang/host/linux-x86/clang-r383902b/lib64/clang/11.0.2/include/stdarg.h" 3
typedef __builtin_va_list va_list;
# 32 "/home/fann/src/DB1101/kernel/android/R/prebuilts/clang/host/linux-x86/clang-r383902b/lib64/clang/11.0.2/include/stdarg.h" 3
typedef __builtin_va_list __gnuc_va_list;
# 7 "./include/linux/kernel.h" 2
# 1 "./include/linux/bitops.h" 1
# 1 "./include/linux/bits.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/bitsperlong.h" 1
# 5 "./include/linux/bits.h" 2
# 6 "./include/linux/bitops.h" 2
extern unsigned int __sw_hweight8(unsigned int w);
extern unsigned int __sw_hweight16(unsigned int w);
extern unsigned int __sw_hweight32(unsigned int w);
extern unsigned long __sw_hweight64(__u64 w);
# 1 "./arch/arm/include/asm/bitops.h" 1
# 28 "./arch/arm/include/asm/bitops.h"
# 1 "./include/linux/irqflags.h" 1
# 15 "./include/linux/irqflags.h"
# 1 "./include/linux/typecheck.h" 1
# 16 "./include/linux/irqflags.h" 2
# 1 "./arch/arm/include/asm/irqflags.h" 1
# 1 "./arch/arm/include/asm/ptrace.h" 1
# 13 "./arch/arm/include/asm/ptrace.h"
# 1 "./arch/arm/include/uapi/asm/ptrace.h" 1
# 14 "./arch/arm/include/uapi/asm/ptrace.h"
# 1 "./arch/arm/include/asm/hwcap.h" 1
# 1 "./arch/arm/include/uapi/asm/hwcap.h" 1
# 6 "./arch/arm/include/asm/hwcap.h" 2
extern unsigned int elf_hwcap, elf_hwcap2;
# 15 "./arch/arm/include/uapi/asm/ptrace.h" 2
# 14 "./arch/arm/include/asm/ptrace.h" 2
struct pt_regs {
unsigned long uregs[18];
};
struct svc_pt_regs {
struct pt_regs regs;
u32 dacr;
u32 addr_limit;
};
# 60 "./arch/arm/include/asm/ptrace.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int valid_user_regs(struct pt_regs *regs)
{
unsigned long mode = regs->uregs[16] & 0x0000001f;
regs->uregs[16] &= ~(0x00000040 | 0x00000100);
if ((regs->uregs[16] & 0x00000080) == 0) {
if (mode == 0x00000010)
return 1;
if (elf_hwcap & (1 << 3) && mode == 0x00000000)
return 1;
}
regs->uregs[16] &= 0xff000000 | 0x00ff0000 | 0x0000ff00 | 0x00000020 | 0x00000010;
if (!(elf_hwcap & (1 << 3)))
regs->uregs[16] |= 0x00000010;
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long regs_return_value(struct pt_regs *regs)
{
return regs->uregs[0];
}
# 103 "./arch/arm/include/asm/ptrace.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void instruction_pointer_set(struct pt_regs *regs,
unsigned long val)
{
(regs)->uregs[15] = val;
}
extern unsigned long profile_pc(struct pt_regs *regs);
# 132 "./arch/arm/include/asm/ptrace.h"
extern int regs_query_register_offset(const char *name);
extern const char *regs_query_register_name(unsigned int offset);
extern bool regs_within_kernel_stack(struct pt_regs *regs, unsigned long addr);
extern unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs,
unsigned int n);
# 147 "./arch/arm/include/asm/ptrace.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long regs_get_register(struct pt_regs *regs,
unsigned int offset)
{
if (__builtin_expect(!!(offset > (__builtin_offsetof(struct pt_regs, uregs[17]))), 0))
return 0;
return *(unsigned long *)((unsigned long)regs + offset);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long kernel_stack_pointer(struct pt_regs *regs)
{
return regs->uregs[13];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long user_stack_pointer(struct pt_regs *regs)
{
return regs->uregs[13];
}
# 8 "./arch/arm/include/asm/irqflags.h" 2
# 25 "./arch/arm/include/asm/irqflags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long arch_local_irq_save(void)
{
unsigned long flags;
asm volatile(
" mrs %0, " "cpsr" " @ arch_local_irq_save\n"
" cpsid i"
: "=r" (flags) : : "memory", "cc");
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_local_irq_enable(void)
{
asm volatile(
" cpsie i @ arch_local_irq_enable"
:
:
: "memory", "cc");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_local_irq_disable(void)
{
asm volatile(
" cpsid i @ arch_local_irq_disable"
:
:
: "memory", "cc");
}
# 156 "./arch/arm/include/asm/irqflags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long arch_local_save_flags(void)
{
unsigned long flags;
asm volatile(
" mrs %0, " "cpsr" " @ local_save_flags"
: "=r" (flags) : : "memory", "cc");
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_local_irq_restore(unsigned long flags)
{
asm volatile(
" msr " "cpsr_c" ", %0 @ local_irq_restore"
:
: "r" (flags)
: "memory", "cc");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_irqs_disabled_flags(unsigned long flags)
{
return flags & 0x00000080;
}
# 1 "./include/asm-generic/irqflags.h" 1
# 61 "./include/asm-generic/irqflags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_irqs_disabled(void)
{
return arch_irqs_disabled_flags(arch_local_save_flags());
}
# 185 "./arch/arm/include/asm/irqflags.h" 2
# 17 "./include/linux/irqflags.h" 2
# 29 "./arch/arm/include/asm/bitops.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ____atomic_set_bit(unsigned int bit, volatile unsigned long *p)
{
unsigned long flags;
unsigned long mask = (1UL << ((bit) % 32));
p += ((bit) / 32);
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
*p |= mask;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ____atomic_clear_bit(unsigned int bit, volatile unsigned long *p)
{
unsigned long flags;
unsigned long mask = (1UL << ((bit) % 32));
p += ((bit) / 32);
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
*p &= ~mask;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ____atomic_change_bit(unsigned int bit, volatile unsigned long *p)
{
unsigned long flags;
unsigned long mask = (1UL << ((bit) % 32));
p += ((bit) / 32);
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
*p ^= mask;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
____atomic_test_and_set_bit(unsigned int bit, volatile unsigned long *p)
{
unsigned long flags;
unsigned int res;
unsigned long mask = (1UL << ((bit) % 32));
p += ((bit) / 32);
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
res = *p;
*p = res | mask;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
return (res & mask) != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
____atomic_test_and_clear_bit(unsigned int bit, volatile unsigned long *p)
{
unsigned long flags;
unsigned int res;
unsigned long mask = (1UL << ((bit) % 32));
p += ((bit) / 32);
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
res = *p;
*p = res & ~mask;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
return (res & mask) != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
____atomic_test_and_change_bit(unsigned int bit, volatile unsigned long *p)
{
unsigned long flags;
unsigned int res;
unsigned long mask = (1UL << ((bit) % 32));
p += ((bit) / 32);
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
res = *p;
*p = res ^ mask;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
return (res & mask) != 0;
}
# 1 "./include/asm-generic/bitops/non-atomic.h" 1
# 16 "./include/asm-generic/bitops/non-atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __set_bit(int nr, volatile unsigned long *addr)
{
unsigned long mask = (1UL << ((nr) % 32));
unsigned long *p = ((unsigned long *)addr) + ((nr) / 32);
*p |= mask;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __clear_bit(int nr, volatile unsigned long *addr)
{
unsigned long mask = (1UL << ((nr) % 32));
unsigned long *p = ((unsigned long *)addr) + ((nr) / 32);
*p &= ~mask;
}
# 41 "./include/asm-generic/bitops/non-atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __change_bit(int nr, volatile unsigned long *addr)
{
unsigned long mask = (1UL << ((nr) % 32));
unsigned long *p = ((unsigned long *)addr) + ((nr) / 32);
*p ^= mask;
}
# 58 "./include/asm-generic/bitops/non-atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __test_and_set_bit(int nr, volatile unsigned long *addr)
{
unsigned long mask = (1UL << ((nr) % 32));
unsigned long *p = ((unsigned long *)addr) + ((nr) / 32);
unsigned long old = *p;
*p = old | mask;
return (old & mask) != 0;
}
# 77 "./include/asm-generic/bitops/non-atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __test_and_clear_bit(int nr, volatile unsigned long *addr)
{
unsigned long mask = (1UL << ((nr) % 32));
unsigned long *p = ((unsigned long *)addr) + ((nr) / 32);
unsigned long old = *p;
*p = old & ~mask;
return (old & mask) != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __test_and_change_bit(int nr,
volatile unsigned long *addr)
{
unsigned long mask = (1UL << ((nr) % 32));
unsigned long *p = ((unsigned long *)addr) + ((nr) / 32);
unsigned long old = *p;
*p = old ^ mask;
return (old & mask) != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_bit(int nr, const volatile unsigned long *addr)
{
return 1UL & (addr[((nr) / 32)] >> (nr & (32 -1)));
}
# 124 "./arch/arm/include/asm/bitops.h" 2
# 153 "./arch/arm/include/asm/bitops.h"
extern void _set_bit(int nr, volatile unsigned long * p);
extern void _clear_bit(int nr, volatile unsigned long * p);
extern void _change_bit(int nr, volatile unsigned long * p);
extern int _test_and_set_bit(int nr, volatile unsigned long * p);
extern int _test_and_clear_bit(int nr, volatile unsigned long * p);
extern int _test_and_change_bit(int nr, volatile unsigned long * p);
extern int _find_first_zero_bit_le(const unsigned long *p, unsigned size);
extern int _find_next_zero_bit_le(const unsigned long *p, int size, int offset);
extern int _find_first_bit_le(const unsigned long *p, unsigned size);
extern int _find_next_bit_le(const unsigned long *p, int size, int offset);
extern int _find_first_zero_bit_be(const unsigned long *p, unsigned size);
extern int _find_next_zero_bit_be(const unsigned long *p, int size, int offset);
extern int _find_first_bit_be(const unsigned long *p, unsigned size);
extern int _find_next_bit_be(const unsigned long *p, int size, int offset);
# 226 "./arch/arm/include/asm/bitops.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int constant_fls(int x)
{
int r = 32;
if (!x)
return 0;
if (!(x & 0xffff0000u)) {
x <<= 16;
r -= 16;
}
if (!(x & 0xff000000u)) {
x <<= 8;
r -= 8;
}
if (!(x & 0xf0000000u)) {
x <<= 4;
r -= 4;
}
if (!(x & 0xc0000000u)) {
x <<= 2;
r -= 2;
}
if (!(x & 0x80000000u)) {
x <<= 1;
r -= 1;
}
return r;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int __clz(unsigned int x)
{
unsigned int ret;
asm("clz\t%0, %1" : "=r" (ret) : "r" (x));
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int fls(int x)
{
if (__builtin_constant_p(x))
return constant_fls(x);
return 32 - __clz(x);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __fls(unsigned long x)
{
return fls(x) - 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int ffs(int x)
{
return fls(x & -x);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __ffs(unsigned long x)
{
return ffs(x) - 1;
}
# 1 "./include/asm-generic/bitops/fls64.h" 1
# 19 "./include/asm-generic/bitops/fls64.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int fls64(__u64 x)
{
__u32 h = x >> 32;
if (h)
return fls(h) + 32;
return fls(x);
}
# 314 "./arch/arm/include/asm/bitops.h" 2
# 1 "./include/asm-generic/bitops/sched.h" 1
# 13 "./include/asm-generic/bitops/sched.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int sched_find_first_bit(const unsigned long *b)
{
if (b[0])
return __ffs(b[0]);
if (b[1])
return __ffs(b[1]) + 32;
if (b[2])
return __ffs(b[2]) + 64;
return __ffs(b[3]) + 96;
}
# 316 "./arch/arm/include/asm/bitops.h" 2
# 1 "./include/asm-generic/bitops/hweight.h" 1
# 1 "./include/asm-generic/bitops/arch_hweight.h" 1
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int __arch_hweight32(unsigned int w)
{
return __sw_hweight32(w);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int __arch_hweight16(unsigned int w)
{
return __sw_hweight16(w);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int __arch_hweight8(unsigned int w)
{
return __sw_hweight8(w);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __arch_hweight64(__u64 w)
{
return __sw_hweight64(w);
}
# 6 "./include/asm-generic/bitops/hweight.h" 2
# 1 "./include/asm-generic/bitops/const_hweight.h" 1
# 7 "./include/asm-generic/bitops/hweight.h" 2
# 317 "./arch/arm/include/asm/bitops.h" 2
# 1 "./include/asm-generic/bitops/lock.h" 1
# 318 "./arch/arm/include/asm/bitops.h" 2
# 341 "./arch/arm/include/asm/bitops.h"
# 1 "./include/asm-generic/bitops/le.h" 1
# 1 "./arch/arm/include/uapi/asm/byteorder.h" 1
# 22 "./arch/arm/include/uapi/asm/byteorder.h"
# 1 "./include/linux/byteorder/little_endian.h" 1
# 1 "./include/uapi/linux/byteorder/little_endian.h" 1
# 44 "./include/uapi/linux/byteorder/little_endian.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __le64 __cpu_to_le64p(const __u64 *p)
{
return ( __le64)*p;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u64 __le64_to_cpup(const __le64 *p)
{
return ( __u64)*p;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __le32 __cpu_to_le32p(const __u32 *p)
{
return ( __le32)*p;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u32 __le32_to_cpup(const __le32 *p)
{
return ( __u32)*p;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __le16 __cpu_to_le16p(const __u16 *p)
{
return ( __le16)*p;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u16 __le16_to_cpup(const __le16 *p)
{
return ( __u16)*p;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __be64 __cpu_to_be64p(const __u64 *p)
{
return ( __be64)__swab64p(p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u64 __be64_to_cpup(const __be64 *p)
{
return __swab64p((__u64 *)p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __be32 __cpu_to_be32p(const __u32 *p)
{
return ( __be32)__swab32p(p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u32 __be32_to_cpup(const __be32 *p)
{
return __swab32p((__u32 *)p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __be16 __cpu_to_be16p(const __u16 *p)
{
return ( __be16)__swab16p(p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) __u16 __be16_to_cpup(const __be16 *p)
{
return __swab16p((__u16 *)p);
}
# 6 "./include/linux/byteorder/little_endian.h" 2
# 1 "./include/linux/byteorder/generic.h" 1
# 144 "./include/linux/byteorder/generic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void le16_add_cpu(__le16 *var, u16 val)
{
*var = (( __le16)(__u16)((( __u16)(__le16)(*var)) + val));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void le32_add_cpu(__le32 *var, u32 val)
{
*var = (( __le32)(__u32)((( __u32)(__le32)(*var)) + val));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void le64_add_cpu(__le64 *var, u64 val)
{
*var = (( __le64)(__u64)((( __u64)(__le64)(*var)) + val));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void be16_add_cpu(__be16 *var, u16 val)
{
*var = (( __be16)(__builtin_constant_p((__u16)(((__builtin_constant_p((__u16)(( __u16)(__be16)(*var))) ? ((__u16)( (((__u16)(( __u16)(__be16)(*var)) & (__u16)0x00ffU) << 8) | (((__u16)(( __u16)(__be16)(*var)) & (__u16)0xff00U) >> 8))) : __fswab16(( __u16)(__be16)(*var))) + val))) ? ((__u16)( (((__u16)(((__builtin_constant_p((__u16)(( __u16)(__be16)(*var))) ? ((__u16)( (((__u16)(( __u16)(__be16)(*var)) & (__u16)0x00ffU) << 8) | (((__u16)(( __u16)(__be16)(*var)) & (__u16)0xff00U) >> 8))) : __fswab16(( __u16)(__be16)(*var))) + val)) & (__u16)0x00ffU) << 8) | (((__u16)(((__builtin_constant_p((__u16)(( __u16)(__be16)(*var))) ? ((__u16)( (((__u16)(( __u16)(__be16)(*var)) & (__u16)0x00ffU) << 8) | (((__u16)(( __u16)(__be16)(*var)) & (__u16)0xff00U) >> 8))) : __fswab16(( __u16)(__be16)(*var))) + val)) & (__u16)0xff00U) >> 8))) : __fswab16(((__builtin_constant_p((__u16)(( __u16)(__be16)(*var))) ? ((__u16)( (((__u16)(( __u16)(__be16)(*var)) & (__u16)0x00ffU) << 8) | (((__u16)(( __u16)(__be16)(*var)) & (__u16)0xff00U) >> 8))) : __fswab16(( __u16)(__be16)(*var))) + val))));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void be32_add_cpu(__be32 *var, u32 val)
{
*var = (( __be32)(__builtin_constant_p((__u32)(((__builtin_constant_p((__u32)(( __u32)(__be32)(*var))) ? ((__u32)( (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x000000ffUL) << 24) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x0000ff00UL) << 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0xff000000UL) >> 24))) : __fswab32(( __u32)(__be32)(*var))) + val))) ? ((__u32)( (((__u32)(((__builtin_constant_p((__u32)(( __u32)(__be32)(*var))) ? ((__u32)( (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x000000ffUL) << 24) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x0000ff00UL) << 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0xff000000UL) >> 24))) : __fswab32(( __u32)(__be32)(*var))) + val)) & (__u32)0x000000ffUL) << 24) | (((__u32)(((__builtin_constant_p((__u32)(( __u32)(__be32)(*var))) ? ((__u32)( (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x000000ffUL) << 24) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x0000ff00UL) << 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0xff000000UL) >> 24))) : __fswab32(( __u32)(__be32)(*var))) + val)) & (__u32)0x0000ff00UL) << 8) | (((__u32)(((__builtin_constant_p((__u32)(( __u32)(__be32)(*var))) ? ((__u32)( (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x000000ffUL) << 24) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x0000ff00UL) << 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0xff000000UL) >> 24))) : __fswab32(( __u32)(__be32)(*var))) + val)) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(((__builtin_constant_p((__u32)(( __u32)(__be32)(*var))) ? ((__u32)( (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x000000ffUL) << 24) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x0000ff00UL) << 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0xff000000UL) >> 24))) : __fswab32(( __u32)(__be32)(*var))) + val)) & (__u32)0xff000000UL) >> 24))) : __fswab32(((__builtin_constant_p((__u32)(( __u32)(__be32)(*var))) ? ((__u32)( (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x000000ffUL) << 24) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x0000ff00UL) << 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0x00ff0000UL) >> 8) | (((__u32)(( __u32)(__be32)(*var)) & (__u32)0xff000000UL) >> 24))) : __fswab32(( __u32)(__be32)(*var))) + val))));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void be64_add_cpu(__be64 *var, u64 val)
{
*var = (( __be64)(__builtin_constant_p((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val))) ? ((__u64)( (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(((__builtin_constant_p((__u64)(( __u64)(__be64)(*var))) ? ((__u64)( (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000000000ffULL) << 56) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000000000ff00ULL) << 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000000000ff0000ULL) << 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00000000ff000000ULL) << 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x000000ff00000000ULL) >> 8) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x0000ff0000000000ULL) >> 24) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0x00ff000000000000ULL) >> 40) | (((__u64)(( __u64)(__be64)(*var)) & (__u64)0xff00000000000000ULL) >> 56))) : __fswab64(( __u64)(__be64)(*var))) + val))));
}
# 12 "./include/linux/byteorder/little_endian.h" 2
# 23 "./arch/arm/include/uapi/asm/byteorder.h" 2
# 7 "./include/asm-generic/bitops/le.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long find_next_zero_bit_le(const void *addr,
unsigned long size, unsigned long offset)
{
return _find_next_zero_bit_le(addr,size,offset);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long find_next_bit_le(const void *addr,
unsigned long size, unsigned long offset)
{
return _find_next_bit_le(addr,size,offset);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long find_first_zero_bit_le(const void *addr,
unsigned long size)
{
return _find_first_zero_bit_le(addr,size);
}
# 53 "./include/asm-generic/bitops/le.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_bit_le(int nr, const void *addr)
{
return test_bit(nr ^ 0, addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_bit_le(int nr, void *addr)
{
_set_bit(nr ^ 0,addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_bit_le(int nr, void *addr)
{
_clear_bit(nr ^ 0,addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __set_bit_le(int nr, void *addr)
{
__set_bit(nr ^ 0, addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __clear_bit_le(int nr, void *addr)
{
__clear_bit(nr ^ 0, addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_and_set_bit_le(int nr, void *addr)
{
return _test_and_set_bit(nr ^ 0,addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_and_clear_bit_le(int nr, void *addr)
{
return _test_and_clear_bit(nr ^ 0,addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __test_and_set_bit_le(int nr, void *addr)
{
return __test_and_set_bit(nr ^ 0, addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __test_and_clear_bit_le(int nr, void *addr)
{
return __test_and_clear_bit(nr ^ 0, addr);
}
# 342 "./arch/arm/include/asm/bitops.h" 2
# 1 "./include/asm-generic/bitops/ext2-atomic-setbit.h" 1
# 347 "./arch/arm/include/asm/bitops.h" 2
# 20 "./include/linux/bitops.h" 2
# 43 "./include/linux/bitops.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_bitmask_order(unsigned int count)
{
int order;
order = fls(count);
return order;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long hweight_long(unsigned long w)
{
return sizeof(w) == 4 ? (__builtin_constant_p(w) ? ((((unsigned int) ((!!((w) & (1ULL << 0))) + (!!((w) & (1ULL << 1))) + (!!((w) & (1ULL << 2))) + (!!((w) & (1ULL << 3))) + (!!((w) & (1ULL << 4))) + (!!((w) & (1ULL << 5))) + (!!((w) & (1ULL << 6))) + (!!((w) & (1ULL << 7))))) + ((unsigned int) ((!!(((w) >> 8) & (1ULL << 0))) + (!!(((w) >> 8) & (1ULL << 1))) + (!!(((w) >> 8) & (1ULL << 2))) + (!!(((w) >> 8) & (1ULL << 3))) + (!!(((w) >> 8) & (1ULL << 4))) + (!!(((w) >> 8) & (1ULL << 5))) + (!!(((w) >> 8) & (1ULL << 6))) + (!!(((w) >> 8) & (1ULL << 7)))))) + (((unsigned int) ((!!(((w) >> 16) & (1ULL << 0))) + (!!(((w) >> 16) & (1ULL << 1))) + (!!(((w) >> 16) & (1ULL << 2))) + (!!(((w) >> 16) & (1ULL << 3))) + (!!(((w) >> 16) & (1ULL << 4))) + (!!(((w) >> 16) & (1ULL << 5))) + (!!(((w) >> 16) & (1ULL << 6))) + (!!(((w) >> 16) & (1ULL << 7))))) + ((unsigned int) ((!!((((w) >> 16) >> 8) & (1ULL << 0))) + (!!((((w) >> 16) >> 8) & (1ULL << 1))) + (!!((((w) >> 16) >> 8) & (1ULL << 2))) + (!!((((w) >> 16) >> 8) & (1ULL << 3))) + (!!((((w) >> 16) >> 8) & (1ULL << 4))) + (!!((((w) >> 16) >> 8) & (1ULL << 5))) + (!!((((w) >> 16) >> 8) & (1ULL << 6))) + (!!((((w) >> 16) >> 8) & (1ULL << 7))))))) : __arch_hweight32(w)) : (__builtin_constant_p((__u64)w) ? (((((unsigned int) ((!!(((__u64)w) & (1ULL << 0))) + (!!(((__u64)w) & (1ULL << 1))) + (!!(((__u64)w) & (1ULL << 2))) + (!!(((__u64)w) & (1ULL << 3))) + (!!(((__u64)w) & (1ULL << 4))) + (!!(((__u64)w) & (1ULL << 5))) + (!!(((__u64)w) & (1ULL << 6))) + (!!(((__u64)w) & (1ULL << 7))))) + ((unsigned int) ((!!((((__u64)w) >> 8) & (1ULL << 0))) + (!!((((__u64)w) >> 8) & (1ULL << 1))) + (!!((((__u64)w) >> 8) & (1ULL << 2))) + (!!((((__u64)w) >> 8) & (1ULL << 3))) + (!!((((__u64)w) >> 8) & (1ULL << 4))) + (!!((((__u64)w) >> 8) & (1ULL << 5))) + (!!((((__u64)w) >> 8) & (1ULL << 6))) + (!!((((__u64)w) >> 8) & (1ULL << 7)))))) + (((unsigned int) ((!!((((__u64)w) >> 16) & (1ULL << 0))) + (!!((((__u64)w) >> 16) & (1ULL << 1))) + (!!((((__u64)w) >> 16) & (1ULL << 2))) + (!!((((__u64)w) >> 16) & (1ULL << 3))) + (!!((((__u64)w) >> 16) & (1ULL << 4))) + (!!((((__u64)w) >> 16) & (1ULL << 5))) + (!!((((__u64)w) >> 16) & (1ULL << 6))) + (!!((((__u64)w) >> 16) & (1ULL << 7))))) + ((unsigned int) ((!!(((((__u64)w) >> 16) >> 8) & (1ULL << 0))) + (!!(((((__u64)w) >> 16) >> 8) & (1ULL << 1))) + (!!(((((__u64)w) >> 16) >> 8) & (1ULL << 2))) + (!!(((((__u64)w) >> 16) >> 8) & (1ULL << 3))) + (!!(((((__u64)w) >> 16) >> 8) & (1ULL << 4))) + (!!(((((__u64)w) >> 16) >> 8) & (1ULL << 5))) + (!!(((((__u64)w) >> 16) >> 8) & (1ULL << 6))) + (!!(((((__u64)w) >> 16) >> 8) & (1ULL << 7))))))) + ((((unsigned int) ((!!((((__u64)w) >> 32) & (1ULL << 0))) + (!!((((__u64)w) >> 32) & (1ULL << 1))) + (!!((((__u64)w) >> 32) & (1ULL << 2))) + (!!((((__u64)w) >> 32) & (1ULL << 3))) + (!!((((__u64)w) >> 32) & (1ULL << 4))) + (!!((((__u64)w) >> 32) & (1ULL << 5))) + (!!((((__u64)w) >> 32) & (1ULL << 6))) + (!!((((__u64)w) >> 32) & (1ULL << 7))))) + ((unsigned int) ((!!(((((__u64)w) >> 32) >> 8) & (1ULL << 0))) + (!!(((((__u64)w) >> 32) >> 8) & (1ULL << 1))) + (!!(((((__u64)w) >> 32) >> 8) & (1ULL << 2))) + (!!(((((__u64)w) >> 32) >> 8) & (1ULL << 3))) + (!!(((((__u64)w) >> 32) >> 8) & (1ULL << 4))) + (!!(((((__u64)w) >> 32) >> 8) & (1ULL << 5))) + (!!(((((__u64)w) >> 32) >> 8) & (1ULL << 6))) + (!!(((((__u64)w) >> 32) >> 8) & (1ULL << 7)))))) + (((unsigned int) ((!!(((((__u64)w) >> 32) >> 16) & (1ULL << 0))) + (!!(((((__u64)w) >> 32) >> 16) & (1ULL << 1))) + (!!(((((__u64)w) >> 32) >> 16) & (1ULL << 2))) + (!!(((((__u64)w) >> 32) >> 16) & (1ULL << 3))) + (!!(((((__u64)w) >> 32) >> 16) & (1ULL << 4))) + (!!(((((__u64)w) >> 32) >> 16) & (1ULL << 5))) + (!!(((((__u64)w) >> 32) >> 16) & (1ULL << 6))) + (!!(((((__u64)w) >> 32) >> 16) & (1ULL << 7))))) + ((unsigned int) ((!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 0))) + (!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 1))) + (!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 2))) + (!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 3))) + (!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 4))) + (!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 5))) + (!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 6))) + (!!((((((__u64)w) >> 32) >> 16) >> 8) & (1ULL << 7)))))))) : __arch_hweight64((__u64)w));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u64 rol64(__u64 word, unsigned int shift)
{
return (word << (shift & 63)) | (word >> ((-shift) & 63));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u64 ror64(__u64 word, unsigned int shift)
{
return (word >> (shift & 63)) | (word << ((-shift) & 63));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u32 rol32(__u32 word, unsigned int shift)
{
return (word << (shift & 31)) | (word >> ((-shift) & 31));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u32 ror32(__u32 word, unsigned int shift)
{
return (word >> (shift & 31)) | (word << ((-shift) & 31));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u16 rol16(__u16 word, unsigned int shift)
{
return (word << (shift & 15)) | (word >> ((-shift) & 15));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u16 ror16(__u16 word, unsigned int shift)
{
return (word >> (shift & 15)) | (word << ((-shift) & 15));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u8 rol8(__u8 word, unsigned int shift)
{
return (word << (shift & 7)) | (word >> ((-shift) & 7));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u8 ror8(__u8 word, unsigned int shift)
{
return (word >> (shift & 7)) | (word << ((-shift) & 7));
}
# 143 "./include/linux/bitops.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __s32 sign_extend32(__u32 value, int index)
{
__u8 shift = 31 - index;
return (__s32)(value << shift) >> shift;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __s64 sign_extend64(__u64 value, int index)
{
__u8 shift = 63 - index;
return (__s64)(value << shift) >> shift;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned fls_long(unsigned long l)
{
if (sizeof(l) == 4)
return fls(l);
return fls64(l);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_count_order(unsigned int count)
{
int order;
order = fls(count) - 1;
if (count & (count - 1))
order++;
return order;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_count_order_long(unsigned long l)
{
if (l == 0UL)
return -1;
else if (l & (l - 1UL))
return (int)fls_long(l);
else
return (int)fls_long(l) - 1;
}
# 201 "./include/linux/bitops.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __ffs64(u64 word)
{
if (((u32)word) == 0UL)
return __ffs((u32)(word >> 32)) + 32;
return __ffs((unsigned long)word);
}
# 253 "./include/linux/bitops.h"
extern unsigned long find_last_bit(const unsigned long *addr,
unsigned long size);
# 12 "./include/linux/kernel.h" 2
# 1 "./include/linux/log2.h" 1
# 25 "./include/linux/log2.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((const))
int __ilog2_u32(u32 n)
{
return fls(n) - 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((const))
int __ilog2_u64(u64 n)
{
return fls64(n) - 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((const))
bool is_power_of_2(unsigned long n)
{
return (n != 0 && ((n & (n - 1)) == 0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((const))
unsigned long __roundup_pow_of_two(unsigned long n)
{
return 1UL << fls_long(n - 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((const))
unsigned long __rounddown_pow_of_two(unsigned long n)
{
return 1UL << (fls_long(n) - 1);
}
# 210 "./include/linux/log2.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__))
int __order_base_2(unsigned long n)
{
return n > 1 ? ( __builtin_constant_p(n - 1) ? ( __builtin_constant_p(n - 1) ? ( (n - 1) < 2 ? 0 : (n - 1) & (1ULL << 63) ? 63 : (n - 1) & (1ULL << 62) ? 62 : (n - 1) & (1ULL << 61) ? 61 : (n - 1) & (1ULL << 60) ? 60 : (n - 1) & (1ULL << 59) ? 59 : (n - 1) & (1ULL << 58) ? 58 : (n - 1) & (1ULL << 57) ? 57 : (n - 1) & (1ULL << 56) ? 56 : (n - 1) & (1ULL << 55) ? 55 : (n - 1) & (1ULL << 54) ? 54 : (n - 1) & (1ULL << 53) ? 53 : (n - 1) & (1ULL << 52) ? 52 : (n - 1) & (1ULL << 51) ? 51 : (n - 1) & (1ULL << 50) ? 50 : (n - 1) & (1ULL << 49) ? 49 : (n - 1) & (1ULL << 48) ? 48 : (n - 1) & (1ULL << 47) ? 47 : (n - 1) & (1ULL << 46) ? 46 : (n - 1) & (1ULL << 45) ? 45 : (n - 1) & (1ULL << 44) ? 44 : (n - 1) & (1ULL << 43) ? 43 : (n - 1) & (1ULL << 42) ? 42 : (n - 1) & (1ULL << 41) ? 41 : (n - 1) & (1ULL << 40) ? 40 : (n - 1) & (1ULL << 39) ? 39 : (n - 1) & (1ULL << 38) ? 38 : (n - 1) & (1ULL << 37) ? 37 : (n - 1) & (1ULL << 36) ? 36 : (n - 1) & (1ULL << 35) ? 35 : (n - 1) & (1ULL << 34) ? 34 : (n - 1) & (1ULL << 33) ? 33 : (n - 1) & (1ULL << 32) ? 32 : (n - 1) & (1ULL << 31) ? 31 : (n - 1) & (1ULL << 30) ? 30 : (n - 1) & (1ULL << 29) ? 29 : (n - 1) & (1ULL << 28) ? 28 : (n - 1) & (1ULL << 27) ? 27 : (n - 1) & (1ULL << 26) ? 26 : (n - 1) & (1ULL << 25) ? 25 : (n - 1) & (1ULL << 24) ? 24 : (n - 1) & (1ULL << 23) ? 23 : (n - 1) & (1ULL << 22) ? 22 : (n - 1) & (1ULL << 21) ? 21 : (n - 1) & (1ULL << 20) ? 20 : (n - 1) & (1ULL << 19) ? 19 : (n - 1) & (1ULL << 18) ? 18 : (n - 1) & (1ULL << 17) ? 17 : (n - 1) & (1ULL << 16) ? 16 : (n - 1) & (1ULL << 15) ? 15 : (n - 1) & (1ULL << 14) ? 14 : (n - 1) & (1ULL << 13) ? 13 : (n - 1) & (1ULL << 12) ? 12 : (n - 1) & (1ULL << 11) ? 11 : (n - 1) & (1ULL << 10) ? 10 : (n - 1) & (1ULL << 9) ? 9 : (n - 1) & (1ULL << 8) ? 8 : (n - 1) & (1ULL << 7) ? 7 : (n - 1) & (1ULL << 6) ? 6 : (n - 1) & (1ULL << 5) ? 5 : (n - 1) & (1ULL << 4) ? 4 : (n - 1) & (1ULL << 3) ? 3 : (n - 1) & (1ULL << 2) ? 2 : 1) : -1) : (sizeof(n - 1) <= 4) ? __ilog2_u32(n - 1) : __ilog2_u64(n - 1) ) + 1 : 0;
}
# 13 "./include/linux/kernel.h" 2
# 1 "./include/linux/printk.h" 1
# 1 "./include/linux/init.h" 1
# 118 "./include/linux/init.h"
typedef int (*initcall_t)(void);
typedef void (*exitcall_t)(void);
extern initcall_t __con_initcall_start[], __con_initcall_end[];
extern initcall_t __security_initcall_start[], __security_initcall_end[];
typedef void (*ctor_fn_t)(void);
extern int do_one_initcall(initcall_t fn);
extern char __attribute__ ((__section__(".init.data"))) boot_command_line[];
extern char *saved_command_line;
extern unsigned int reset_devices;
void setup_arch(char **);
void prepare_namespace(void);
void __attribute__ ((__section__(".init.text"))) load_default_modules(void);
int __attribute__ ((__section__(".init.text"))) init_rootfs(void);
extern bool rodata_enabled;
void mark_rodata_ro(void);
extern void (*late_time_init)(void);
extern bool initcall_debug;
# 237 "./include/linux/init.h"
struct user_initcall_param {
char *name;
initcall_t fn;
};
# 254 "./include/linux/init.h"
struct obs_kernel_param {
const char *str;
int (*setup_func)(char *);
int early;
};
# 303 "./include/linux/init.h"
void __attribute__ ((__section__(".init.text"))) parse_early_param(void);
void __attribute__ ((__section__(".init.text"))) parse_early_options(char *cmdline);
# 7 "./include/linux/printk.h" 2
# 1 "./include/linux/kern_levels.h" 1
# 8 "./include/linux/printk.h" 2
# 1 "./include/linux/cache.h" 1
# 1 "./include/uapi/linux/kernel.h" 1
# 1 "./include/uapi/linux/sysinfo.h" 1
struct sysinfo {
__kernel_long_t uptime;
__kernel_ulong_t loads[3];
__kernel_ulong_t totalram;
__kernel_ulong_t freeram;
__kernel_ulong_t sharedram;
__kernel_ulong_t bufferram;
__kernel_ulong_t totalswap;
__kernel_ulong_t freeswap;
__u16 procs;
__u16 pad;
__kernel_ulong_t totalhigh;
__kernel_ulong_t freehigh;
__u32 mem_unit;
char _f[20-2*sizeof(__kernel_ulong_t)-sizeof(__u32)];
};
# 6 "./include/uapi/linux/kernel.h" 2
# 6 "./include/linux/cache.h" 2
# 1 "./arch/arm/include/asm/cache.h" 1
# 7 "./include/linux/cache.h" 2
# 10 "./include/linux/printk.h" 2
extern const char linux_banner[];
extern const char linux_proc_banner[];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int printk_get_level(const char *buffer)
{
if (buffer[0] == '\001' && buffer[1]) {
switch (buffer[1]) {
case '0' ... '7':
case 'd':
case 'c':
return buffer[1];
}
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const char *printk_skip_level(const char *buffer)
{
if (printk_get_level(buffer))
return buffer + 2;
return buffer;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const char *printk_skip_headers(const char *buffer)
{
while (printk_get_level(buffer))
buffer = printk_skip_level(buffer);
return buffer;
}
# 63 "./include/linux/printk.h"
extern int console_printk[];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void console_silent(void)
{
(console_printk[0]) = 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void console_verbose(void)
{
if (1 )
(console_printk[0]) = 15;
}
extern char devkmsg_log_str[];
struct ctl_table;
struct va_format {
const char *fmt;
va_list *va;
};
# 143 "./include/linux/printk.h"
extern __attribute__((format(printf, 1, 2)))
void early_printk(const char *fmt, ...);
extern void printk_nmi_enter(void);
extern void printk_nmi_exit(void);
extern void printk_nmi_direct_enter(void);
extern void printk_nmi_direct_exit(void);
# 163 "./include/linux/printk.h"
__attribute__((format(printf, 5, 0)))
int vprintk_emit(int facility, int level,
const char *dict, size_t dictlen,
const char *fmt, va_list args);
__attribute__((format(printf, 1, 0)))
int vprintk(const char *fmt, va_list args);
__attribute__((format(printf, 5, 6)))
int printk_emit(int facility, int level,
const char *dict, size_t dictlen,
const char *fmt, ...);
__attribute__((format(printf, 1, 2)))
int printk(const char *fmt, ...);
__attribute__((format(printf, 1, 2))) int printk_deferred(const char *fmt, ...);
extern int __printk_ratelimit(const char *func);
extern bool printk_timed_ratelimit(unsigned long *caller_jiffies,
unsigned int interval_msec);
extern int printk_delay_msec;
extern int dmesg_restrict;
extern int kptr_restrict;
extern int
devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write, void *buf,
size_t *lenp, loff_t *ppos);
extern void wake_up_klogd(void);
char *log_buf_addr_get(void);
u32 log_buf_len_get(void);
void log_buf_vmcoreinfo_setup(void);
void __attribute__ ((__section__(".init.text"))) setup_log_buf(int early);
__attribute__((format(printf, 1, 2))) void dump_stack_set_arch_desc(const char *fmt, ...);
void dump_stack_print_info(const char *log_lvl);
void show_regs_print_info(const char *log_lvl);
extern void printk_safe_init(void);
extern void printk_safe_flush(void);
extern void printk_safe_flush_on_panic(void);
# 287 "./include/linux/printk.h"
extern void dump_stack(void) ;
# 334 "./include/linux/printk.h"
# 1 "./include/linux/dynamic_debug.h" 1
# 14 "./include/linux/dynamic_debug.h"
struct _ddebug {
const char *modname;
const char *function;
const char *filename;
const char *format;
unsigned int lineno:18;
# 40 "./include/linux/dynamic_debug.h"
unsigned int flags:8;
} __attribute__((aligned(8)));
int ddebug_add_module(struct _ddebug *tab, unsigned int n,
const char *modname);
extern int ddebug_remove_module(const char *mod_name);
extern __attribute__((format(printf, 2, 3)))
void __dynamic_pr_debug(struct _ddebug *descriptor, const char *fmt, ...);
extern int ddebug_dyndbg_module_param_cb(char *param, char *val,
const char *modname);
struct device;
extern __attribute__((format(printf, 3, 4)))
void __dynamic_dev_dbg(struct _ddebug *descriptor, const struct device *dev,
const char *fmt, ...);
struct net_device;
extern __attribute__((format(printf, 3, 4)))
void __dynamic_netdev_dbg(struct _ddebug *descriptor,
const struct net_device *dev,
const char *fmt, ...);
# 335 "./include/linux/printk.h" 2
# 479 "./include/linux/printk.h"
extern const struct file_operations kmsg_fops;
enum {
DUMP_PREFIX_NONE,
DUMP_PREFIX_ADDRESS,
DUMP_PREFIX_OFFSET
};
extern int hex_dump_to_buffer(const void *buf, size_t len, int rowsize,
int groupsize, char *linebuf, size_t linebuflen,
bool ascii);
extern void print_hex_dump(const char *level, const char *prefix_str,
int prefix_type, int rowsize, int groupsize,
const void *buf, size_t len, bool ascii);
# 538 "./include/linux/printk.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dump_ddr_reg(void)
{
}
# 15 "./include/linux/kernel.h" 2
# 1 "./include/linux/build_bug.h" 1
# 16 "./include/linux/kernel.h" 2
# 178 "./include/linux/kernel.h"
# 1 "./arch/arm/include/asm/div64.h" 1
# 1 "./arch/arm/include/asm/div64_443.h" 1
# 1 "./arch/arm/include/asm/compiler.h" 1
# 6 "./arch/arm/include/asm/div64_443.h" 2
# 63 "./arch/arm/include/asm/div64_443.h"
# 1 "./arch/arm/include/asm/bug.h" 1
# 64 "./arch/arm/include/asm/div64_443.h" 2
# 4 "./arch/arm/include/asm/div64.h" 2
# 179 "./include/linux/kernel.h" 2
# 207 "./include/linux/kernel.h"
struct completion;
struct pt_regs;
struct user;
# 235 "./include/linux/kernel.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ___might_sleep(const char *file, int line,
int preempt_offset) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __might_sleep(const char *file, int line,
int preempt_offset) { }
# 282 "./include/linux/kernel.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 reciprocal_scale(u32 val, u32 ep_ro)
{
return (u32)(((u64) val * ep_ro) >> 32);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void might_fault(void) { }
extern struct atomic_notifier_head panic_notifier_list;
extern long (*panic_blink)(int state);
__attribute__((format(printf, 1, 2)))
void panic(const char *fmt, ...) __attribute__((noreturn)) ;
void nmi_panic(struct pt_regs *regs, const char *msg);
extern void oops_enter(void);
extern void oops_exit(void);
void print_oops_end_marker(void);
extern int oops_may_print(void);
void do_exit(long error_code) __attribute__((noreturn));
void complete_and_exit(struct completion *, long) __attribute__((noreturn));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void refcount_error_report(struct pt_regs *regs, const char *err)
{ }
int __attribute__((warn_unused_result)) _kstrtoul(const char *s, unsigned int base, unsigned long *res);
int __attribute__((warn_unused_result)) _kstrtol(const char *s, unsigned int base, long *res);
int __attribute__((warn_unused_result)) kstrtoull(const char *s, unsigned int base, unsigned long long *res);
int __attribute__((warn_unused_result)) kstrtoll(const char *s, unsigned int base, long long *res);
# 337 "./include/linux/kernel.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtoul(const char *s, unsigned int base, unsigned long *res)
{
if (sizeof(unsigned long) == sizeof(unsigned long long) &&
__alignof__(unsigned long) == __alignof__(unsigned long long))
return kstrtoull(s, base, (unsigned long long *)res);
else
return _kstrtoul(s, base, res);
}
# 366 "./include/linux/kernel.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtol(const char *s, unsigned int base, long *res)
{
if (sizeof(long) == sizeof(long long) &&
__alignof__(long) == __alignof__(long long))
return kstrtoll(s, base, (long long *)res);
else
return _kstrtol(s, base, res);
}
int __attribute__((warn_unused_result)) kstrtouint(const char *s, unsigned int base, unsigned int *res);
int __attribute__((warn_unused_result)) kstrtoint(const char *s, unsigned int base, int *res);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtou64(const char *s, unsigned int base, u64 *res)
{
return kstrtoull(s, base, res);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtos64(const char *s, unsigned int base, s64 *res)
{
return kstrtoll(s, base, res);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtou32(const char *s, unsigned int base, u32 *res)
{
return kstrtouint(s, base, res);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtos32(const char *s, unsigned int base, s32 *res)
{
return kstrtoint(s, base, res);
}
int __attribute__((warn_unused_result)) kstrtou16(const char *s, unsigned int base, u16 *res);
int __attribute__((warn_unused_result)) kstrtos16(const char *s, unsigned int base, s16 *res);
int __attribute__((warn_unused_result)) kstrtou8(const char *s, unsigned int base, u8 *res);
int __attribute__((warn_unused_result)) kstrtos8(const char *s, unsigned int base, s8 *res);
int __attribute__((warn_unused_result)) kstrtobool(const char *s, bool *res);
int __attribute__((warn_unused_result)) kstrtoull_from_user(const char *s, size_t count, unsigned int base, unsigned long long *res);
int __attribute__((warn_unused_result)) kstrtoll_from_user(const char *s, size_t count, unsigned int base, long long *res);
int __attribute__((warn_unused_result)) kstrtoul_from_user(const char *s, size_t count, unsigned int base, unsigned long *res);
int __attribute__((warn_unused_result)) kstrtol_from_user(const char *s, size_t count, unsigned int base, long *res);
int __attribute__((warn_unused_result)) kstrtouint_from_user(const char *s, size_t count, unsigned int base, unsigned int *res);
int __attribute__((warn_unused_result)) kstrtoint_from_user(const char *s, size_t count, unsigned int base, int *res);
int __attribute__((warn_unused_result)) kstrtou16_from_user(const char *s, size_t count, unsigned int base, u16 *res);
int __attribute__((warn_unused_result)) kstrtos16_from_user(const char *s, size_t count, unsigned int base, s16 *res);
int __attribute__((warn_unused_result)) kstrtou8_from_user(const char *s, size_t count, unsigned int base, u8 *res);
int __attribute__((warn_unused_result)) kstrtos8_from_user(const char *s, size_t count, unsigned int base, s8 *res);
int __attribute__((warn_unused_result)) kstrtobool_from_user(const char *s, size_t count, bool *res);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtou64_from_user(const char *s, size_t count, unsigned int base, u64 *res)
{
return kstrtoull_from_user(s, count, base, res);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtos64_from_user(const char *s, size_t count, unsigned int base, s64 *res)
{
return kstrtoll_from_user(s, count, base, res);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtou32_from_user(const char *s, size_t count, unsigned int base, u32 *res)
{
return kstrtouint_from_user(s, count, base, res);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kstrtos32_from_user(const char *s, size_t count, unsigned int base, s32 *res)
{
return kstrtoint_from_user(s, count, base, res);
}
extern unsigned long simple_strtoul(const char *,char **,unsigned int);
extern long simple_strtol(const char *,char **,unsigned int);
extern unsigned long long simple_strtoull(const char *,char **,unsigned int);
extern long long simple_strtoll(const char *,char **,unsigned int);
extern int num_to_str(char *buf, int size, unsigned long long num);
extern __attribute__((format(printf, 2, 3))) int sprintf(char *buf, const char * fmt, ...);
extern __attribute__((format(printf, 2, 0))) int vsprintf(char *buf, const char *, va_list);
extern __attribute__((format(printf, 3, 4)))
int snprintf(char *buf, size_t size, const char *fmt, ...);
extern __attribute__((format(printf, 3, 0)))
int vsnprintf(char *buf, size_t size, const char *fmt, va_list args);
extern __attribute__((format(printf, 3, 4)))
int scnprintf(char *buf, size_t size, const char *fmt, ...);
extern __attribute__((format(printf, 3, 0)))
int vscnprintf(char *buf, size_t size, const char *fmt, va_list args);
extern __attribute__((format(printf, 2, 3))) __attribute__((__malloc__))
char *kasprintf(gfp_t gfp, const char *fmt, ...);
extern __attribute__((format(printf, 2, 0))) __attribute__((__malloc__))
char *kvasprintf(gfp_t gfp, const char *fmt, va_list args);
extern __attribute__((format(printf, 2, 0)))
const char *kvasprintf_const(gfp_t gfp, const char *fmt, va_list args);
extern __attribute__((format(scanf, 2, 3)))
int sscanf(const char *, const char *, ...);
extern __attribute__((format(scanf, 2, 0)))
int vsscanf(const char *, const char *, va_list);
extern int get_option(char **str, int *pint);
extern char *get_options(const char *str, int nints, int *ints);
extern unsigned long long memparse(const char *ptr, char **retptr);
extern bool parse_option_str(const char *str, const char *option);
extern char *next_arg(char *args, char **param, char **val);
extern int core_kernel_text(unsigned long addr);
extern int core_kernel_data(unsigned long addr);
extern int __kernel_text_address(unsigned long addr);
extern int kernel_text_address(unsigned long addr);
extern int func_ptr_is_kernel_text(void *ptr);
unsigned long int_sqrt(unsigned long);
extern void bust_spinlocks(int yes);
extern int oops_in_progress;
extern int panic_timeout;
extern int panic_on_oops;
extern int panic_on_unrecovered_nmi;
extern int panic_on_io_nmi;
extern int panic_on_warn;
extern int sysctl_panic_on_rcu_stall;
extern int sysctl_panic_on_stackoverflow;
extern bool crash_kexec_post_notifiers;
extern atomic_t panic_cpu;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_arch_panic_timeout(int timeout, int arch_default_timeout)
{
if (panic_timeout == arch_default_timeout)
panic_timeout = timeout;
}
extern const char *print_tainted(void);
enum lockdep_ok {
LOCKDEP_STILL_OK,
LOCKDEP_NOW_UNRELIABLE
};
extern void add_taint(unsigned flag, enum lockdep_ok);
extern int test_taint(unsigned flag);
extern unsigned long get_taint(void);
extern int root_mountflags;
extern bool early_boot_irqs_disabled;
extern enum system_states {
SYSTEM_BOOTING,
SYSTEM_SCHEDULING,
SYSTEM_RUNNING,
SYSTEM_HALT,
SYSTEM_POWER_OFF,
SYSTEM_RESTART,
} system_state;
# 559 "./include/linux/kernel.h"
struct taint_flag {
char c_true;
char c_false;
bool module;
};
extern const struct taint_flag taint_flags[16];
extern const char hex_asc[];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) char *hex_byte_pack(char *buf, u8 byte)
{
*buf++ = hex_asc[((byte) & 0xf0) >> 4];
*buf++ = hex_asc[((byte) & 0x0f)];
return buf;
}
extern const char hex_asc_upper[];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) char *hex_byte_pack_upper(char *buf, u8 byte)
{
*buf++ = hex_asc_upper[((byte) & 0xf0) >> 4];
*buf++ = hex_asc_upper[((byte) & 0x0f)];
return buf;
}
extern int hex_to_bin(char ch);
extern int __attribute__((warn_unused_result)) hex2bin(u8 *dst, const char *src, size_t count);
extern char *bin2hex(char *dst, const void *src, size_t count);
bool mac_pton(const char *s, u8 *mac);
# 615 "./include/linux/kernel.h"
enum ftrace_dump_mode {
DUMP_NONE,
DUMP_ALL,
DUMP_ORIG,
};
void tracing_on(void);
void tracing_off(void);
int tracing_is_on(void);
void tracing_snapshot(void);
void tracing_snapshot_alloc(void);
extern void tracing_start(void);
extern void tracing_stop(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((format(printf, 1, 2)))
void ____trace_printk_check_format(const char *fmt, ...)
{
}
# 694 "./include/linux/kernel.h"
extern __attribute__((format(printf, 2, 3)))
int __trace_bprintk(unsigned long ip, const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
int __trace_printk(unsigned long ip, const char *fmt, ...);
# 735 "./include/linux/kernel.h"
extern int __trace_bputs(unsigned long ip, const char *str);
extern int __trace_puts(unsigned long ip, const char *str, int size);
extern void trace_dump_stack(int skip);
# 757 "./include/linux/kernel.h"
extern __attribute__((format(printf, 2, 0))) int
__ftrace_vbprintk(unsigned long ip, const char *fmt, va_list ap);
extern __attribute__((format(printf, 2, 0))) int
__ftrace_vprintk(unsigned long ip, const char *fmt, va_list ap);
extern void ftrace_dump(enum ftrace_dump_mode oops_dump_mode);
# 961 "./include/linux/kernel.h"
extern unsigned int *sched_log_buf_head;
extern unsigned int *sched_log_buf_tail;
extern unsigned int *sched_log_buf_ptr;
extern unsigned int sched_log_flag;
extern unsigned int sched_log_time_scale;
extern unsigned int sched_log_start_time;
extern unsigned int log_get_time_stamp(void);
extern void log_sched(int cpu, int pid, int type);
extern void log_intr_enter(int cpu, int irq);
extern void log_intr_exit(int cpu, int irq);
# 17 "./include/asm-generic/bug.h" 2
struct bug_entry {
unsigned long bug_addr;
const char *file;
unsigned short line;
unsigned short flags;
};
# 81 "./include/asm-generic/bug.h"
extern __attribute__((format(printf, 3, 4)))
void warn_slowpath_fmt(const char *file, const int line,
const char *fmt, ...);
extern __attribute__((format(printf, 4, 5)))
void warn_slowpath_fmt_taint(const char *file, const int line, unsigned taint,
const char *fmt, ...);
extern void warn_slowpath_null(const char *file, const int line);
# 101 "./include/asm-generic/bug.h"
struct warn_args;
struct pt_regs;
void __warn(const char *file, int line, void *caller, unsigned taint,
struct pt_regs *regs, struct warn_args *args);
# 61 "./arch/arm/include/asm/bug.h" 2
struct pt_regs;
void die(const char *msg, struct pt_regs *regs, int err);
struct siginfo;
void arm_notify_die(const char *str, struct pt_regs *regs, struct siginfo *info,
unsigned long err, unsigned long trap);
# 77 "./arch/arm/include/asm/bug.h"
void hook_fault_code(int nr, int (*fn)(unsigned long, unsigned int,
struct pt_regs *),
int sig, int code, const char *name);
void hook_ifault_code(int nr, int (*fn)(unsigned long, unsigned int,
struct pt_regs *),
int sig, int code, const char *name);
extern void c_backtrace(unsigned long fp, int pmode);
struct mm_struct;
extern void show_pte(struct mm_struct *mm, unsigned long addr);
extern void __show_regs(struct pt_regs *);
# 6 "./include/linux/bug.h" 2
enum bug_trap_type {
BUG_TRAP_TYPE_NONE = 0,
BUG_TRAP_TYPE_WARN = 1,
BUG_TRAP_TYPE_BUG = 2,
};
struct pt_regs;
# 34 "./include/linux/bug.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_warning_bug(const struct bug_entry *bug)
{
return bug->flags & (1 << 0);
}
struct bug_entry *find_bug(unsigned long bugaddr);
enum bug_trap_type report_bug(unsigned long bug_addr, struct pt_regs *regs);
int is_valid_bugaddr(unsigned long addr);
# 65 "./include/linux/bug.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result)) bool check_data_corruption(bool v) { return v; }
# 13 "./include/linux/thread_info.h" 2
# 1 "./include/linux/restart_block.h" 1
# 11 "./include/linux/restart_block.h"
struct timespec;
struct compat_timespec;
struct pollfd;
enum timespec_type {
TT_NONE = 0,
TT_NATIVE = 1,
};
struct restart_block {
long (*fn)(struct restart_block *);
union {
struct {
u32 *uaddr;
u32 val;
u32 flags;
u32 bitset;
u64 time;
u32 *uaddr2;
} futex;
struct {
clockid_t clockid;
enum timespec_type type;
union {
struct timespec *rmtp;
};
u64 expires;
} nanosleep;
struct {
struct pollfd *ufds;
int nfds;
int has_timeout;
unsigned long tv_sec;
unsigned long tv_nsec;
} poll;
};
};
extern long do_no_restart_syscall(struct restart_block *parm);
# 14 "./include/linux/thread_info.h" 2
# 31 "./include/linux/thread_info.h"
enum {
BAD_STACK = -1,
NOT_STACK = 0,
GOOD_FRAME,
GOOD_STACK,
};
# 1 "./arch/arm/include/asm/thread_info.h" 1
# 16 "./arch/arm/include/asm/thread_info.h"
# 1 "./arch/arm/include/asm/fpstate.h" 1
# 26 "./arch/arm/include/asm/fpstate.h"
struct vfp_hard_struct {
__u64 fpregs[32];
__u32 fpexc;
__u32 fpscr;
__u32 fpinst;
__u32 fpinst2;
__u32 cpu;
};
union vfp_state {
struct vfp_hard_struct hard;
};
extern void vfp_flush_thread(union vfp_state *);
extern void vfp_release_thread(union vfp_state *);
struct fp_hard_struct {
unsigned int save[35];
};
struct fp_soft_struct {
unsigned int save[35];
};
struct iwmmxt_struct {
unsigned int save[0x98 / sizeof(unsigned int)];
};
union fp_state {
struct fp_hard_struct hard;
struct fp_soft_struct soft;
};
struct crunch_state {
unsigned int mvdx[16][2];
unsigned int mvax[4][3];
unsigned int dspsc[2];
};
# 17 "./arch/arm/include/asm/thread_info.h" 2
# 1 "./arch/arm/include/asm/page.h" 1
# 26 "./arch/arm/include/asm/page.h"
# 1 "./arch/arm/include/asm/glue.h" 1
# 27 "./arch/arm/include/asm/page.h" 2
# 110 "./arch/arm/include/asm/page.h"
struct page;
struct vm_area_struct;
struct cpu_user_fns {
void (*cpu_clear_user_highpage)(struct page *page, unsigned long vaddr);
void (*cpu_copy_user_highpage)(struct page *to, struct page *from,
unsigned long vaddr, struct vm_area_struct *vma);
};
extern struct cpu_user_fns cpu_user;
# 143 "./arch/arm/include/asm/page.h"
extern void copy_page(void *to, const void *from);
# 152 "./arch/arm/include/asm/page.h"
# 1 "./arch/arm/include/asm/pgtable-2level-types.h" 1
# 24 "./arch/arm/include/asm/pgtable-2level-types.h"
typedef u32 pteval_t;
typedef u32 pmdval_t;
# 51 "./arch/arm/include/asm/pgtable-2level-types.h"
typedef pteval_t pte_t;
typedef pmdval_t pmd_t;
typedef pmdval_t pgd_t[2];
typedef pteval_t pgprot_t;
# 153 "./arch/arm/include/asm/page.h" 2
typedef struct page *pgtable_t;
extern int pfn_valid(unsigned long);
# 1 "./arch/arm/include/asm/memory.h" 1
# 17 "./arch/arm/include/asm/memory.h"
# 1 "./include/uapi/linux/const.h" 1
# 18 "./arch/arm/include/asm/memory.h" 2
# 1 "./include/linux/sizes.h" 1
# 20 "./arch/arm/include/asm/memory.h" 2
# 271 "./arch/arm/include/asm/memory.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) phys_addr_t __virt_to_phys_nodebug(unsigned long x)
{
return (phys_addr_t)x - (0xC0000000UL) + (0x00000000UL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __phys_to_virt(phys_addr_t x)
{
return x - (0x00000000UL) + (0xC0000000UL);
}
# 304 "./arch/arm/include/asm/memory.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) phys_addr_t virt_to_phys(const volatile void *x)
{
return __virt_to_phys_nodebug((unsigned long)(x));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *phys_to_virt(phys_addr_t x)
{
return (void *)__phys_to_virt(x);
}
# 323 "./arch/arm/include/asm/memory.h"
extern long long arch_phys_to_idmap_offset;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool arm_has_idmap_alias(void)
{
return 1 && arch_phys_to_idmap_offset != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long phys_to_idmap(phys_addr_t addr)
{
if (1 && arch_phys_to_idmap_offset) {
addr += arch_phys_to_idmap_offset;
if (addr > (u32)~0)
addr = ((u32)~0);
}
return addr;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) phys_addr_t idmap_to_phys(unsigned long idmap)
{
phys_addr_t addr = idmap;
if (1 && arch_phys_to_idmap_offset)
addr -= arch_phys_to_idmap_offset;
return addr;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __virt_to_idmap(unsigned long x)
{
return phys_to_idmap(__virt_to_phys_nodebug(x));
}
# 394 "./arch/arm/include/asm/memory.h"
# 1 "./include/asm-generic/memory_model.h" 1
# 1 "./include/linux/pfn.h" 1
# 13 "./include/linux/pfn.h"
typedef struct {
u64 val;
} pfn_t;
# 6 "./include/asm-generic/memory_model.h" 2
# 395 "./arch/arm/include/asm/memory.h" 2
# 164 "./arch/arm/include/asm/page.h" 2
# 1 "./include/asm-generic/getorder.h" 1
# 29 "./include/asm-generic/getorder.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((__const__)) int get_order(unsigned long size)
{
if (__builtin_constant_p(size)) {
if (!size)
return 32 - 12;
if (size < (1UL << 12))
return 0;
return ( __builtin_constant_p((size) - 1) ? ( __builtin_constant_p((size) - 1) ? ( ((size) - 1) < 2 ? 0 : ((size) - 1) & (1ULL << 63) ? 63 : ((size) - 1) & (1ULL << 62) ? 62 : ((size) - 1) & (1ULL << 61) ? 61 : ((size) - 1) & (1ULL << 60) ? 60 : ((size) - 1) & (1ULL << 59) ? 59 : ((size) - 1) & (1ULL << 58) ? 58 : ((size) - 1) & (1ULL << 57) ? 57 : ((size) - 1) & (1ULL << 56) ? 56 : ((size) - 1) & (1ULL << 55) ? 55 : ((size) - 1) & (1ULL << 54) ? 54 : ((size) - 1) & (1ULL << 53) ? 53 : ((size) - 1) & (1ULL << 52) ? 52 : ((size) - 1) & (1ULL << 51) ? 51 : ((size) - 1) & (1ULL << 50) ? 50 : ((size) - 1) & (1ULL << 49) ? 49 : ((size) - 1) & (1ULL << 48) ? 48 : ((size) - 1) & (1ULL << 47) ? 47 : ((size) - 1) & (1ULL << 46) ? 46 : ((size) - 1) & (1ULL << 45) ? 45 : ((size) - 1) & (1ULL << 44) ? 44 : ((size) - 1) & (1ULL << 43) ? 43 : ((size) - 1) & (1ULL << 42) ? 42 : ((size) - 1) & (1ULL << 41) ? 41 : ((size) - 1) & (1ULL << 40) ? 40 : ((size) - 1) & (1ULL << 39) ? 39 : ((size) - 1) & (1ULL << 38) ? 38 : ((size) - 1) & (1ULL << 37) ? 37 : ((size) - 1) & (1ULL << 36) ? 36 : ((size) - 1) & (1ULL << 35) ? 35 : ((size) - 1) & (1ULL << 34) ? 34 : ((size) - 1) & (1ULL << 33) ? 33 : ((size) - 1) & (1ULL << 32) ? 32 : ((size) - 1) & (1ULL << 31) ? 31 : ((size) - 1) & (1ULL << 30) ? 30 : ((size) - 1) & (1ULL << 29) ? 29 : ((size) - 1) & (1ULL << 28) ? 28 : ((size) - 1) & (1ULL << 27) ? 27 : ((size) - 1) & (1ULL << 26) ? 26 : ((size) - 1) & (1ULL << 25) ? 25 : ((size) - 1) & (1ULL << 24) ? 24 : ((size) - 1) & (1ULL << 23) ? 23 : ((size) - 1) & (1ULL << 22) ? 22 : ((size) - 1) & (1ULL << 21) ? 21 : ((size) - 1) & (1ULL << 20) ? 20 : ((size) - 1) & (1ULL << 19) ? 19 : ((size) - 1) & (1ULL << 18) ? 18 : ((size) - 1) & (1ULL << 17) ? 17 : ((size) - 1) & (1ULL << 16) ? 16 : ((size) - 1) & (1ULL << 15) ? 15 : ((size) - 1) & (1ULL << 14) ? 14 : ((size) - 1) & (1ULL << 13) ? 13 : ((size) - 1) & (1ULL << 12) ? 12 : ((size) - 1) & (1ULL << 11) ? 11 : ((size) - 1) & (1ULL << 10) ? 10 : ((size) - 1) & (1ULL << 9) ? 9 : ((size) - 1) & (1ULL << 8) ? 8 : ((size) - 1) & (1ULL << 7) ? 7 : ((size) - 1) & (1ULL << 6) ? 6 : ((size) - 1) & (1ULL << 5) ? 5 : ((size) - 1) & (1ULL << 4) ? 4 : ((size) - 1) & (1ULL << 3) ? 3 : ((size) - 1) & (1ULL << 2) ? 2 : 1) : -1) : (sizeof((size) - 1) <= 4) ? __ilog2_u32((size) - 1) : __ilog2_u64((size) - 1) ) - 12 + 1;
}
size--;
size >>= 12;
return fls(size);
}
# 172 "./arch/arm/include/asm/page.h" 2
# 18 "./arch/arm/include/asm/thread_info.h" 2
# 29 "./arch/arm/include/asm/thread_info.h"
struct task_struct;
typedef unsigned long mm_segment_t;
struct cpu_context_save {
__u32 r4;
__u32 r5;
__u32 r6;
__u32 r7;
__u32 r8;
__u32 r9;
__u32 sl;
__u32 fp;
__u32 sp;
__u32 pc;
__u32 extra[2];
};
struct thread_info {
unsigned long flags;
int preempt_count;
mm_segment_t addr_limit;
struct task_struct *task;
__u32 cpu;
__u32 cpu_domain;
struct cpu_context_save cpu_context;
__u32 syscall;
__u8 used_cp[16];
unsigned long tp_value[2];
union fp_state fpstate __attribute__((aligned(8)));
union vfp_state vfpstate;
unsigned long thumbee_state;
};
# 88 "./arch/arm/include/asm/thread_info.h"
register unsigned long current_stack_pointer asm ("sp");
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct thread_info *current_thread_info(void) __attribute__((__const__));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct thread_info *current_thread_info(void)
{
return (struct thread_info *)
(current_stack_pointer & ~((((1UL) << 12) << 1) - 1));
}
# 114 "./arch/arm/include/asm/thread_info.h"
extern void crunch_task_disable(struct thread_info *);
extern void crunch_task_copy(struct thread_info *, void *);
extern void crunch_task_restore(struct thread_info *, void *);
extern void crunch_task_release(struct thread_info *);
extern void iwmmxt_task_disable(struct thread_info *);
extern void iwmmxt_task_copy(struct thread_info *, void *);
extern void iwmmxt_task_restore(struct thread_info *, void *);
extern void iwmmxt_task_release(struct thread_info *);
extern void iwmmxt_task_switch(struct thread_info *);
extern void vfp_sync_hwstate(struct thread_info *);
extern void vfp_flush_hwstate(struct thread_info *);
struct user_vfp;
struct user_vfp_exc;
extern int vfp_preserve_user_clear_hwstate(struct user_vfp *,
struct user_vfp_exc *);
extern int vfp_restore_user_hwstate(struct user_vfp *,
struct user_vfp_exc *);
# 39 "./include/linux/thread_info.h" 2
# 53 "./include/linux/thread_info.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_ti_thread_flag(struct thread_info *ti, int flag)
{
_set_bit(flag,(unsigned long *)&ti->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_ti_thread_flag(struct thread_info *ti, int flag)
{
_clear_bit(flag,(unsigned long *)&ti->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void update_ti_thread_flag(struct thread_info *ti, int flag,
bool value)
{
if (value)
set_ti_thread_flag(ti, flag);
else
clear_ti_thread_flag(ti, flag);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_and_set_ti_thread_flag(struct thread_info *ti, int flag)
{
return _test_and_set_bit(flag,(unsigned long *)&ti->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_and_clear_ti_thread_flag(struct thread_info *ti, int flag)
{
return _test_and_clear_bit(flag,(unsigned long *)&ti->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_ti_thread_flag(struct thread_info *ti, int flag)
{
return test_bit(flag, (unsigned long *)&ti->flags);
}
# 103 "./include/linux/thread_info.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_within_stack_frames(const void * const stack,
const void * const stackend,
const void *obj, unsigned long len)
{
return 0;
}
extern void __check_object_size(const void *ptr, unsigned long n,
bool to_user);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void check_object_size(const void *ptr, unsigned long n,
bool to_user)
{
if (!__builtin_constant_p(n))
__check_object_size(ptr, n, to_user);
}
extern void
__bad_copy_from(void);
extern void
__bad_copy_to(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void copy_overflow(int size, unsigned long count)
{
({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_fmt("include/linux/thread_info.h", 134, "Buffer overflow detected (%d < %lu)!\n", size, count); __builtin_expect(!!(__ret_warn_on), 0); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) bool
check_copy_size(const void *addr, size_t bytes, bool is_source)
{
int sz = __builtin_object_size(addr, 0);
if (__builtin_expect(!!(sz >= 0 && sz < bytes), 0)) {
if (!__builtin_constant_p(bytes))
copy_overflow(sz, bytes);
else if (is_source)
__bad_copy_from();
else
__bad_copy_to();
return false;
}
check_object_size(addr, bytes, is_source);
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_setup_new_exec(void) { }
# 6 "./include/asm-generic/current.h" 2
# 2 "./arch/arm/include/generated/asm/current.h" 2
# 13 "./include/linux/sched.h" 2
# 1 "./include/linux/pid.h" 1
# 1 "./include/linux/rculist.h" 1
# 10 "./include/linux/rculist.h"
# 1 "./include/linux/list.h" 1
# 1 "./include/linux/poison.h" 1
# 8 "./include/linux/list.h" 2
# 26 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void INIT_LIST_HEAD(struct list_head *list)
{
({ union { typeof(list->next) __val; char __c[1]; } __u = { .__val = ( typeof(list->next)) (list) }; __write_once_size(&(list->next), __u.__c, sizeof(list->next)); __u.__val; });
list->prev = list;
}
extern bool __list_add_valid(struct list_head *new,
struct list_head *prev,
struct list_head *next);
extern bool __list_del_entry_valid(struct list_head *entry);
# 56 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __list_add(struct list_head *new,
struct list_head *prev,
struct list_head *next)
{
if (!__list_add_valid(new, prev, next))
return;
next->prev = new;
new->next = next;
new->prev = prev;
({ union { typeof(prev->next) __val; char __c[1]; } __u = { .__val = ( typeof(prev->next)) (new) }; __write_once_size(&(prev->next), __u.__c, sizeof(prev->next)); __u.__val; });
}
# 77 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_add(struct list_head *new, struct list_head *head)
{
__list_add(new, head, head->next);
}
# 91 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_add_tail(struct list_head *new, struct list_head *head)
{
__list_add(new, head->prev, head);
}
# 103 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __list_del(struct list_head * prev, struct list_head * next)
{
next->prev = prev;
({ union { typeof(prev->next) __val; char __c[1]; } __u = { .__val = ( typeof(prev->next)) (next) }; __write_once_size(&(prev->next), __u.__c, sizeof(prev->next)); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __list_del_entry(struct list_head *entry)
{
if (!__list_del_entry_valid(entry))
return;
__list_del(entry->prev, entry->next);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_del(struct list_head *entry)
{
__list_del_entry(entry);
entry->next = ((void *) 0x100 + 0);
entry->prev = ((void *) 0x200 + 0);
}
# 137 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_replace(struct list_head *old,
struct list_head *new)
{
new->next = old->next;
new->next->prev = new;
new->prev = old->prev;
new->prev->next = new;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_replace_init(struct list_head *old,
struct list_head *new)
{
list_replace(old, new);
INIT_LIST_HEAD(old);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_del_init(struct list_head *entry)
{
__list_del_entry(entry);
INIT_LIST_HEAD(entry);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_move(struct list_head *list, struct list_head *head)
{
__list_del_entry(list);
list_add(list, head);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_move_tail(struct list_head *list,
struct list_head *head)
{
__list_del_entry(list);
list_add_tail(list, head);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int list_is_last(const struct list_head *list,
const struct list_head *head)
{
return list->next == head;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int list_empty(const struct list_head *head)
{
return ({ union { typeof(head->next) __val; char __c[1]; } __u; if (1) __read_once_size(&(head->next), __u.__c, sizeof(head->next)); else __read_once_size_nocheck(&(head->next), __u.__c, sizeof(head->next)); do { } while (0); __u.__val; }) == head;
}
# 219 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int list_empty_careful(const struct list_head *head)
{
struct list_head *next = head->next;
return (next == head) && (next == head->prev);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_rotate_left(struct list_head *head)
{
struct list_head *first;
if (!list_empty(head)) {
first = head->next;
list_move_tail(first, head);
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int list_is_singular(const struct list_head *head)
{
return !list_empty(head) && (head->next == head->prev);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __list_cut_position(struct list_head *list,
struct list_head *head, struct list_head *entry)
{
struct list_head *new_first = entry->next;
list->next = head->next;
list->next->prev = list;
list->prev = entry;
entry->next = list;
head->next = new_first;
new_first->prev = head;
}
# 274 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_cut_position(struct list_head *list,
struct list_head *head, struct list_head *entry)
{
if (list_empty(head))
return;
if (list_is_singular(head) &&
(head->next != entry && head != entry))
return;
if (entry == head)
INIT_LIST_HEAD(list);
else
__list_cut_position(list, head, entry);
}
# 302 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_cut_before(struct list_head *list,
struct list_head *head,
struct list_head *entry)
{
if (head->next == entry) {
INIT_LIST_HEAD(list);
return;
}
list->next = head->next;
list->next->prev = list;
list->prev = entry->prev;
list->prev->next = list;
head->next = entry;
entry->prev = head;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __list_splice(const struct list_head *list,
struct list_head *prev,
struct list_head *next)
{
struct list_head *first = list->next;
struct list_head *last = list->prev;
first->prev = prev;
prev->next = first;
last->next = next;
next->prev = last;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_splice(const struct list_head *list,
struct list_head *head)
{
if (!list_empty(list))
__list_splice(list, head, head->next);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_splice_tail(struct list_head *list,
struct list_head *head)
{
if (!list_empty(list))
__list_splice(list, head->prev, head);
}
# 363 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_splice_init(struct list_head *list,
struct list_head *head)
{
if (!list_empty(list)) {
__list_splice(list, head, head->next);
INIT_LIST_HEAD(list);
}
}
# 380 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_splice_tail_init(struct list_head *list,
struct list_head *head)
{
if (!list_empty(list)) {
__list_splice(list, head->prev, head);
INIT_LIST_HEAD(list);
}
}
# 658 "./include/linux/list.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void INIT_HLIST_NODE(struct hlist_node *h)
{
h->next = ((void *)0);
h->pprev = ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hlist_unhashed(const struct hlist_node *h)
{
return !h->pprev;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hlist_empty(const struct hlist_head *h)
{
return !({ union { typeof(h->first) __val; char __c[1]; } __u; if (1) __read_once_size(&(h->first), __u.__c, sizeof(h->first)); else __read_once_size_nocheck(&(h->first), __u.__c, sizeof(h->first)); do { } while (0); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __hlist_del(struct hlist_node *n)
{
struct hlist_node *next = n->next;
struct hlist_node **pprev = n->pprev;
({ union { typeof(*pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*pprev)) (next) }; __write_once_size(&(*pprev), __u.__c, sizeof(*pprev)); __u.__val; });
if (next)
next->pprev = pprev;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_del(struct hlist_node *n)
{
__hlist_del(n);
n->next = ((void *) 0x100 + 0);
n->pprev = ((void *) 0x200 + 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_del_init(struct hlist_node *n)
{
if (!hlist_unhashed(n)) {
__hlist_del(n);
INIT_HLIST_NODE(n);
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_head(struct hlist_node *n, struct hlist_head *h)
{
struct hlist_node *first = h->first;
n->next = first;
if (first)
first->pprev = &n->next;
({ union { typeof(h->first) __val; char __c[1]; } __u = { .__val = ( typeof(h->first)) (n) }; __write_once_size(&(h->first), __u.__c, sizeof(h->first)); __u.__val; });
n->pprev = &h->first;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_before(struct hlist_node *n,
struct hlist_node *next)
{
n->pprev = next->pprev;
n->next = next;
next->pprev = &n->next;
({ union { typeof(*(n->pprev)) __val; char __c[1]; } __u = { .__val = ( typeof(*(n->pprev))) (n) }; __write_once_size(&(*(n->pprev)), __u.__c, sizeof(*(n->pprev))); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_behind(struct hlist_node *n,
struct hlist_node *prev)
{
n->next = prev->next;
({ union { typeof(prev->next) __val; char __c[1]; } __u = { .__val = ( typeof(prev->next)) (n) }; __write_once_size(&(prev->next), __u.__c, sizeof(prev->next)); __u.__val; });
n->pprev = &prev->next;
if (n->next)
n->next->pprev = &n->next;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_fake(struct hlist_node *n)
{
n->pprev = &n->next;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool hlist_fake(struct hlist_node *h)
{
return h->pprev == &h->next;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool
hlist_is_singular_node(struct hlist_node *n, struct hlist_head *h)
{
return !n->next && n->pprev == &h->first;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_move_list(struct hlist_head *old,
struct hlist_head *new)
{
new->first = old->first;
if (new->first)
new->first->pprev = &new->first;
old->first = ((void *)0);
}
# 11 "./include/linux/rculist.h" 2
# 1 "./include/linux/rcupdate.h" 1
# 38 "./include/linux/rcupdate.h"
# 1 "./include/linux/atomic.h" 1
# 1 "./arch/arm/include/asm/atomic.h" 1
# 15 "./arch/arm/include/asm/atomic.h"
# 1 "./include/linux/prefetch.h" 1
# 15 "./include/linux/prefetch.h"
# 1 "./arch/arm/include/asm/processor.h" 1
# 22 "./arch/arm/include/asm/processor.h"
# 1 "./arch/arm/include/asm/hw_breakpoint.h" 1
struct task_struct;
struct arch_hw_breakpoint_ctrl {
u32 __reserved : 9,
mismatch : 1,
: 9,
len : 8,
type : 2,
privilege : 2,
enabled : 1;
};
struct arch_hw_breakpoint {
u32 address;
u32 trigger;
struct arch_hw_breakpoint_ctrl step_ctrl;
struct arch_hw_breakpoint_ctrl ctrl;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 encode_ctrl_reg(struct arch_hw_breakpoint_ctrl ctrl)
{
return (ctrl.mismatch << 22) | (ctrl.len << 5) | (ctrl.type << 3) |
(ctrl.privilege << 1) | ctrl.enabled;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void decode_ctrl_reg(u32 reg,
struct arch_hw_breakpoint_ctrl *ctrl)
{
ctrl->enabled = reg & 0x1;
reg >>= 1;
ctrl->privilege = reg & 0x3;
reg >>= 2;
ctrl->type = reg & 0x3;
reg >>= 2;
ctrl->len = reg & 0xff;
reg >>= 17;
ctrl->mismatch = reg & 0x1;
}
# 114 "./arch/arm/include/asm/hw_breakpoint.h"
struct notifier_block;
struct perf_event;
struct pmu;
extern int arch_bp_generic_fields(struct arch_hw_breakpoint_ctrl ctrl,
int *gen_len, int *gen_type);
extern int arch_check_bp_in_kernelspace(struct perf_event *bp);
extern int arch_validate_hwbkpt_settings(struct perf_event *bp);
extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
unsigned long val, void *data);
extern u8 arch_get_debug_arch(void);
extern u8 arch_get_max_wp_len(void);
extern void clear_ptrace_hw_breakpoint(struct task_struct *tsk);
int arch_install_hw_breakpoint(struct perf_event *bp);
void arch_uninstall_hw_breakpoint(struct perf_event *bp);
void hw_breakpoint_pmu_read(struct perf_event *bp);
int hw_breakpoint_slots(int type);
# 23 "./arch/arm/include/asm/processor.h" 2
# 1 "./arch/arm/include/asm/unified.h" 1
# 26 "./arch/arm/include/asm/unified.h"
__asm__(".syntax unified");
# 26 "./arch/arm/include/asm/processor.h" 2
struct debug_info {
struct perf_event *hbp[(16 + 16)];
};
struct thread_struct {
unsigned long address;
unsigned long trap_no;
unsigned long error_code;
struct debug_info debug;
};
# 72 "./arch/arm/include/asm/processor.h"
struct task_struct;
extern void release_thread(struct task_struct *);
unsigned long get_wchan(struct task_struct *p);
# 112 "./arch/arm/include/asm/processor.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void prefetch(const void *ptr)
{
__asm__ __volatile__(
"pld\t%a0"
:: "p" (ptr));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void prefetchw(const void *ptr)
{
__asm__ __volatile__(
".arch_extension mp\n"
"9998: " "pldw" "\t%a0" "\n" " .pushsection \".alt.smp.init\", \"a\"\n" " .long 9998b\n" " " "pld" "\t%a0" "\n" " .popsection\n"
:: "p" (ptr));
}
# 16 "./include/linux/prefetch.h" 2
# 54 "./include/linux/prefetch.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void prefetch_range(void *addr, size_t len)
{
char *cp;
char *end = addr + len;
for (cp = addr; cp < end; cp += (4*(1 << 6)))
prefetch(cp);
}
# 16 "./arch/arm/include/asm/atomic.h" 2
# 1 "./arch/arm/include/asm/cmpxchg.h" 1
# 28 "./arch/arm/include/asm/cmpxchg.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __xchg(unsigned long x, volatile void *ptr, int size)
{
extern void __bad_xchg(volatile void *, int);
unsigned long ret;
unsigned int tmp;
prefetchw((const void *)ptr);
switch (size) {
case 1:
asm volatile("@ __xchg1\n"
"1: ldrexb %0, [%3]\n"
" strexb %1, %2, [%3]\n"
" teq %1, #0\n"
" bne 1b"
: "=&r" (ret), "=&r" (tmp)
: "r" (x), "r" (ptr)
: "memory", "cc");
break;
case 2:
asm volatile("@ __xchg2\n"
"1: ldrexh %0, [%3]\n"
" strexh %1, %2, [%3]\n"
" teq %1, #0\n"
" bne 1b"
: "=&r" (ret), "=&r" (tmp)
: "r" (x), "r" (ptr)
: "memory", "cc");
break;
case 4:
asm volatile("@ __xchg4\n"
"1: ldrex %0, [%3]\n"
" strex %1, %2, [%3]\n"
" teq %1, #0\n"
" bne 1b"
: "=&r" (ret), "=&r" (tmp)
: "r" (x), "r" (ptr)
: "memory", "cc");
break;
# 108 "./arch/arm/include/asm/cmpxchg.h"
default:
__bad_xchg(ptr, size), ret = 0;
break;
}
return ret;
}
# 1 "./include/asm-generic/cmpxchg-local.h" 1
extern unsigned long wrong_size_cmpxchg(volatile void *ptr)
__attribute__((noreturn));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __cmpxchg_local_generic(volatile void *ptr,
unsigned long old, unsigned long new, int size)
{
unsigned long flags, prev;
if (size == 8 && sizeof(unsigned long) != 8)
wrong_size_cmpxchg(ptr);
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
switch (size) {
case 1: prev = *(u8 *)ptr;
if (prev == old)
*(u8 *)ptr = (u8)new;
break;
case 2: prev = *(u16 *)ptr;
if (prev == old)
*(u16 *)ptr = (u16)new;
break;
case 4: prev = *(u32 *)ptr;
if (prev == old)
*(u32 *)ptr = (u32)new;
break;
case 8: prev = *(u64 *)ptr;
if (prev == old)
*(u64 *)ptr = (u64)new;
break;
default:
wrong_size_cmpxchg(ptr);
}
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
return prev;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 __cmpxchg64_local_generic(volatile void *ptr,
u64 old, u64 new)
{
u64 prev;
unsigned long flags;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0);
prev = *(u64 *)ptr;
if (prev == old)
*(u64 *)ptr = new;
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0);
return prev;
}
# 123 "./arch/arm/include/asm/cmpxchg.h" 2
# 150 "./arch/arm/include/asm/cmpxchg.h"
extern void __bad_cmpxchg(volatile void *ptr, int size);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
unsigned long new, int size)
{
unsigned long oldval, res;
prefetchw((const void *)ptr);
switch (size) {
case 1:
do {
asm volatile("@ __cmpxchg1\n"
" ldrexb %1, [%2]\n"
" mov %0, #0\n"
" teq %1, %3\n"
" strexbeq %0, %4, [%2]\n"
: "=&r" (res), "=&r" (oldval)
: "r" (ptr), "Ir" (old), "r" (new)
: "memory", "cc");
} while (res);
break;
case 2:
do {
asm volatile("@ __cmpxchg1\n"
" ldrexh %1, [%2]\n"
" mov %0, #0\n"
" teq %1, %3\n"
" strexheq %0, %4, [%2]\n"
: "=&r" (res), "=&r" (oldval)
: "r" (ptr), "Ir" (old), "r" (new)
: "memory", "cc");
} while (res);
break;
case 4:
do {
asm volatile("@ __cmpxchg4\n"
" ldrex %1, [%2]\n"
" mov %0, #0\n"
" teq %1, %3\n"
" strexeq %0, %4, [%2]\n"
: "=&r" (res), "=&r" (oldval)
: "r" (ptr), "Ir" (old), "r" (new)
: "memory", "cc");
} while (res);
break;
default:
__bad_cmpxchg(ptr, size);
oldval = 0;
}
return oldval;
}
# 217 "./arch/arm/include/asm/cmpxchg.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __cmpxchg_local(volatile void *ptr,
unsigned long old,
unsigned long new, int size)
{
unsigned long ret;
switch (size) {
default:
ret = __cmpxchg(ptr, old, new, size);
}
return ret;
}
# 244 "./arch/arm/include/asm/cmpxchg.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long long __cmpxchg64(unsigned long long *ptr,
unsigned long long old,
unsigned long long new)
{
unsigned long long oldval;
unsigned long res;
prefetchw(ptr);
__asm__ __volatile__(
"1: ldrexd %1, %H1, [%3]\n"
" teq %1, %4\n"
" teqeq %H1, %H4\n"
" bne 2f\n"
" strexd %0, %5, %H5, [%3]\n"
" teq %0, #0\n"
" bne 1b\n"
"2:"
: "=&r" (res), "=&r" (oldval), "+Qo" (*ptr)
: "r" (ptr), "r" (old), "r" (new)
: "cc");
return oldval;
}
# 20 "./arch/arm/include/asm/atomic.h" 2
# 111 "./arch/arm/include/asm/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_cmpxchg_relaxed(atomic_t *ptr, int old, int new)
{
int oldval;
unsigned long res;
prefetchw(&ptr->counter);
do {
__asm__ __volatile__("@ atomic_cmpxchg\n"
"ldrex %1, [%3]\n"
"mov %0, #0\n"
"teq %1, %4\n"
"strexeq %0, %5, [%3]\n"
: "=&r" (res), "=&r" (oldval), "+Qo" (ptr->counter)
: "r" (&ptr->counter), "Ir" (old), "r" (new)
: "cc");
} while (res);
return oldval;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __atomic_add_unless(atomic_t *v, int a, int u)
{
int oldval, newval;
unsigned long tmp;
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
prefetchw(&v->counter);
__asm__ __volatile__ ("@ atomic_add_unless\n"
"1: ldrex %0, [%4]\n"
" teq %0, %5\n"
" beq 2f\n"
" add %1, %0, %6\n"
" strex %2, %1, [%4]\n"
" teq %2, #0\n"
" bne 1b\n"
"2:"
: "=&r" (oldval), "=&r" (newval), "=&r" (tmp), "+Qo" (v->counter)
: "r" (&v->counter), "r" (u), "r" (a)
: "cc");
if (oldval != u)
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return oldval;
}
# 235 "./arch/arm/include/asm/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_add(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "add" "\n" "1: ldrex %0, [%3]\n" " " "add" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_add_return_relaxed(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "add" "_return\n" "1: ldrex %0, [%3]\n" " " "add" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_fetch_add_relaxed(int i, atomic_t *v) { unsigned long tmp; int result, val; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_fetch_" "add" "\n" "1: ldrex %0, [%4]\n" " " "add" " %1, %0, %5\n" " strex %2, %1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_sub(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "sub" "\n" "1: ldrex %0, [%3]\n" " " "sub" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_sub_return_relaxed(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "sub" "_return\n" "1: ldrex %0, [%3]\n" " " "sub" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_fetch_sub_relaxed(int i, atomic_t *v) { unsigned long tmp; int result, val; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_fetch_" "sub" "\n" "1: ldrex %0, [%4]\n" " " "sub" " %1, %0, %5\n" " strex %2, %1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; }
# 245 "./arch/arm/include/asm/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_and(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "and" "\n" "1: ldrex %0, [%3]\n" " " "and" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_fetch_and_relaxed(int i, atomic_t *v) { unsigned long tmp; int result, val; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_fetch_" "and" "\n" "1: ldrex %0, [%4]\n" " " "and" " %1, %0, %5\n" " strex %2, %1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_andnot(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "andnot" "\n" "1: ldrex %0, [%3]\n" " " "bic" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_fetch_andnot_relaxed(int i, atomic_t *v) { unsigned long tmp; int result, val; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_fetch_" "andnot" "\n" "1: ldrex %0, [%4]\n" " " "bic" " %1, %0, %5\n" " strex %2, %1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_or(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "or" "\n" "1: ldrex %0, [%3]\n" " " "orr" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_fetch_or_relaxed(int i, atomic_t *v) { unsigned long tmp; int result, val; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_fetch_" "or" "\n" "1: ldrex %0, [%4]\n" " " "orr" " %1, %0, %5\n" " strex %2, %1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_xor(int i, atomic_t *v) { unsigned long tmp; int result; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_" "xor" "\n" "1: ldrex %0, [%3]\n" " " "eor" " %0, %0, %4\n" " strex %1, %0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_fetch_xor_relaxed(int i, atomic_t *v) { unsigned long tmp; int result, val; prefetchw(&v->counter); __asm__ __volatile__("@ atomic_fetch_" "xor" "\n" "1: ldrex %0, [%4]\n" " " "eor" " %1, %0, %5\n" " strex %2, %1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "Ir" (i) : "cc"); return result; }
# 269 "./arch/arm/include/asm/atomic.h"
typedef struct {
long long counter;
} atomic64_t;
# 298 "./arch/arm/include/asm/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_read(const atomic64_t *v)
{
long long result;
__asm__ __volatile__("@ atomic64_read\n"
" ldrexd %0, %H0, [%1]"
: "=&r" (result)
: "r" (&v->counter), "Qo" (v->counter)
);
return result;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic64_set(atomic64_t *v, long long i)
{
long long tmp;
prefetchw(&v->counter);
__asm__ __volatile__("@ atomic64_set\n"
"1: ldrexd %0, %H0, [%2]\n"
" strexd %0, %3, %H3, [%2]\n"
" teq %0, #0\n"
" bne 1b"
: "=&r" (tmp), "=Qo" (v->counter)
: "r" (&v->counter), "r" (i)
: "cc");
}
# 397 "./arch/arm/include/asm/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic64_add(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "add" "\n" "1: ldrexd %0, %H0, [%3]\n" " " "adds" " %Q0, %Q0, %Q4\n" " " "adc" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_add_return_relaxed(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "add" "_return\n" "1: ldrexd %0, %H0, [%3]\n" " " "adds" " %Q0, %Q0, %Q4\n" " " "adc" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_fetch_add_relaxed(long long i, atomic64_t *v) { long long result, val; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_fetch_" "add" "\n" "1: ldrexd %0, %H0, [%4]\n" " " "adds" " %Q1, %Q0, %Q5\n" " " "adc" " %R1, %R0, %R5\n" " strexd %2, %1, %H1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic64_sub(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "sub" "\n" "1: ldrexd %0, %H0, [%3]\n" " " "subs" " %Q0, %Q0, %Q4\n" " " "sbc" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_sub_return_relaxed(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "sub" "_return\n" "1: ldrexd %0, %H0, [%3]\n" " " "subs" " %Q0, %Q0, %Q4\n" " " "sbc" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_fetch_sub_relaxed(long long i, atomic64_t *v) { long long result, val; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_fetch_" "sub" "\n" "1: ldrexd %0, %H0, [%4]\n" " " "subs" " %Q1, %Q0, %Q5\n" " " "sbc" " %R1, %R0, %R5\n" " strexd %2, %1, %H1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; }
# 412 "./arch/arm/include/asm/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic64_and(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "and" "\n" "1: ldrexd %0, %H0, [%3]\n" " " "and" " %Q0, %Q0, %Q4\n" " " "and" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_fetch_and_relaxed(long long i, atomic64_t *v) { long long result, val; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_fetch_" "and" "\n" "1: ldrexd %0, %H0, [%4]\n" " " "and" " %Q1, %Q0, %Q5\n" " " "and" " %R1, %R0, %R5\n" " strexd %2, %1, %H1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic64_andnot(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "andnot" "\n" "1: ldrexd %0, %H0, [%3]\n" " " "bic" " %Q0, %Q0, %Q4\n" " " "bic" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_fetch_andnot_relaxed(long long i, atomic64_t *v) { long long result, val; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_fetch_" "andnot" "\n" "1: ldrexd %0, %H0, [%4]\n" " " "bic" " %Q1, %Q0, %Q5\n" " " "bic" " %R1, %R0, %R5\n" " strexd %2, %1, %H1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic64_or(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "or" "\n" "1: ldrexd %0, %H0, [%3]\n" " " "orr" " %Q0, %Q0, %Q4\n" " " "orr" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_fetch_or_relaxed(long long i, atomic64_t *v) { long long result, val; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_fetch_" "or" "\n" "1: ldrexd %0, %H0, [%4]\n" " " "orr" " %Q1, %Q0, %Q5\n" " " "orr" " %R1, %R0, %R5\n" " strexd %2, %1, %H1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic64_xor(long long i, atomic64_t *v) { long long result; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_" "xor" "\n" "1: ldrexd %0, %H0, [%3]\n" " " "eor" " %Q0, %Q0, %Q4\n" " " "eor" " %R0, %R0, %R4\n" " strexd %1, %0, %H0, [%3]\n" " teq %1, #0\n" " bne 1b" : "=&r" (result), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_fetch_xor_relaxed(long long i, atomic64_t *v) { long long result, val; unsigned long tmp; prefetchw(&v->counter); __asm__ __volatile__("@ atomic64_fetch_" "xor" "\n" "1: ldrexd %0, %H0, [%4]\n" " " "eor" " %Q1, %Q0, %Q5\n" " " "eor" " %R1, %R0, %R5\n" " strexd %2, %1, %H1, [%4]\n" " teq %2, #0\n" " bne 1b" : "=&r" (result), "=&r" (val), "=&r" (tmp), "+Qo" (v->counter) : "r" (&v->counter), "r" (i) : "cc"); return result; }
# 427 "./arch/arm/include/asm/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long
atomic64_cmpxchg_relaxed(atomic64_t *ptr, long long old, long long new)
{
long long oldval;
unsigned long res;
prefetchw(&ptr->counter);
do {
__asm__ __volatile__("@ atomic64_cmpxchg\n"
"ldrexd %1, %H1, [%3]\n"
"mov %0, #0\n"
"teq %1, %4\n"
"teqeq %H1, %H4\n"
"strexdeq %0, %5, %H5, [%3]"
: "=&r" (res), "=&r" (oldval), "+Qo" (ptr->counter)
: "r" (&ptr->counter), "r" (old), "r" (new)
: "cc");
} while (res);
return oldval;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_xchg_relaxed(atomic64_t *ptr, long long new)
{
long long result;
unsigned long tmp;
prefetchw(&ptr->counter);
__asm__ __volatile__("@ atomic64_xchg\n"
"1: ldrexd %0, %H0, [%3]\n"
" strexd %1, %4, %H4, [%3]\n"
" teq %1, #0\n"
" bne 1b"
: "=&r" (result), "=&r" (tmp), "+Qo" (ptr->counter)
: "r" (&ptr->counter), "r" (new)
: "cc");
return result;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long long atomic64_dec_if_positive(atomic64_t *v)
{
long long result;
unsigned long tmp;
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
prefetchw(&v->counter);
__asm__ __volatile__("@ atomic64_dec_if_positive\n"
"1: ldrexd %0, %H0, [%3]\n"
" subs %Q0, %Q0, #1\n"
" sbc %R0, %R0, #0\n"
" teq %R0, #0\n"
" bmi 2f\n"
" strexd %1, %0, %H0, [%3]\n"
" teq %1, #0\n"
" bne 1b\n"
"2:"
: "=&r" (result), "=&r" (tmp), "+Qo" (v->counter)
: "r" (&v->counter)
: "cc");
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return result;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic64_add_unless(atomic64_t *v, long long a, long long u)
{
long long val;
unsigned long tmp;
int ret = 1;
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
prefetchw(&v->counter);
__asm__ __volatile__("@ atomic64_add_unless\n"
"1: ldrexd %0, %H0, [%4]\n"
" teq %0, %5\n"
" teqeq %H0, %H5\n"
" moveq %1, #0\n"
" beq 2f\n"
" adds %Q0, %Q0, %Q6\n"
" adc %R0, %R0, %R6\n"
" strexd %2, %0, %H0, [%4]\n"
" teq %2, #0\n"
" bne 1b\n"
"2:"
: "=&r" (val), "+r" (ret), "=&r" (tmp), "+Qo" (v->counter)
: "r" (&v->counter), "r" (u), "r" (a)
: "cc");
if (ret)
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return ret;
}
# 6 "./include/linux/atomic.h" 2
# 531 "./include/linux/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_add_unless(atomic_t *v, int a, int u)
{
return __atomic_add_unless(v, a, u) != u;
}
# 587 "./include/linux/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_inc_not_zero_hint(atomic_t *v, int hint)
{
int val, c = hint;
if (!hint)
return atomic_add_unless((v), 1, 0);
do {
val = ({ typeof(atomic_cmpxchg_relaxed(v, c, c + 1)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_cmpxchg_relaxed(v, c, c + 1); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
if (val == c)
return 1;
c = val;
} while (c);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_inc_unless_negative(atomic_t *p)
{
int v, v1;
for (v = 0; v >= 0; v = v1) {
v1 = ({ typeof(atomic_cmpxchg_relaxed(p, v, v + 1)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_cmpxchg_relaxed(p, v, v + 1); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
if (__builtin_expect(!!(v1 == v), 1))
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_dec_unless_positive(atomic_t *p)
{
int v, v1;
for (v = 0; v <= 0; v = v1) {
v1 = ({ typeof(atomic_cmpxchg_relaxed(p, v, v - 1)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_cmpxchg_relaxed(p, v, v - 1); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
if (__builtin_expect(!!(v1 == v), 1))
return 1;
}
return 0;
}
# 640 "./include/linux/atomic.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_dec_if_positive(atomic_t *v)
{
int c, old, dec;
c = ({ union { typeof((v)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((v)->counter), __u.__c, sizeof((v)->counter)); else __read_once_size_nocheck(&((v)->counter), __u.__c, sizeof((v)->counter)); do { } while (0); __u.__val; });
for (;;) {
dec = c - 1;
if (__builtin_expect(!!(dec < 0), 0))
break;
old = ({ typeof(atomic_cmpxchg_relaxed((v), c, dec)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_cmpxchg_relaxed((v), c, dec); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
if (__builtin_expect(!!(old == c), 1))
break;
c = old;
}
return dec;
}
# 1076 "./include/linux/atomic.h"
# 1 "./include/asm-generic/atomic-long.h" 1
# 31 "./include/asm-generic/atomic-long.h"
typedef atomic_t atomic_long_t;
# 45 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_read(const atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ union { typeof((v)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((v)->counter), __u.__c, sizeof((v)->counter)); else __read_once_size_nocheck(&((v)->counter), __u.__c, sizeof((v)->counter)); do { } while (0); __u.__val; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_read_acquire(const atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(*&(v)->counter) ___p1 = ({ union { typeof(*&(v)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&(*&(v)->counter), __u.__c, sizeof(*&(v)->counter)); else __read_once_size_nocheck(&(*&(v)->counter), __u.__c, sizeof(*&(v)->counter)); do { } while (0); __u.__val; }); do { bool __cond = !((sizeof(*&(v)->counter) == sizeof(char) || sizeof(*&(v)->counter) == sizeof(short) || sizeof(*&(v)->counter) == sizeof(int) || sizeof(*&(v)->counter) == sizeof(long))); extern void __compiletime_assert_0(void) ; if (__cond) __compiletime_assert_0(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ___p1; }); }
# 57 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_long_set(atomic_long_t *l, long i) { atomic_t *v = (atomic_t *)l; ({ union { typeof(((v)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((v)->counter))) ((i)) }; __write_once_size(&(((v)->counter)), __u.__c, sizeof(((v)->counter))); __u.__val; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void atomic_long_set_release(atomic_long_t *l, long i) { atomic_t *v = (atomic_t *)l; do { do { bool __cond = !((sizeof(*&(v)->counter) == sizeof(char) || sizeof(*&(v)->counter) == sizeof(short) || sizeof(*&(v)->counter) == sizeof(int) || sizeof(*&(v)->counter) == sizeof(long))); extern void __compiletime_assert_1(void) ; if (__cond) __compiletime_assert_1(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(v)->counter) __val; char __c[1]; } __u = { .__val = ( typeof(*&(v)->counter)) ((i)) }; __write_once_size(&(*&(v)->counter), __u.__c, sizeof(*&(v)->counter)); __u.__val; }); } while (0); }
# 70 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_add_return(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_add_return_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_add_return_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_add_return_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_add_return_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_add_return_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_add_return_relaxed(i, v)) __ret = atomic_add_return_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_add_return_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_add_return_relaxed(i, v); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_sub_return(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_sub_return_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_sub_return_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_sub_return_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_sub_return_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_sub_return_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_sub_return_relaxed(i, v)) __ret = atomic_sub_return_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_sub_return_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_sub_return_relaxed(i, v); }); }
# 102 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_inc(atomic_long_t *l)
{
atomic_t *v = (atomic_t *)l;
atomic_add(1, v);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_dec(atomic_long_t *l)
{
atomic_t *v = (atomic_t *)l;
atomic_sub(1, v);
}
# 125 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_add(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_add_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_add_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_add_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_add_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_add_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_add_relaxed(i, v)) __ret = atomic_fetch_add_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_add_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_add_relaxed(i, v); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_sub(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_sub_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_sub_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_sub_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_sub_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_sub_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_sub_relaxed(i, v)) __ret = atomic_fetch_sub_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_sub_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_sub_relaxed(i, v); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_and(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_and_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_and_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_and_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_and_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_and_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_and_relaxed(i, v)) __ret = atomic_fetch_and_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_and_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_and_relaxed(i, v); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_andnot(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_andnot_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_andnot_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_andnot_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_andnot_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_andnot_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_andnot_relaxed(i, v)) __ret = atomic_fetch_andnot_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_andnot_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_andnot_relaxed(i, v); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_or(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_or_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_or_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_or_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_or_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_or_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_or_relaxed(i, v)) __ret = atomic_fetch_or_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_or_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_or_relaxed(i, v); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_xor(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_xor_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_xor_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_xor_relaxed(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_xor_relaxed(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_xor_acquire(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_xor_relaxed(i, v)) __ret = atomic_fetch_xor_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_xor_release(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_xor_relaxed(i, v); }); }
# 161 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_inc(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_add_relaxed(1, (v))) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_add_relaxed(1, (v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_inc_relaxed(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_add_relaxed(1, (v)); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_inc_acquire(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_add_relaxed(1, (v))) __ret = atomic_fetch_add_relaxed(1, (v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_inc_release(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_add_relaxed(1, (v)); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_dec(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_sub_relaxed(1, (v))) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_fetch_sub_relaxed(1, (v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_dec_relaxed(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)atomic_fetch_sub_relaxed(1, (v)); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_dec_acquire(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof(atomic_fetch_sub_relaxed(1, (v))) __ret = atomic_fetch_sub_relaxed(1, (v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_fetch_dec_release(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); atomic_fetch_sub_relaxed(1, (v)); }); }
# 181 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_add(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; atomic_add(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_sub(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; atomic_sub(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_and(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; atomic_and(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_andnot(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; atomic_andnot(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_or(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; atomic_or(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void atomic_long_xor(long i, atomic_long_t *l) { atomic_t *v = (atomic_t *)l; atomic_xor(i, v); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_long_sub_and_test(long i, atomic_long_t *l)
{
atomic_t *v = (atomic_t *)l;
return (({ typeof(atomic_sub_return_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_sub_return_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_long_dec_and_test(atomic_long_t *l)
{
atomic_t *v = (atomic_t *)l;
return (({ typeof(atomic_sub_return_relaxed(1, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_sub_return_relaxed(1, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_long_inc_and_test(atomic_long_t *l)
{
atomic_t *v = (atomic_t *)l;
return (({ typeof(atomic_add_return_relaxed(1, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_add_return_relaxed(1, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int atomic_long_add_negative(long i, atomic_long_t *l)
{
atomic_t *v = (atomic_t *)l;
return (({ typeof(atomic_add_return_relaxed(i, v)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_add_return_relaxed(i, v); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) < 0);
}
# 226 "./include/asm-generic/atomic-long.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_inc_return(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof((atomic_add_return_relaxed(1, v))) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = (atomic_add_return_relaxed(1, v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_inc_return_relaxed(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)(atomic_add_return_relaxed(1, v)); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_inc_return_acquire(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof((atomic_add_return_relaxed(1, v))) __ret = (atomic_add_return_relaxed(1, v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_inc_return_release(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); (atomic_add_return_relaxed(1, v)); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_dec_return(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof((atomic_sub_return_relaxed(1, v))) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = (atomic_sub_return_relaxed(1, v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_dec_return_relaxed(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)(atomic_sub_return_relaxed(1, v)); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_dec_return_acquire(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ typeof((atomic_sub_return_relaxed(1, v))) __ret = (atomic_sub_return_relaxed(1, v)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_dec_return_release(atomic_long_t *l) { atomic_t *v = (atomic_t *)l; return (long)({ __asm__ __volatile__ ("dmb " "ish" : : : "memory"); (atomic_sub_return_relaxed(1, v)); }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long atomic_long_add_unless(atomic_long_t *l, long a, long u)
{
atomic_t *v = (atomic_t *)l;
return (long)atomic_add_unless(v, a, u);
}
# 1077 "./include/linux/atomic.h" 2
# 39 "./include/linux/rcupdate.h" 2
# 1 "./include/linux/preempt.h" 1
# 81 "./include/linux/preempt.h"
# 1 "./arch/arm/include/generated/asm/preempt.h" 1
# 1 "./include/asm-generic/preempt.h" 1
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int preempt_count(void)
{
return ({ union { typeof(current_thread_info()->preempt_count) __val; char __c[1]; } __u; if (1) __read_once_size(&(current_thread_info()->preempt_count), __u.__c, sizeof(current_thread_info()->preempt_count)); else __read_once_size_nocheck(&(current_thread_info()->preempt_count), __u.__c, sizeof(current_thread_info()->preempt_count)); do { } while (0); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) volatile int *preempt_count_ptr(void)
{
return ¤t_thread_info()->preempt_count;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void preempt_count_set(int pc)
{
*preempt_count_ptr() = pc;
}
# 35 "./include/asm-generic/preempt.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void set_preempt_need_resched(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void clear_preempt_need_resched(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) bool test_preempt_need_resched(void)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __preempt_count_add(int val)
{
*preempt_count_ptr() += val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __preempt_count_sub(int val)
{
*preempt_count_ptr() -= val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) bool __preempt_count_dec_and_test(void)
{
return !--*preempt_count_ptr() && test_ti_thread_flag(current_thread_info(), 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) bool should_resched(int preempt_offset)
{
return __builtin_expect(!!(preempt_count() == preempt_offset && test_ti_thread_flag(current_thread_info(), 1)), 0);
}
extern void preempt_schedule(void);
extern void preempt_schedule_notrace(void);
# 2 "./arch/arm/include/generated/asm/preempt.h" 2
# 82 "./include/linux/preempt.h" 2
# 41 "./include/linux/rcupdate.h" 2
# 1 "./include/linux/bottom_half.h" 1
# 10 "./include/linux/bottom_half.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
{
__preempt_count_add(cnt);
__asm__ __volatile__("": : :"memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void local_bh_disable(void)
{
__local_bh_disable_ip(({ __label__ __here; __here: (unsigned long)&&__here; }), (2 * (1UL << (0 + 8))));
}
extern void _local_bh_enable(void);
extern void __local_bh_enable_ip(unsigned long ip, unsigned int cnt);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void local_bh_enable_ip(unsigned long ip)
{
__local_bh_enable_ip(ip, (2 * (1UL << (0 + 8))));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void local_bh_enable(void)
{
__local_bh_enable_ip(({ __label__ __here; __here: (unsigned long)&&__here; }), (2 * (1UL << (0 + 8))));
}
# 42 "./include/linux/rcupdate.h" 2
# 1 "./include/linux/lockdep.h" 1
# 13 "./include/linux/lockdep.h"
struct task_struct;
struct lockdep_map;
extern int prove_locking;
extern int lock_stat;
# 491 "./include/linux/lockdep.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void lockdep_off(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void lockdep_on(void)
{
}
# 529 "./include/linux/lockdep.h"
struct lock_class_key { };
# 542 "./include/linux/lockdep.h"
struct pin_cookie { };
# 552 "./include/linux/lockdep.h"
enum xhlock_context_t {
XHLOCK_HARD,
XHLOCK_SOFT,
XHLOCK_CTX_NR,
};
# 597 "./include/linux/lockdep.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void crossrelease_hist_start(enum xhlock_context_t c) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void crossrelease_hist_end(enum xhlock_context_t c) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void lockdep_invariant_state(bool force) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void lockdep_init_task(struct task_struct *task) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void lockdep_free_task(struct task_struct *task) {}
# 663 "./include/linux/lockdep.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void print_irqtrace_events(struct task_struct *curr)
{
}
# 731 "./include/linux/lockdep.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
lockdep_rcu_suspicious(const char *file, const int line, const char *s)
{
}
# 43 "./include/linux/rcupdate.h" 2
# 1 "./include/linux/cpumask.h" 1
# 11 "./include/linux/cpumask.h"
# 1 "./include/linux/threads.h" 1
# 12 "./include/linux/cpumask.h" 2
# 1 "./include/linux/bitmap.h" 1
# 1 "./include/linux/string.h" 1
# 10 "./include/linux/string.h"
# 1 "./include/uapi/linux/string.h" 1
# 11 "./include/linux/string.h" 2
extern char *strndup_user(const char *, long);
extern void *memdup_user(const void *, size_t);
extern void *memdup_user_nul(const void *, size_t);
# 1 "./arch/arm/include/asm/string.h" 1
# 11 "./arch/arm/include/asm/string.h"
extern char * strrchr(const char * s, int c);
extern char * strchr(const char * s, int c);
extern void * memcpy(void *, const void *, __kernel_size_t);
extern void * __memcpy(void *, const void *, __kernel_size_t);
extern void * memmove(void *, const void *, __kernel_size_t);
extern void * __memmove(void *, const void *, __kernel_size_t);
extern void * memchr(const void *, int, __kernel_size_t);
extern void * memset(void *, int, __kernel_size_t);
extern void * __memset(void *, int, __kernel_size_t);
extern void *__memset32(uint32_t *, uint32_t v, __kernel_size_t);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *memset32(uint32_t *p, uint32_t v, __kernel_size_t n)
{
return __memset32(p, v, n * 4);
}
extern void *__memset64(uint64_t *, uint32_t low, __kernel_size_t, uint32_t hi);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *memset64(uint64_t *p, uint64_t v, __kernel_size_t n)
{
return __memset64(p, v, n * 8, v >> 32);
}
extern void __memzero(void *ptr, __kernel_size_t n);
# 20 "./include/linux/string.h" 2
extern char * strcpy(char *,const char *);
extern char * strncpy(char *,const char *, __kernel_size_t);
size_t strlcpy(char *, const char *, size_t);
ssize_t strscpy(char *, const char *, size_t);
extern char * strcat(char *, const char *);
extern char * strncat(char *, const char *, __kernel_size_t);
extern size_t strlcat(char *, const char *, __kernel_size_t);
extern int strcmp(const char *,const char *);
extern int strncmp(const char *,const char *,__kernel_size_t);
extern int strcasecmp(const char *s1, const char *s2);
extern int strncasecmp(const char *s1, const char *s2, size_t n);
extern char * strchrnul(const char *,int);
extern char * strnchr(const char *, size_t, int);
extern char * __attribute__((warn_unused_result)) skip_spaces(const char *);
extern char *strim(char *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result)) char *strstrip(char *str)
{
return strim(str);
}
extern char * strstr(const char *, const char *);
extern char * strnstr(const char *, const char *, size_t);
extern __kernel_size_t strlen(const char *);
extern __kernel_size_t strnlen(const char *,__kernel_size_t);
extern char * strpbrk(const char *,const char *);
extern char * strsep(char **,const char *);
extern __kernel_size_t strspn(const char *,const char *);
extern __kernel_size_t strcspn(const char *,const char *);
extern void *memset16(uint16_t *, uint16_t, __kernel_size_t);
# 116 "./include/linux/string.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *memset_l(unsigned long *p, unsigned long v,
__kernel_size_t n)
{
if (32 == 32)
return memset32((uint32_t *)p, v, n);
else
return memset64((uint64_t *)p, v, n);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *memset_p(void **p, void *v, __kernel_size_t n)
{
if (32 == 32)
return memset32((uint32_t *)p, (uintptr_t)v, n);
else
return memset64((uint64_t *)p, (uintptr_t)v, n);
}
# 140 "./include/linux/string.h"
extern void * memscan(void *,int,__kernel_size_t);
extern int memcmp(const void *,const void *,__kernel_size_t);
extern int bcmp(const void *,const void *,__kernel_size_t);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result)) int memcpy_mcsafe(void *dst, const void *src,
size_t cnt)
{
memcpy(dst, src, cnt);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void memcpy_flushcache(void *dst, const void *src, size_t cnt)
{
memcpy(dst, src, cnt);
}
void *memchr_inv(const void *s, int c, size_t n);
char *strreplace(char *s, char old, char new);
extern void kfree_const(const void *x);
extern char *kstrdup(const char *s, gfp_t gfp) __attribute__((__malloc__));
extern const char *kstrdup_const(const char *s, gfp_t gfp);
extern char *kstrndup(const char *s, size_t len, gfp_t gfp);
extern void *kmemdup(const void *src, size_t len, gfp_t gfp);
extern char *kmemdup_nul(const char *s, size_t len, gfp_t gfp);
extern char **argv_split(gfp_t gfp, const char *str, int *argcp);
extern void argv_free(char **argv);
extern bool sysfs_streq(const char *s1, const char *s2);
# 191 "./include/linux/string.h"
int match_string(const char * const *array, size_t n, const char *string);
int __sysfs_match_string(const char * const *array, size_t n, const char *s);
# 204 "./include/linux/string.h"
int vbin_printf(u32 *bin_buf, size_t size, const char *fmt, va_list args);
int bstr_printf(char *buf, size_t size, const char *fmt, const u32 *bin_buf);
int bprintf(u32 *bin_buf, size_t size, const char *fmt, ...) __attribute__((format(printf, 3, 4)));
extern ssize_t memory_read_from_buffer(void *to, size_t count, loff_t *ppos,
const void *from, size_t available);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool strstarts(const char *str, const char *prefix)
{
return strncmp(str, prefix, strlen(prefix)) == 0;
}
size_t memweight(const void *ptr, size_t bytes);
void memzero_explicit(void *s, size_t count);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const char *kbasename(const char *path)
{
const char *tail = strrchr(path, '/');
return tail ? tail + 1 : path;
}
void fortify_panic(const char *name) __attribute__((noreturn)) ;
void __read_overflow(void) ;
void __read_overflow2(void) ;
void __read_overflow3(void) ;
void __write_overflow(void) ;
# 485 "./include/linux/string.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void memcpy_and_pad(void *dest, size_t dest_len,
const void *src, size_t count, int pad)
{
if (dest_len > count) {
memcpy(dest, src, count);
({ void *__p = (dest + count); size_t __n = dest_len - count; if ((__n) != 0) { if (__builtin_constant_p((pad)) && (pad) == 0) __memzero((__p),(__n)); else memset((__p),(pad),(__n)); } (__p); });
} else
memcpy(dest, src, dest_len);
}
# 10 "./include/linux/bitmap.h" 2
# 93 "./include/linux/bitmap.h"
extern unsigned long *bitmap_alloc(unsigned int nbits, gfp_t flags);
extern unsigned long *bitmap_zalloc(unsigned int nbits, gfp_t flags);
extern void bitmap_free(const unsigned long *bitmap);
extern int __bitmap_empty(const unsigned long *bitmap, unsigned int nbits);
extern int __bitmap_full(const unsigned long *bitmap, unsigned int nbits);
extern int __bitmap_equal(const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
extern void __bitmap_complement(unsigned long *dst, const unsigned long *src,
unsigned int nbits);
extern void __bitmap_shift_right(unsigned long *dst, const unsigned long *src,
unsigned int shift, unsigned int nbits);
extern void __bitmap_shift_left(unsigned long *dst, const unsigned long *src,
unsigned int shift, unsigned int nbits);
extern int __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
extern void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
extern void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
extern int __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
extern int __bitmap_intersects(const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
extern int __bitmap_subset(const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
extern int __bitmap_weight(const unsigned long *bitmap, unsigned int nbits);
extern void __bitmap_set(unsigned long *map, unsigned int start, int len);
extern void __bitmap_clear(unsigned long *map, unsigned int start, int len);
extern unsigned long bitmap_find_next_zero_area_off(unsigned long *map,
unsigned long size,
unsigned long start,
unsigned int nr,
unsigned long align_mask,
unsigned long align_offset);
# 146 "./include/linux/bitmap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
bitmap_find_next_zero_area(unsigned long *map,
unsigned long size,
unsigned long start,
unsigned int nr,
unsigned long align_mask)
{
return bitmap_find_next_zero_area_off(map, size, start, nr,
align_mask, 0);
}
extern int __bitmap_parse(const char *buf, unsigned int buflen, int is_user,
unsigned long *dst, int nbits);
extern int bitmap_parse_user(const char *ubuf, unsigned int ulen,
unsigned long *dst, int nbits);
extern int bitmap_parselist(const char *buf, unsigned long *maskp,
int nmaskbits);
extern int bitmap_parselist_user(const char *ubuf, unsigned int ulen,
unsigned long *dst, int nbits);
extern void bitmap_remap(unsigned long *dst, const unsigned long *src,
const unsigned long *old, const unsigned long *new, unsigned int nbits);
extern int bitmap_bitremap(int oldbit,
const unsigned long *old, const unsigned long *new, int bits);
extern void bitmap_onto(unsigned long *dst, const unsigned long *orig,
const unsigned long *relmap, unsigned int bits);
extern void bitmap_fold(unsigned long *dst, const unsigned long *orig,
unsigned int sz, unsigned int nbits);
extern int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, int order);
extern void bitmap_release_region(unsigned long *bitmap, unsigned int pos, int order);
extern int bitmap_allocate_region(unsigned long *bitmap, unsigned int pos, int order);
extern unsigned int bitmap_from_u32array(unsigned long *bitmap,
unsigned int nbits,
const u32 *buf,
unsigned int nwords);
extern unsigned int bitmap_to_u32array(u32 *buf,
unsigned int nwords,
const unsigned long *bitmap,
unsigned int nbits);
extern unsigned int bitmap_ord_to_pos(const unsigned long *bitmap, unsigned int ord, unsigned int nbits);
extern int bitmap_print_to_pagebuf(bool list, char *buf,
const unsigned long *maskp, int nmaskbits);
# 204 "./include/linux/bitmap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_zero(unsigned long *dst, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
*dst = 0UL;
else {
unsigned int len = (((nbits) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8))) * sizeof(unsigned long);
({ void *__p = (dst); size_t __n = len; if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_fill(unsigned long *dst, unsigned int nbits)
{
unsigned int nlongs = (((nbits) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8)));
if (!(__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0)) {
unsigned int len = (nlongs - 1) * sizeof(unsigned long);
({ void *__p = (dst); size_t __n = len; if ((__n) != 0) { if (__builtin_constant_p((0xff)) && (0xff) == 0) __memzero((__p),(__n)); else memset((__p),(0xff),(__n)); } (__p); });
}
dst[nlongs - 1] = (~0UL >> (-(nbits) & (32 - 1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_copy(unsigned long *dst, const unsigned long *src,
unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
*dst = *src;
else {
unsigned int len = (((nbits) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8))) * sizeof(unsigned long);
memcpy(dst, src, len);
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_and(unsigned long *dst, const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return (*dst = *src1 & *src2 & (~0UL >> (-(nbits) & (32 - 1)))) != 0;
return __bitmap_and(dst, src1, src2, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_or(unsigned long *dst, const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
*dst = *src1 | *src2;
else
__bitmap_or(dst, src1, src2, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_xor(unsigned long *dst, const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
*dst = *src1 ^ *src2;
else
__bitmap_xor(dst, src1, src2, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_andnot(unsigned long *dst, const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return (*dst = *src1 & ~(*src2) & (~0UL >> (-(nbits) & (32 - 1)))) != 0;
return __bitmap_andnot(dst, src1, src2, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_complement(unsigned long *dst, const unsigned long *src,
unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
*dst = ~(*src);
else
__bitmap_complement(dst, src, nbits);
}
# 285 "./include/linux/bitmap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_equal(const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return !((*src1 ^ *src2) & (~0UL >> (-(nbits) & (32 - 1))));
if (__builtin_constant_p(nbits & (8 - 1)) &&
(((nbits) & ((typeof(nbits))(8) - 1)) == 0))
return !memcmp(src1, src2, nbits / 8);
return __bitmap_equal(src1, src2, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_intersects(const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return ((*src1 & *src2) & (~0UL >> (-(nbits) & (32 - 1)))) != 0;
else
return __bitmap_intersects(src1, src2, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_subset(const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return ! ((*src1 & ~(*src2)) & (~0UL >> (-(nbits) & (32 - 1))));
else
return __bitmap_subset(src1, src2, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_empty(const unsigned long *src, unsigned nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return ! (*src & (~0UL >> (-(nbits) & (32 - 1))));
return _find_first_bit_le(src,nbits) == nbits;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_full(const unsigned long *src, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return ! (~(*src) & (~0UL >> (-(nbits) & (32 - 1))));
return _find_first_zero_bit_le(src,nbits) == nbits;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int bitmap_weight(const unsigned long *src, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
return hweight_long(*src & (~0UL >> (-(nbits) & (32 - 1))));
return __bitmap_weight(src, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void bitmap_set(unsigned long *map, unsigned int start,
unsigned int nbits)
{
if (__builtin_constant_p(nbits) && nbits == 1)
__set_bit(start, map);
else if (__builtin_constant_p(start & (8 - 1)) &&
(((start) & ((typeof(start))(8) - 1)) == 0) &&
__builtin_constant_p(nbits & (8 - 1)) &&
(((nbits) & ((typeof(nbits))(8) - 1)) == 0))
({ void *__p = ((char *)map + start / 8); size_t __n = nbits / 8; if ((__n) != 0) { if (__builtin_constant_p((0xff)) && (0xff) == 0) __memzero((__p),(__n)); else memset((__p),(0xff),(__n)); } (__p); });
else
__bitmap_set(map, start, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void bitmap_clear(unsigned long *map, unsigned int start,
unsigned int nbits)
{
if (__builtin_constant_p(nbits) && nbits == 1)
__clear_bit(start, map);
else if (__builtin_constant_p(start & (8 - 1)) &&
(((start) & ((typeof(start))(8) - 1)) == 0) &&
__builtin_constant_p(nbits & (8 - 1)) &&
(((nbits) & ((typeof(nbits))(8) - 1)) == 0))
({ void *__p = ((char *)map + start / 8); size_t __n = nbits / 8; if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
else
__bitmap_clear(map, start, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_shift_right(unsigned long *dst, const unsigned long *src,
unsigned int shift, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
*dst = (*src & (~0UL >> (-(nbits) & (32 - 1)))) >> shift;
else
__bitmap_shift_right(dst, src, shift, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_shift_left(unsigned long *dst, const unsigned long *src,
unsigned int shift, unsigned int nbits)
{
if ((__builtin_constant_p(nbits) && (nbits) <= 32 && (nbits) > 0))
*dst = (*src << shift) & (~0UL >> (-(nbits) & (32 - 1)));
else
__bitmap_shift_left(dst, src, shift, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bitmap_parse(const char *buf, unsigned int buflen,
unsigned long *maskp, int nmaskbits)
{
return __bitmap_parse(buf, buflen, 0, maskp, nmaskbits);
}
# 431 "./include/linux/bitmap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bitmap_from_u64(unsigned long *dst, u64 mask)
{
dst[0] = mask & (~0UL);
if (sizeof(mask) > sizeof(unsigned long))
dst[1] = mask >> 32;
}
# 13 "./include/linux/cpumask.h" 2
typedef struct cpumask { unsigned long bits[(((4) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8)))]; } cpumask_t;
# 38 "./include/linux/cpumask.h"
extern unsigned int nr_cpu_ids;
# 89 "./include/linux/cpumask.h"
extern struct cpumask __cpu_possible_mask;
extern struct cpumask __cpu_online_mask;
extern struct cpumask __cpu_present_mask;
extern struct cpumask __cpu_active_mask;
# 119 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int cpumask_check(unsigned int cpu)
{
return cpu;
}
# 179 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int cpumask_first(const struct cpumask *srcp)
{
return _find_first_bit_le(((srcp)->bits),((unsigned int)4));
}
unsigned int cpumask_next(int n, const struct cpumask *srcp);
# 193 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int cpumask_next_zero(int n, const struct cpumask *srcp)
{
if (n != -1)
cpumask_check(n);
return _find_next_zero_bit_le(((srcp)->bits),((unsigned int)4),n+1);
}
int cpumask_next_and(int n, const struct cpumask *, const struct cpumask *);
int cpumask_any_but(const struct cpumask *mask, unsigned int cpu);
unsigned int cpumask_local_spread(unsigned int i, int node);
# 229 "./include/linux/cpumask.h"
extern int cpumask_next_wrap(int n, const struct cpumask *mask, int start, bool wrap);
# 281 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_set_cpu(unsigned int cpu, struct cpumask *dstp)
{
_set_bit(cpumask_check(cpu),((dstp)->bits));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __cpumask_set_cpu(unsigned int cpu, struct cpumask *dstp)
{
__set_bit(cpumask_check(cpu), ((dstp)->bits));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_clear_cpu(int cpu, struct cpumask *dstp)
{
_clear_bit(cpumask_check(cpu),((dstp)->bits));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __cpumask_clear_cpu(int cpu, struct cpumask *dstp)
{
__clear_bit(cpumask_check(cpu), ((dstp)->bits));
}
# 314 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_test_cpu(int cpu, const struct cpumask *cpumask)
{
return test_bit(cpumask_check(cpu), (((cpumask))->bits));
}
# 328 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_test_and_set_cpu(int cpu, struct cpumask *cpumask)
{
return _test_and_set_bit(cpumask_check(cpu),((cpumask)->bits));
}
# 342 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_test_and_clear_cpu(int cpu, struct cpumask *cpumask)
{
return _test_and_clear_bit(cpumask_check(cpu),((cpumask)->bits));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_setall(struct cpumask *dstp)
{
bitmap_fill(((dstp)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_clear(struct cpumask *dstp)
{
bitmap_zero(((dstp)->bits), ((unsigned int)4));
}
# 373 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_and(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
return bitmap_and(((dstp)->bits), ((src1p)->bits),
((src2p)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_or(struct cpumask *dstp, const struct cpumask *src1p,
const struct cpumask *src2p)
{
bitmap_or(((dstp)->bits), ((src1p)->bits),
((src2p)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_xor(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
bitmap_xor(((dstp)->bits), ((src1p)->bits),
((src2p)->bits), ((unsigned int)4));
}
# 416 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_andnot(struct cpumask *dstp,
const struct cpumask *src1p,
const struct cpumask *src2p)
{
return bitmap_andnot(((dstp)->bits), ((src1p)->bits),
((src2p)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_complement(struct cpumask *dstp,
const struct cpumask *srcp)
{
bitmap_complement(((dstp)->bits), ((srcp)->bits),
((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cpumask_equal(const struct cpumask *src1p,
const struct cpumask *src2p)
{
return bitmap_equal(((src1p)->bits), ((src2p)->bits),
((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cpumask_intersects(const struct cpumask *src1p,
const struct cpumask *src2p)
{
return bitmap_intersects(((src1p)->bits), ((src2p)->bits),
((unsigned int)4));
}
# 467 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_subset(const struct cpumask *src1p,
const struct cpumask *src2p)
{
return bitmap_subset(((src1p)->bits), ((src2p)->bits),
((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cpumask_empty(const struct cpumask *srcp)
{
return bitmap_empty(((srcp)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cpumask_full(const struct cpumask *srcp)
{
return bitmap_full(((srcp)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int cpumask_weight(const struct cpumask *srcp)
{
return bitmap_weight(((srcp)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_shift_right(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
bitmap_shift_right(((dstp)->bits), ((srcp)->bits), n,
((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_shift_left(struct cpumask *dstp,
const struct cpumask *srcp, int n)
{
bitmap_shift_left(((dstp)->bits), ((srcp)->bits), n,
((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cpumask_copy(struct cpumask *dstp,
const struct cpumask *srcp)
{
bitmap_copy(((dstp)->bits), ((srcp)->bits), ((unsigned int)4));
}
# 578 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_parse_user(const char *buf, int len,
struct cpumask *dstp)
{
return bitmap_parse_user(buf, len, ((dstp)->bits), ((unsigned int)4));
}
# 592 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_parselist_user(const char *buf, int len,
struct cpumask *dstp)
{
return bitmap_parselist_user(buf, len, ((dstp)->bits),
((unsigned int)4));
}
# 606 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpumask_parse(const char *buf, struct cpumask *dstp)
{
char *nl = strchr(buf, '\n');
unsigned int len = nl ? (unsigned int)(nl - buf) : strlen(buf);
return bitmap_parse(buf, len, ((dstp)->bits), ((unsigned int)4));
}
# 621 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpulist_parse(const char *buf, struct cpumask *dstp)
{
return bitmap_parselist(buf, ((dstp)->bits), ((unsigned int)4));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) size_t cpumask_size(void)
{
return (((((unsigned int)4)) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8))) * sizeof(long);
}
# 694 "./include/linux/cpumask.h"
typedef struct cpumask cpumask_var_t[1];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool alloc_cpumask_var(cpumask_var_t *mask, gfp_t flags)
{
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool alloc_cpumask_var_node(cpumask_var_t *mask, gfp_t flags,
int node)
{
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool zalloc_cpumask_var(cpumask_var_t *mask, gfp_t flags)
{
cpumask_clear(*mask);
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool zalloc_cpumask_var_node(cpumask_var_t *mask, gfp_t flags,
int node)
{
cpumask_clear(*mask);
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void alloc_bootmem_cpumask_var(cpumask_var_t *mask)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void free_cpumask_var(cpumask_var_t mask)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void free_bootmem_cpumask_var(cpumask_var_t mask)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cpumask_available(cpumask_var_t mask)
{
return true;
}
extern const unsigned long cpu_all_bits[(((4) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8)))];
# 754 "./include/linux/cpumask.h"
void init_cpu_present(const struct cpumask *src);
void init_cpu_possible(const struct cpumask *src);
void init_cpu_online(const struct cpumask *src);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void reset_cpu_possible_mask(void)
{
bitmap_zero(((&__cpu_possible_mask)->bits), 4);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
set_cpu_possible(unsigned int cpu, bool possible)
{
if (possible)
cpumask_set_cpu(cpu, &__cpu_possible_mask);
else
cpumask_clear_cpu(cpu, &__cpu_possible_mask);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
set_cpu_present(unsigned int cpu, bool present)
{
if (present)
cpumask_set_cpu(cpu, &__cpu_present_mask);
else
cpumask_clear_cpu(cpu, &__cpu_present_mask);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
set_cpu_online(unsigned int cpu, bool online)
{
if (online)
cpumask_set_cpu(cpu, &__cpu_online_mask);
else
cpumask_clear_cpu(cpu, &__cpu_online_mask);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
set_cpu_active(unsigned int cpu, bool active)
{
if (active)
cpumask_set_cpu(cpu, &__cpu_active_mask);
else
cpumask_clear_cpu(cpu, &__cpu_active_mask);
}
# 814 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __check_is_bitmap(const unsigned long *bitmap)
{
return 1;
}
# 826 "./include/linux/cpumask.h"
extern const unsigned long
cpu_bit_bitmap[32 +1][(((4) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8)))];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const struct cpumask *get_cpu_mask(unsigned int cpu)
{
const unsigned long *p = cpu_bit_bitmap[1 + cpu % 32];
p -= cpu / 32;
return ((struct cpumask *)(1 ? (p) : (void *)sizeof(__check_is_bitmap(p))));
}
# 863 "./include/linux/cpumask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ssize_t
cpumap_print_to_pagebuf(bool list, char *buf, const struct cpumask *mask)
{
return bitmap_print_to_pagebuf(list, buf, ((mask)->bits),
nr_cpu_ids);
}
# 45 "./include/linux/rcupdate.h" 2
void call_rcu(struct callback_head *head, rcu_callback_t func);
void call_rcu_bh(struct callback_head *head, rcu_callback_t func);
void call_rcu_sched(struct callback_head *head, rcu_callback_t func);
void synchronize_sched(void);
void rcu_barrier_tasks(void);
void __rcu_read_lock(void);
void __rcu_read_unlock(void);
void rcu_read_unlock_special(struct task_struct *t);
void synchronize_rcu(void);
# 103 "./include/linux/rcupdate.h"
void rcu_init(void);
extern int rcu_scheduler_active __attribute__((__section__(".data..read_mostly")));
void rcu_sched_qs(void);
void rcu_bh_qs(void);
void rcu_check_callbacks(int user);
void rcu_report_dead(unsigned int cpu);
void rcu_cpu_starting(unsigned int cpu);
void rcutree_migrate_callbacks(int cpu);
void rcu_sysrq_start(void);
void rcu_sysrq_end(void);
# 124 "./include/linux/rcupdate.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_user_enter(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_user_exit(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_init_nohz(void) { }
# 175 "./include/linux/rcupdate.h"
void call_rcu_tasks(struct callback_head *head, rcu_callback_t func);
void synchronize_rcu_tasks(void);
void exit_tasks_rcu_start(void);
void exit_tasks_rcu_finish(void);
# 207 "./include/linux/rcupdate.h"
# 1 "./include/linux/rcutree.h" 1
# 33 "./include/linux/rcutree.h"
void rcu_note_context_switch(bool preempt);
int rcu_needs_cpu(u64 basem, u64 *nextevt);
void rcu_cpu_stall_reset(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_virt_note_context_switch(int cpu)
{
rcu_note_context_switch(false);
}
void synchronize_rcu_bh(void);
void synchronize_sched_expedited(void);
void synchronize_rcu_expedited(void);
void kfree_call_rcu(struct callback_head *head, rcu_callback_t func);
# 69 "./include/linux/rcutree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void synchronize_rcu_bh_expedited(void)
{
synchronize_sched_expedited();
}
void rcu_barrier(void);
void rcu_barrier_bh(void);
void rcu_barrier_sched(void);
unsigned long get_state_synchronize_rcu(void);
void cond_synchronize_rcu(unsigned long oldstate);
unsigned long get_state_synchronize_sched(void);
void cond_synchronize_sched(unsigned long oldstate);
void rcu_idle_enter(void);
void rcu_idle_exit(void);
void rcu_irq_enter(void);
void rcu_irq_exit(void);
void rcu_irq_enter_irqson(void);
void rcu_irq_exit_irqson(void);
bool rcu_irq_enter_disabled(void);
void exit_rcu(void);
void rcu_scheduler_starting(void);
extern int rcu_scheduler_active __attribute__((__section__(".data..read_mostly")));
void rcu_end_inkernel_boot(void);
bool rcu_is_watching(void);
void rcu_all_qs(void);
int rcutree_prepare_cpu(unsigned int cpu);
int rcutree_online_cpu(unsigned int cpu);
int rcutree_offline_cpu(unsigned int cpu);
int rcutree_dead_cpu(unsigned int cpu);
int rcutree_dying_cpu(unsigned int cpu);
# 208 "./include/linux/rcupdate.h" 2
# 226 "./include/linux/rcupdate.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_rcu_head(struct callback_head *head) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void destroy_rcu_head(struct callback_head *head) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_rcu_head_on_stack(struct callback_head *head) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void destroy_rcu_head_on_stack(struct callback_head *head) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rcu_lockdep_current_cpu_online(void) { return true; }
# 264 "./include/linux/rcupdate.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rcu_read_lock_held(void)
{
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rcu_read_lock_bh_held(void)
{
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rcu_read_lock_sched_held(void)
{
return !(preempt_count() == 0 && !({ unsigned long _flags; do { ({ unsigned long __dummy; typeof(_flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); _flags = arch_local_save_flags(); } while (0); ({ ({ unsigned long __dummy; typeof(_flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_irqs_disabled_flags(_flags); }); }));
}
# 625 "./include/linux/rcupdate.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void rcu_read_lock(void)
{
__rcu_read_lock();
(void)0;
do { } while (0);
do { } while (0);
}
# 679 "./include/linux/rcupdate.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_read_unlock(void)
{
do { } while (0);
(void)0;
__rcu_read_unlock();
do { } while (0);
}
# 705 "./include/linux/rcupdate.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_read_lock_bh(void)
{
local_bh_disable();
(void)0;
do { } while (0);
do { } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_read_unlock_bh(void)
{
do { } while (0);
do { } while (0);
(void)0;
local_bh_enable();
}
# 741 "./include/linux/rcupdate.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_read_lock_sched(void)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
(void)0;
do { } while (0);
do { } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((no_instrument_function)) void rcu_read_lock_sched_notrace(void)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
(void)0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcu_read_unlock_sched(void)
{
do { } while (0);
do { } while (0);
(void)0;
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((no_instrument_function)) void rcu_read_unlock_sched_notrace(void)
{
(void)0;
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule_notrace(); } while (0);
}
# 12 "./include/linux/rculist.h" 2
# 31 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void INIT_LIST_HEAD_RCU(struct list_head *list)
{
({ union { typeof(list->next) __val; char __c[1]; } __u = { .__val = ( typeof(list->next)) (list) }; __write_once_size(&(list->next), __u.__c, sizeof(list->next)); __u.__val; });
({ union { typeof(list->prev) __val; char __c[1]; } __u = { .__val = ( typeof(list->prev)) (list) }; __write_once_size(&(list->prev), __u.__c, sizeof(list->prev)); __u.__val; });
}
# 49 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __list_add_rcu(struct list_head *new,
struct list_head *prev, struct list_head *next)
{
if (!__list_add_valid(new, prev, next))
return;
new->next = next;
new->prev = prev;
({ uintptr_t _r_a_p__v = (uintptr_t)(new); if (__builtin_constant_p(new) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof(((*((struct list_head **)(&(prev)->next))))) __val; char __c[1]; } __u = { .__val = ( typeof(((*((struct list_head **)(&(prev)->next)))))) ((typeof((*((struct list_head **)(&(prev)->next)))))(_r_a_p__v)) }; __write_once_size(&(((*((struct list_head **)(&(prev)->next))))), __u.__c, sizeof(((*((struct list_head **)(&(prev)->next)))))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(char) || sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(short) || sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(int) || sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(long))); extern void __compiletime_assert_2(void) ; if (__cond) __compiletime_assert_2(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(*((struct list_head **)(&(prev)->next)))) __val; char __c[1]; } __u = { .__val = ( typeof(*&(*((struct list_head **)(&(prev)->next))))) ((typeof(*((typeof((*((struct list_head **)(&(prev)->next)))))_r_a_p__v)) *)((typeof((*((struct list_head **)(&(prev)->next)))))_r_a_p__v)) }; __write_once_size(&(*&(*((struct list_head **)(&(prev)->next)))), __u.__c, sizeof(*&(*((struct list_head **)(&(prev)->next))))); __u.__val; }); } while (0); _r_a_p__v; });
next->prev = new;
}
# 77 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_add_rcu(struct list_head *new, struct list_head *head)
{
__list_add_rcu(new, head, head->next);
}
# 98 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_add_tail_rcu(struct list_head *new,
struct list_head *head)
{
__list_add_rcu(new, head->prev, head);
}
# 128 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_del_rcu(struct list_head *entry)
{
__list_del_entry(entry);
entry->prev = ((void *) 0x200 + 0);
}
# 154 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_del_init_rcu(struct hlist_node *n)
{
if (!hlist_unhashed(n)) {
__hlist_del(n);
n->pprev = ((void *)0);
}
}
# 170 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_replace_rcu(struct list_head *old,
struct list_head *new)
{
new->next = old->next;
new->prev = old->prev;
({ uintptr_t _r_a_p__v = (uintptr_t)(new); if (__builtin_constant_p(new) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof(((*((struct list_head **)(&(new->prev)->next))))) __val; char __c[1]; } __u = { .__val = ( typeof(((*((struct list_head **)(&(new->prev)->next)))))) ((typeof((*((struct list_head **)(&(new->prev)->next)))))(_r_a_p__v)) }; __write_once_size(&(((*((struct list_head **)(&(new->prev)->next))))), __u.__c, sizeof(((*((struct list_head **)(&(new->prev)->next)))))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&(*((struct list_head **)(&(new->prev)->next)))) == sizeof(char) || sizeof(*&(*((struct list_head **)(&(new->prev)->next)))) == sizeof(short) || sizeof(*&(*((struct list_head **)(&(new->prev)->next)))) == sizeof(int) || sizeof(*&(*((struct list_head **)(&(new->prev)->next)))) == sizeof(long))); extern void __compiletime_assert_3(void) ; if (__cond) __compiletime_assert_3(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(*((struct list_head **)(&(new->prev)->next)))) __val; char __c[1]; } __u = { .__val = ( typeof(*&(*((struct list_head **)(&(new->prev)->next))))) ((typeof(*((typeof((*((struct list_head **)(&(new->prev)->next)))))_r_a_p__v)) *)((typeof((*((struct list_head **)(&(new->prev)->next)))))_r_a_p__v)) }; __write_once_size(&(*&(*((struct list_head **)(&(new->prev)->next)))), __u.__c, sizeof(*&(*((struct list_head **)(&(new->prev)->next))))); __u.__val; }); } while (0); _r_a_p__v; });
new->next->prev = new;
old->prev = ((void *) 0x200 + 0);
}
# 199 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __list_splice_init_rcu(struct list_head *list,
struct list_head *prev,
struct list_head *next,
void (*sync)(void))
{
struct list_head *first = list->next;
struct list_head *last = list->prev;
INIT_LIST_HEAD_RCU(list);
# 222 "./include/linux/rculist.h"
sync();
# 232 "./include/linux/rculist.h"
last->next = next;
({ uintptr_t _r_a_p__v = (uintptr_t)(first); if (__builtin_constant_p(first) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof(((*((struct list_head **)(&(prev)->next))))) __val; char __c[1]; } __u = { .__val = ( typeof(((*((struct list_head **)(&(prev)->next)))))) ((typeof((*((struct list_head **)(&(prev)->next)))))(_r_a_p__v)) }; __write_once_size(&(((*((struct list_head **)(&(prev)->next))))), __u.__c, sizeof(((*((struct list_head **)(&(prev)->next)))))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(char) || sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(short) || sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(int) || sizeof(*&(*((struct list_head **)(&(prev)->next)))) == sizeof(long))); extern void __compiletime_assert_4(void) ; if (__cond) __compiletime_assert_4(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(*((struct list_head **)(&(prev)->next)))) __val; char __c[1]; } __u = { .__val = ( typeof(*&(*((struct list_head **)(&(prev)->next))))) ((typeof(*((typeof((*((struct list_head **)(&(prev)->next)))))_r_a_p__v)) *)((typeof((*((struct list_head **)(&(prev)->next)))))_r_a_p__v)) }; __write_once_size(&(*&(*((struct list_head **)(&(prev)->next)))), __u.__c, sizeof(*&(*((struct list_head **)(&(prev)->next))))); __u.__val; }); } while (0); _r_a_p__v; });
first->prev = prev;
next->prev = last;
}
# 245 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_splice_init_rcu(struct list_head *list,
struct list_head *head,
void (*sync)(void))
{
if (!list_empty(list))
__list_splice_init_rcu(list, head, head->next, sync);
}
# 260 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void list_splice_tail_init_rcu(struct list_head *list,
struct list_head *head,
void (*sync)(void))
{
if (!list_empty(list))
__list_splice_init_rcu(list, head->prev, head, sync);
}
# 425 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_del_rcu(struct hlist_node *n)
{
__hlist_del(n);
n->pprev = ((void *) 0x200 + 0);
}
# 438 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_replace_rcu(struct hlist_node *old,
struct hlist_node *new)
{
struct hlist_node *next = old->next;
new->next = next;
new->pprev = old->pprev;
({ uintptr_t _r_a_p__v = (uintptr_t)(new); if (__builtin_constant_p(new) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((*(struct hlist_node **)new->pprev)) __val; char __c[1]; } __u = { .__val = ( typeof((*(struct hlist_node **)new->pprev))) ((typeof(*(struct hlist_node **)new->pprev))(_r_a_p__v)) }; __write_once_size(&((*(struct hlist_node **)new->pprev)), __u.__c, sizeof((*(struct hlist_node **)new->pprev))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&*(struct hlist_node **)new->pprev) == sizeof(char) || sizeof(*&*(struct hlist_node **)new->pprev) == sizeof(short) || sizeof(*&*(struct hlist_node **)new->pprev) == sizeof(int) || sizeof(*&*(struct hlist_node **)new->pprev) == sizeof(long))); extern void __compiletime_assert_5(void) ; if (__cond) __compiletime_assert_5(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&*(struct hlist_node **)new->pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*&*(struct hlist_node **)new->pprev)) ((typeof(*((typeof(*(struct hlist_node **)new->pprev))_r_a_p__v)) *)((typeof(*(struct hlist_node **)new->pprev))_r_a_p__v)) }; __write_once_size(&(*&*(struct hlist_node **)new->pprev), __u.__c, sizeof(*&*(struct hlist_node **)new->pprev)); __u.__val; }); } while (0); _r_a_p__v; });
if (next)
new->next->pprev = &new->next;
old->pprev = ((void *) 0x200 + 0);
}
# 477 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_head_rcu(struct hlist_node *n,
struct hlist_head *h)
{
struct hlist_node *first = h->first;
n->next = first;
n->pprev = &h->first;
({ uintptr_t _r_a_p__v = (uintptr_t)(n); if (__builtin_constant_p(n) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof(((*((struct hlist_node **)(&(h)->first))))) __val; char __c[1]; } __u = { .__val = ( typeof(((*((struct hlist_node **)(&(h)->first)))))) ((typeof((*((struct hlist_node **)(&(h)->first)))))(_r_a_p__v)) }; __write_once_size(&(((*((struct hlist_node **)(&(h)->first))))), __u.__c, sizeof(((*((struct hlist_node **)(&(h)->first)))))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&(*((struct hlist_node **)(&(h)->first)))) == sizeof(char) || sizeof(*&(*((struct hlist_node **)(&(h)->first)))) == sizeof(short) || sizeof(*&(*((struct hlist_node **)(&(h)->first)))) == sizeof(int) || sizeof(*&(*((struct hlist_node **)(&(h)->first)))) == sizeof(long))); extern void __compiletime_assert_6(void) ; if (__cond) __compiletime_assert_6(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(*((struct hlist_node **)(&(h)->first)))) __val; char __c[1]; } __u = { .__val = ( typeof(*&(*((struct hlist_node **)(&(h)->first))))) ((typeof(*((typeof((*((struct hlist_node **)(&(h)->first)))))_r_a_p__v)) *)((typeof((*((struct hlist_node **)(&(h)->first)))))_r_a_p__v)) }; __write_once_size(&(*&(*((struct hlist_node **)(&(h)->first)))), __u.__c, sizeof(*&(*((struct hlist_node **)(&(h)->first))))); __u.__val; }); } while (0); _r_a_p__v; });
if (first)
first->pprev = &n->next;
}
# 508 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_tail_rcu(struct hlist_node *n,
struct hlist_head *h)
{
struct hlist_node *i, *last = ((void *)0);
for (i = h->first; i; i = i->next)
last = i;
if (last) {
n->next = last->next;
n->pprev = &last->next;
({ uintptr_t _r_a_p__v = (uintptr_t)(n); if (__builtin_constant_p(n) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof(((*((struct hlist_node **)(&(last)->next))))) __val; char __c[1]; } __u = { .__val = ( typeof(((*((struct hlist_node **)(&(last)->next)))))) ((typeof((*((struct hlist_node **)(&(last)->next)))))(_r_a_p__v)) }; __write_once_size(&(((*((struct hlist_node **)(&(last)->next))))), __u.__c, sizeof(((*((struct hlist_node **)(&(last)->next)))))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&(*((struct hlist_node **)(&(last)->next)))) == sizeof(char) || sizeof(*&(*((struct hlist_node **)(&(last)->next)))) == sizeof(short) || sizeof(*&(*((struct hlist_node **)(&(last)->next)))) == sizeof(int) || sizeof(*&(*((struct hlist_node **)(&(last)->next)))) == sizeof(long))); extern void __compiletime_assert_7(void) ; if (__cond) __compiletime_assert_7(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(*((struct hlist_node **)(&(last)->next)))) __val; char __c[1]; } __u = { .__val = ( typeof(*&(*((struct hlist_node **)(&(last)->next))))) ((typeof(*((typeof((*((struct hlist_node **)(&(last)->next)))))_r_a_p__v)) *)((typeof((*((struct hlist_node **)(&(last)->next)))))_r_a_p__v)) }; __write_once_size(&(*&(*((struct hlist_node **)(&(last)->next)))), __u.__c, sizeof(*&(*((struct hlist_node **)(&(last)->next))))); __u.__val; }); } while (0); _r_a_p__v; });
} else {
hlist_add_head_rcu(n, h);
}
}
# 544 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_before_rcu(struct hlist_node *n,
struct hlist_node *next)
{
n->pprev = next->pprev;
n->next = next;
({ uintptr_t _r_a_p__v = (uintptr_t)(n); if (__builtin_constant_p(n) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof(((*((struct hlist_node **)((n)->pprev))))) __val; char __c[1]; } __u = { .__val = ( typeof(((*((struct hlist_node **)((n)->pprev)))))) ((typeof((*((struct hlist_node **)((n)->pprev)))))(_r_a_p__v)) }; __write_once_size(&(((*((struct hlist_node **)((n)->pprev))))), __u.__c, sizeof(((*((struct hlist_node **)((n)->pprev)))))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&(*((struct hlist_node **)((n)->pprev)))) == sizeof(char) || sizeof(*&(*((struct hlist_node **)((n)->pprev)))) == sizeof(short) || sizeof(*&(*((struct hlist_node **)((n)->pprev)))) == sizeof(int) || sizeof(*&(*((struct hlist_node **)((n)->pprev)))) == sizeof(long))); extern void __compiletime_assert_8(void) ; if (__cond) __compiletime_assert_8(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(*((struct hlist_node **)((n)->pprev)))) __val; char __c[1]; } __u = { .__val = ( typeof(*&(*((struct hlist_node **)((n)->pprev))))) ((typeof(*((typeof((*((struct hlist_node **)((n)->pprev)))))_r_a_p__v)) *)((typeof((*((struct hlist_node **)((n)->pprev)))))_r_a_p__v)) }; __write_once_size(&(*&(*((struct hlist_node **)((n)->pprev)))), __u.__c, sizeof(*&(*((struct hlist_node **)((n)->pprev))))); __u.__val; }); } while (0); _r_a_p__v; });
next->pprev = &n->next;
}
# 571 "./include/linux/rculist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_add_behind_rcu(struct hlist_node *n,
struct hlist_node *prev)
{
n->next = prev->next;
n->pprev = &prev->next;
({ uintptr_t _r_a_p__v = (uintptr_t)(n); if (__builtin_constant_p(n) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof(((*((struct hlist_node **)(&(prev)->next))))) __val; char __c[1]; } __u = { .__val = ( typeof(((*((struct hlist_node **)(&(prev)->next)))))) ((typeof((*((struct hlist_node **)(&(prev)->next)))))(_r_a_p__v)) }; __write_once_size(&(((*((struct hlist_node **)(&(prev)->next))))), __u.__c, sizeof(((*((struct hlist_node **)(&(prev)->next)))))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&(*((struct hlist_node **)(&(prev)->next)))) == sizeof(char) || sizeof(*&(*((struct hlist_node **)(&(prev)->next)))) == sizeof(short) || sizeof(*&(*((struct hlist_node **)(&(prev)->next)))) == sizeof(int) || sizeof(*&(*((struct hlist_node **)(&(prev)->next)))) == sizeof(long))); extern void __compiletime_assert_9(void) ; if (__cond) __compiletime_assert_9(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&(*((struct hlist_node **)(&(prev)->next)))) __val; char __c[1]; } __u = { .__val = ( typeof(*&(*((struct hlist_node **)(&(prev)->next))))) ((typeof(*((typeof((*((struct hlist_node **)(&(prev)->next)))))_r_a_p__v)) *)((typeof((*((struct hlist_node **)(&(prev)->next)))))_r_a_p__v)) }; __write_once_size(&(*&(*((struct hlist_node **)(&(prev)->next)))), __u.__c, sizeof(*&(*((struct hlist_node **)(&(prev)->next))))); __u.__val; }); } while (0); _r_a_p__v; });
if (n->next)
n->next->pprev = &n->next;
}
# 6 "./include/linux/pid.h" 2
# 1 "./include/linux/wait.h" 1
# 1 "./include/linux/spinlock.h" 1
# 82 "./include/linux/spinlock.h"
# 1 "./include/linux/spinlock_types.h" 1
# 13 "./include/linux/spinlock_types.h"
# 1 "./arch/arm/include/asm/spinlock_types.h" 1
# 11 "./arch/arm/include/asm/spinlock_types.h"
typedef struct {
union {
u32 slock;
struct __raw_tickets {
u16 owner;
u16 next;
} tickets;
};
} arch_spinlock_t;
typedef struct {
u32 lock;
} arch_rwlock_t;
# 14 "./include/linux/spinlock_types.h" 2
typedef struct raw_spinlock {
arch_spinlock_t raw_lock;
# 32 "./include/linux/spinlock_types.h"
} raw_spinlock_t;
# 64 "./include/linux/spinlock_types.h"
typedef struct spinlock {
union {
struct raw_spinlock rlock;
# 75 "./include/linux/spinlock_types.h"
};
} spinlock_t;
# 86 "./include/linux/spinlock_types.h"
# 1 "./include/linux/rwlock_types.h" 1
# 11 "./include/linux/rwlock_types.h"
typedef struct {
arch_rwlock_t raw_lock;
# 23 "./include/linux/rwlock_types.h"
} rwlock_t;
# 87 "./include/linux/spinlock_types.h" 2
# 83 "./include/linux/spinlock.h" 2
# 1 "./arch/arm/include/asm/spinlock.h" 1
# 41 "./arch/arm/include/asm/spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dsb_sev(void)
{
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
__asm__("9998: " "sev" "\n" " .pushsection \".alt.smp.init\", \"a\"\n" " .long 9998b\n" " " "nop" "\n" " .popsection\n");
}
# 58 "./arch/arm/include/asm/spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_spin_lock(arch_spinlock_t *lock)
{
unsigned long tmp;
u32 newval;
arch_spinlock_t lockval;
prefetchw(&lock->slock);
__asm__ __volatile__(
"1: ldrex %0, [%3]\n"
" add %1, %0, %4\n"
" strex %2, %1, [%3]\n"
" teq %2, #0\n"
" bne 1b"
: "=&r" (lockval), "=&r" (newval), "=&r" (tmp)
: "r" (&lock->slock), "I" (1 << 16)
: "cc");
while (lockval.tickets.next != lockval.tickets.owner) {
__asm__ __volatile__ ("wfe" : : : "memory");
lockval.tickets.owner = (*({ __attribute__((unused)) typeof(lock->tickets.owner) __var = ( typeof(lock->tickets.owner)) 0; (volatile typeof(lock->tickets.owner) *)&(lock->tickets.owner); }));
}
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_spin_trylock(arch_spinlock_t *lock)
{
unsigned long contended, res;
u32 slock;
prefetchw(&lock->slock);
do {
__asm__ __volatile__(
" ldrex %0, [%3]\n"
" mov %2, #0\n"
" subs %1, %0, %0, ror #16\n"
" addeq %0, %0, %4\n"
" strexeq %2, %0, [%3]"
: "=&r" (slock), "=&r" (contended), "=&r" (res)
: "r" (&lock->slock), "I" (1 << 16)
: "cc");
} while (res);
if (!contended) {
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return 1;
} else {
return 0;
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_spin_unlock(arch_spinlock_t *lock)
{
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
lock->tickets.owner++;
dsb_sev();
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_spin_value_unlocked(arch_spinlock_t lock)
{
return lock.tickets.owner == lock.tickets.next;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_spin_is_locked(arch_spinlock_t *lock)
{
return !arch_spin_value_unlocked(({ union { typeof(*lock) __val; char __c[1]; } __u; if (1) __read_once_size(&(*lock), __u.__c, sizeof(*lock)); else __read_once_size_nocheck(&(*lock), __u.__c, sizeof(*lock)); do { } while (0); __u.__val; }));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_spin_is_contended(arch_spinlock_t *lock)
{
struct __raw_tickets tickets = ({ union { typeof(lock->tickets) __val; char __c[1]; } __u; if (1) __read_once_size(&(lock->tickets), __u.__c, sizeof(lock->tickets)); else __read_once_size_nocheck(&(lock->tickets), __u.__c, sizeof(lock->tickets)); do { } while (0); __u.__val; });
return (tickets.next - tickets.owner) > 1;
}
# 141 "./arch/arm/include/asm/spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_write_lock(arch_rwlock_t *rw)
{
unsigned long tmp;
prefetchw(&rw->lock);
__asm__ __volatile__(
"1: ldrex %0, [%1]\n"
" teq %0, #0\n"
"9998: " "wfe" "ne" "\n" " .pushsection \".alt.smp.init\", \"a\"\n" " .long 9998b\n" " " "nop" "\n" " .popsection\n"
" strexeq %0, %2, [%1]\n"
" teq %0, #0\n"
" bne 1b"
: "=&r" (tmp)
: "r" (&rw->lock), "r" (0x80000000)
: "cc");
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_write_trylock(arch_rwlock_t *rw)
{
unsigned long contended, res;
prefetchw(&rw->lock);
do {
__asm__ __volatile__(
" ldrex %0, [%2]\n"
" mov %1, #0\n"
" teq %0, #0\n"
" strexeq %1, %3, [%2]"
: "=&r" (contended), "=&r" (res)
: "r" (&rw->lock), "r" (0x80000000)
: "cc");
} while (res);
if (!contended) {
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return 1;
} else {
return 0;
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_write_unlock(arch_rwlock_t *rw)
{
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
__asm__ __volatile__(
"str %1, [%0]\n"
:
: "r" (&rw->lock), "r" (0)
: "cc");
dsb_sev();
}
# 212 "./arch/arm/include/asm/spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_read_lock(arch_rwlock_t *rw)
{
unsigned long tmp, tmp2;
prefetchw(&rw->lock);
__asm__ __volatile__(
"1: ldrex %0, [%2]\n"
" adds %0, %0, #1\n"
" strexpl %1, %0, [%2]\n"
"9998: " "wfe" "mi" "\n" " .pushsection \".alt.smp.init\", \"a\"\n" " .long 9998b\n" " " "nop" "\n" " .popsection\n"
" rsbpls %0, %1, #0\n"
" bmi 1b"
: "=&r" (tmp), "=&r" (tmp2)
: "r" (&rw->lock)
: "cc");
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_read_unlock(arch_rwlock_t *rw)
{
unsigned long tmp, tmp2;
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
prefetchw(&rw->lock);
__asm__ __volatile__(
"1: ldrex %0, [%2]\n"
" sub %0, %0, #1\n"
" strex %1, %0, [%2]\n"
" teq %1, #0\n"
" bne 1b"
: "=&r" (tmp), "=&r" (tmp2)
: "r" (&rw->lock)
: "cc");
if (tmp == 0)
dsb_sev();
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int arch_read_trylock(arch_rwlock_t *rw)
{
unsigned long contended, res;
prefetchw(&rw->lock);
do {
__asm__ __volatile__(
" ldrex %0, [%2]\n"
" mov %1, #0\n"
" adds %0, %0, #1\n"
" strexpl %1, %0, [%2]"
: "=&r" (contended), "=&r" (res)
: "r" (&rw->lock)
: "cc");
} while (res);
if (contended < 0x80000000) {
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return 1;
} else {
return 0;
}
}
# 89 "./include/linux/spinlock.h" 2
# 163 "./include/linux/spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void do_raw_spin_lock(raw_spinlock_t *lock)
{
(void)0;
arch_spin_lock(&lock->raw_lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
do_raw_spin_lock_flags(raw_spinlock_t *lock, unsigned long *flags)
{
(void)0;
arch_spin_lock(&lock->raw_lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int do_raw_spin_trylock(raw_spinlock_t *lock)
{
return arch_spin_trylock(&(lock)->raw_lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void do_raw_spin_unlock(raw_spinlock_t *lock)
{
arch_spin_unlock(&lock->raw_lock);
(void)0;
}
# 289 "./include/linux/spinlock.h"
# 1 "./include/linux/rwlock.h" 1
# 290 "./include/linux/spinlock.h" 2
# 1 "./include/linux/spinlock_api_smp.h" 1
# 18 "./include/linux/spinlock_api_smp.h"
int in_lock_functions(unsigned long addr);
void __attribute__((section(".spinlock.text"))) _raw_spin_lock(raw_spinlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_spin_lock_nested(raw_spinlock_t *lock, int subclass)
;
void __attribute__((section(".spinlock.text")))
_raw_spin_lock_nest_lock(raw_spinlock_t *lock, struct lockdep_map *map)
;
void __attribute__((section(".spinlock.text"))) _raw_spin_lock_bh(raw_spinlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_spin_lock_irq(raw_spinlock_t *lock)
;
unsigned long __attribute__((section(".spinlock.text"))) _raw_spin_lock_irqsave(raw_spinlock_t *lock)
;
unsigned long __attribute__((section(".spinlock.text")))
_raw_spin_lock_irqsave_nested(raw_spinlock_t *lock, int subclass)
;
int __attribute__((section(".spinlock.text"))) _raw_spin_trylock(raw_spinlock_t *lock);
int __attribute__((section(".spinlock.text"))) _raw_spin_trylock_bh(raw_spinlock_t *lock);
void __attribute__((section(".spinlock.text"))) _raw_spin_unlock(raw_spinlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_spin_unlock_bh(raw_spinlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_spin_unlock_irq(raw_spinlock_t *lock) ;
void __attribute__((section(".spinlock.text")))
_raw_spin_unlock_irqrestore(raw_spinlock_t *lock, unsigned long flags)
;
# 86 "./include/linux/spinlock_api_smp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __raw_spin_trylock(raw_spinlock_t *lock)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
if (do_raw_spin_trylock(lock)) {
do { } while (0);
return 1;
}
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
return 0;
}
# 104 "./include/linux/spinlock_api_smp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __raw_spin_lock_irqsave(raw_spinlock_t *lock)
{
unsigned long flags;
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0); } while (0);
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
# 119 "./include/linux/spinlock_api_smp.h"
do_raw_spin_lock_flags(lock, &flags);
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_spin_lock_irq(raw_spinlock_t *lock)
{
do { arch_local_irq_disable(); } while (0);
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do_raw_spin_lock(lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_spin_lock_bh(raw_spinlock_t *lock)
{
__local_bh_disable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
do { } while (0);
do_raw_spin_lock(lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_spin_lock(raw_spinlock_t *lock)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do_raw_spin_lock(lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_spin_unlock(raw_spinlock_t *lock)
{
do { } while (0);
do_raw_spin_unlock(lock);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_spin_unlock_irqrestore(raw_spinlock_t *lock,
unsigned long flags)
{
do { } while (0);
do_raw_spin_unlock(lock);
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_spin_unlock_irq(raw_spinlock_t *lock)
{
do { } while (0);
do_raw_spin_unlock(lock);
do { arch_local_irq_enable(); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_spin_unlock_bh(raw_spinlock_t *lock)
{
do { } while (0);
do_raw_spin_unlock(lock);
__local_bh_enable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __raw_spin_trylock_bh(raw_spinlock_t *lock)
{
__local_bh_disable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
if (do_raw_spin_trylock(lock)) {
do { } while (0);
return 1;
}
__local_bh_enable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
return 0;
}
# 1 "./include/linux/rwlock_api_smp.h" 1
# 18 "./include/linux/rwlock_api_smp.h"
void __attribute__((section(".spinlock.text"))) _raw_read_lock(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_write_lock(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_read_lock_bh(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_write_lock_bh(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_read_lock_irq(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_write_lock_irq(rwlock_t *lock) ;
unsigned long __attribute__((section(".spinlock.text"))) _raw_read_lock_irqsave(rwlock_t *lock)
;
unsigned long __attribute__((section(".spinlock.text"))) _raw_write_lock_irqsave(rwlock_t *lock)
;
int __attribute__((section(".spinlock.text"))) _raw_read_trylock(rwlock_t *lock);
int __attribute__((section(".spinlock.text"))) _raw_write_trylock(rwlock_t *lock);
void __attribute__((section(".spinlock.text"))) _raw_read_unlock(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_write_unlock(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_read_unlock_bh(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_write_unlock_bh(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_read_unlock_irq(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text"))) _raw_write_unlock_irq(rwlock_t *lock) ;
void __attribute__((section(".spinlock.text")))
_raw_read_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
;
void __attribute__((section(".spinlock.text")))
_raw_write_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
;
# 117 "./include/linux/rwlock_api_smp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __raw_read_trylock(rwlock_t *lock)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
if (arch_read_trylock(&(lock)->raw_lock)) {
do { } while (0);
return 1;
}
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __raw_write_trylock(rwlock_t *lock)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
if (arch_write_trylock(&(lock)->raw_lock)) {
do { } while (0);
return 1;
}
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
return 0;
}
# 146 "./include/linux/rwlock_api_smp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_read_lock(rwlock_t *lock)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do {(void)0; arch_read_lock(&(lock)->raw_lock); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __raw_read_lock_irqsave(rwlock_t *lock)
{
unsigned long flags;
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0); } while (0);
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do {(void)0; arch_read_lock(&((lock))->raw_lock); } while (0);
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_read_lock_irq(rwlock_t *lock)
{
do { arch_local_irq_disable(); } while (0);
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do {(void)0; arch_read_lock(&(lock)->raw_lock); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_read_lock_bh(rwlock_t *lock)
{
__local_bh_disable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
do { } while (0);
do {(void)0; arch_read_lock(&(lock)->raw_lock); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __raw_write_lock_irqsave(rwlock_t *lock)
{
unsigned long flags;
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0); } while (0);
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do {(void)0; arch_write_lock(&((lock))->raw_lock); } while (0);
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_write_lock_irq(rwlock_t *lock)
{
do { arch_local_irq_disable(); } while (0);
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do {(void)0; arch_write_lock(&(lock)->raw_lock); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_write_lock_bh(rwlock_t *lock)
{
__local_bh_disable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
do { } while (0);
do {(void)0; arch_write_lock(&(lock)->raw_lock); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_write_lock(rwlock_t *lock)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
do { } while (0);
do {(void)0; arch_write_lock(&(lock)->raw_lock); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_write_unlock(rwlock_t *lock)
{
do { } while (0);
do {arch_write_unlock(&(lock)->raw_lock); (void)0; } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_read_unlock(rwlock_t *lock)
{
do { } while (0);
do {arch_read_unlock(&(lock)->raw_lock); (void)0; } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
__raw_read_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
{
do { } while (0);
do {arch_read_unlock(&(lock)->raw_lock); (void)0; } while (0);
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_read_unlock_irq(rwlock_t *lock)
{
do { } while (0);
do {arch_read_unlock(&(lock)->raw_lock); (void)0; } while (0);
do { arch_local_irq_enable(); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_read_unlock_bh(rwlock_t *lock)
{
do { } while (0);
do {arch_read_unlock(&(lock)->raw_lock); (void)0; } while (0);
__local_bh_enable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_write_unlock_irqrestore(rwlock_t *lock,
unsigned long flags)
{
do { } while (0);
do {arch_write_unlock(&(lock)->raw_lock); (void)0; } while (0);
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_write_unlock_irq(rwlock_t *lock)
{
do { } while (0);
do {arch_write_unlock(&(lock)->raw_lock); (void)0; } while (0);
do { arch_local_irq_enable(); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_write_unlock_bh(rwlock_t *lock)
{
do { } while (0);
do {arch_write_unlock(&(lock)->raw_lock); (void)0; } while (0);
__local_bh_enable_ip((unsigned long)__builtin_return_address(0), ((2 * (1UL << (0 + 8))) + (1UL << 0)));
}
# 191 "./include/linux/spinlock_api_smp.h" 2
# 296 "./include/linux/spinlock.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) raw_spinlock_t *spinlock_check(spinlock_t *lock)
{
return &lock->rlock;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void spin_lock(spinlock_t *lock)
{
_raw_spin_lock(&lock->rlock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void spin_lock_bh(spinlock_t *lock)
{
_raw_spin_lock_bh(&lock->rlock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int spin_trylock(spinlock_t *lock)
{
return (_raw_spin_trylock(&lock->rlock));
}
# 340 "./include/linux/spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void spin_lock_irq(spinlock_t *lock)
{
_raw_spin_lock_irq(&lock->rlock);
}
# 355 "./include/linux/spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void spin_unlock(spinlock_t *lock)
{
_raw_spin_unlock(&lock->rlock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void spin_unlock_bh(spinlock_t *lock)
{
_raw_spin_unlock_bh(&lock->rlock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void spin_unlock_irq(spinlock_t *lock)
{
_raw_spin_unlock_irq(&lock->rlock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)
{
do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); _raw_spin_unlock_irqrestore(&lock->rlock, flags); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int spin_trylock_bh(spinlock_t *lock)
{
return (_raw_spin_trylock_bh(&lock->rlock));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int spin_trylock_irq(spinlock_t *lock)
{
return ({ do { arch_local_irq_disable(); } while (0); (_raw_spin_trylock(&lock->rlock)) ? 1 : ({ do { arch_local_irq_enable(); } while (0); 0; }); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int spin_is_locked(spinlock_t *lock)
{
return arch_spin_is_locked(&(&lock->rlock)->raw_lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int spin_is_contended(spinlock_t *lock)
{
return arch_spin_is_contended(&(&lock->rlock)->raw_lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int spin_can_lock(spinlock_t *lock)
{
return (!arch_spin_is_locked(&(&lock->rlock)->raw_lock));
}
# 420 "./include/linux/spinlock.h"
extern int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock);
extern int _atomic_dec_and_lock_irqsave(atomic_t *atomic, spinlock_t *lock,
unsigned long *flags);
# 10 "./include/linux/wait.h" 2
# 1 "./arch/arm/include/generated/asm/current.h" 1
# 12 "./include/linux/wait.h" 2
# 1 "./include/uapi/linux/wait.h" 1
# 13 "./include/linux/wait.h" 2
typedef struct wait_queue_entry wait_queue_entry_t;
typedef int (*wait_queue_func_t)(struct wait_queue_entry *wq_entry, unsigned mode, int flags, void *key);
int default_wake_function(struct wait_queue_entry *wq_entry, unsigned mode, int flags, void *key);
# 27 "./include/linux/wait.h"
struct wait_queue_entry {
unsigned int flags;
void *private;
wait_queue_func_t func;
struct list_head entry;
};
struct wait_queue_head {
spinlock_t lock;
struct list_head head;
};
typedef struct wait_queue_head wait_queue_head_t;
struct task_struct;
# 61 "./include/linux/wait.h"
extern void __init_waitqueue_head(struct wait_queue_head *wq_head, const char *name, struct lock_class_key *);
# 79 "./include/linux/wait.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_waitqueue_entry(struct wait_queue_entry *wq_entry, struct task_struct *p)
{
wq_entry->flags = 0;
wq_entry->private = p;
wq_entry->func = default_wake_function;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
init_waitqueue_func_entry(struct wait_queue_entry *wq_entry, wait_queue_func_t func)
{
wq_entry->flags = 0;
wq_entry->private = ((void *)0);
wq_entry->func = func;
}
# 124 "./include/linux/wait.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int waitqueue_active(struct wait_queue_head *wq_head)
{
return !list_empty(&wq_head->head);
}
# 137 "./include/linux/wait.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool wq_has_sleeper(struct wait_queue_head *wq_head)
{
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return waitqueue_active(wq_head);
}
extern void add_wait_queue(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry);
extern void add_wait_queue_exclusive(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry);
extern void remove_wait_queue(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __add_wait_queue(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry)
{
list_add(&wq_entry->entry, &wq_head->head);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
__add_wait_queue_exclusive(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry)
{
wq_entry->flags |= 0x01;
__add_wait_queue(wq_head, wq_entry);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __add_wait_queue_entry_tail(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry)
{
list_add_tail(&wq_entry->entry, &wq_head->head);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
__add_wait_queue_entry_tail_exclusive(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry)
{
wq_entry->flags |= 0x01;
__add_wait_queue_entry_tail(wq_head, wq_entry);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
__remove_wait_queue(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry)
{
list_del(&wq_entry->entry);
}
void __wake_up(struct wait_queue_head *wq_head, unsigned int mode, int nr, void *key);
void __wake_up_locked_key(struct wait_queue_head *wq_head, unsigned int mode, void *key);
void __wake_up_locked_key_bookmark(struct wait_queue_head *wq_head,
unsigned int mode, void *key, wait_queue_entry_t *bookmark);
void __wake_up_sync_key(struct wait_queue_head *wq_head, unsigned int mode, int nr, void *key);
void __wake_up_locked(struct wait_queue_head *wq_head, unsigned int mode, int nr);
void __wake_up_sync(struct wait_queue_head *wq_head, unsigned int mode, int nr);
# 230 "./include/linux/wait.h"
extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags);
# 600 "./include/linux/wait.h"
extern int do_wait_intr(wait_queue_head_t *, wait_queue_entry_t *);
extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *);
# 984 "./include/linux/wait.h"
void prepare_to_wait(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state);
void prepare_to_wait_exclusive(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state);
long prepare_to_wait_event(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state);
void finish_wait(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry);
long wait_woken(struct wait_queue_entry *wq_entry, unsigned mode, long timeout);
int woken_wake_function(struct wait_queue_entry *wq_entry, unsigned mode, int sync, void *key);
int autoremove_wake_function(struct wait_queue_entry *wq_entry, unsigned mode, int sync, void *key);
# 7 "./include/linux/pid.h" 2
enum pid_type
{
PIDTYPE_PID,
PIDTYPE_PGID,
PIDTYPE_SID,
PIDTYPE_MAX,
__PIDTYPE_TGID
};
# 54 "./include/linux/pid.h"
struct upid {
int nr;
struct pid_namespace *ns;
struct hlist_node pid_chain;
};
struct pid
{
atomic_t count;
unsigned int level;
struct hlist_head tasks[PIDTYPE_MAX];
wait_queue_head_t wait_pidfd;
struct callback_head rcu;
struct upid numbers[1];
};
extern struct pid init_struct_pid;
struct pid_link
{
struct hlist_node node;
struct pid *pid;
};
extern const struct file_operations pidfd_fops;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pid *get_pid(struct pid *pid)
{
if (pid)
atomic_add(1, &pid->count);
return pid;
}
extern void put_pid(struct pid *pid);
extern struct task_struct *pid_task(struct pid *pid, enum pid_type);
extern struct task_struct *get_pid_task(struct pid *pid, enum pid_type);
extern struct pid *get_task_pid(struct task_struct *task, enum pid_type type);
extern void attach_pid(struct task_struct *task, enum pid_type);
extern void detach_pid(struct task_struct *task, enum pid_type);
extern void change_pid(struct task_struct *task, enum pid_type,
struct pid *pid);
extern void transfer_pid(struct task_struct *old, struct task_struct *new,
enum pid_type);
struct pid_namespace;
extern struct pid_namespace init_pid_ns;
# 118 "./include/linux/pid.h"
extern struct pid *find_pid_ns(int nr, struct pid_namespace *ns);
extern struct pid *find_vpid(int nr);
extern struct pid *find_get_pid(int nr);
extern struct pid *find_ge_pid(int nr, struct pid_namespace *);
int next_pidmap(struct pid_namespace *pid_ns, unsigned int last);
extern struct pid *alloc_pid(struct pid_namespace *ns);
extern void free_pid(struct pid *pid);
extern void disable_pid_allocation(struct pid_namespace *ns);
# 142 "./include/linux/pid.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pid_namespace *ns_of_pid(struct pid *pid)
{
struct pid_namespace *ns = ((void *)0);
if (pid)
ns = pid->numbers[pid->level].ns;
return ns;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_child_reaper(struct pid *pid)
{
return pid->numbers[pid->level].nr == 1;
}
# 172 "./include/linux/pid.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t pid_nr(struct pid *pid)
{
pid_t nr = 0;
if (pid)
nr = pid->numbers[0].nr;
return nr;
}
pid_t pid_nr_ns(struct pid *pid, struct pid_namespace *ns);
pid_t pid_vnr(struct pid *pid);
# 15 "./include/linux/sched.h" 2
# 1 "./include/linux/sem.h" 1
# 1 "./include/linux/time64.h" 1
# 1 "./include/uapi/linux/time.h" 1
# 10 "./include/uapi/linux/time.h"
struct timespec {
__kernel_time_t tv_sec;
long tv_nsec;
};
struct timeval {
__kernel_time_t tv_sec;
__kernel_suseconds_t tv_usec;
};
struct timezone {
int tz_minuteswest;
int tz_dsttime;
};
# 35 "./include/uapi/linux/time.h"
struct itimerspec {
struct timespec it_interval;
struct timespec it_value;
};
struct itimerval {
struct timeval it_interval;
struct timeval it_value;
};
# 6 "./include/linux/time64.h" 2
# 1 "./include/linux/math64.h" 1
# 1 "./arch/arm/include/asm/div64.h" 1
# 7 "./include/linux/math64.h" 2
# 65 "./include/linux/math64.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 div_u64_rem(u64 dividend, u32 divisor, u32 *remainder)
{
*remainder = ({ unsigned int __r, __b = (divisor); if (!__builtin_constant_p(__b) || __b == 0 || (7 < 4 && (__b & (__b - 1)) != 0)) { __r = ({ register unsigned int __base asm("r4") = __b; register unsigned long long __n asm("r0") = dividend; register unsigned long long __res asm("r2"); register unsigned int __rem asm("r1"); asm( ".ifnc " "%0" "," "r1" "; " ".ifnc " "%0" "r1" ",fpr11; " ".ifnc " "%0" "r1" ",r11fp; " ".ifnc " "%0" "r1" ",ipr12; " ".ifnc " "%0" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r0" "; " ".ifnc " "%2" "r0" ",fpr11; " ".ifnc " "%2" "r0" ",r11fp; " ".ifnc " "%2" "r0" ",ipr12; " ".ifnc " "%2" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r4" "; " ".ifnc " "%3" "r4" ",fpr11; " ".ifnc " "%3" "r4" ",r11fp; " ".ifnc " "%3" "r4" ",ipr12; " ".ifnc " "%3" "r4" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __do_div64" : "=r" (__rem), "=r" (__res) : "r" (__n), "r" (__base) : "ip", "lr", "cc"); dividend = __res; __rem; }); } else if ((__b & (__b - 1)) == 0) { __r = dividend; __r &= (__b - 1); dividend /= __b; } else { unsigned long long __res, __x, __t, __m, __n = dividend; unsigned int __c, __p, __z = 0; __r = __n; __p = 1 << ({ unsigned int __left = (__b), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m = (~0ULL / __b) * __p; __m += (((~0ULL % __b + 1) * __p) + __b - 1) / __b; __x = ~0ULL / __b * __b - 1; __res = (__m & 0xffffffff) * (__x & 0xffffffff); __res >>= 32; __res += (__m & 0xffffffff) * (__x >> 32); __t = __res; __res += (__x & 0xffffffff) * (__m >> 32); __t = (__res < __t) ? (1ULL << 32) : 0; __res = (__res >> 32) + __t; __res += (__m >> 32) * (__x >> 32); __res /= __p; if (~0ULL % (__b / (__b & -__b)) == 0) { __n /= (__b & -__b); __m = ~0ULL / (__b / (__b & -__b)); __p = 1; __c = 1; } else if (__res != __x / __b) { __c = 1; __m = (~0ULL / __b) * __p; __m += ((~0ULL % __b + 1) * __p) / __b; } else { unsigned int __bits = -(__m & -__m); __bits |= __m >> 32; __bits = (~__bits) << 1; if (!__bits) { __p /= (__m & -__m); __m /= (__m & -__m); } else { __p >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); } __c = 0; } if (!__c) { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "=&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { __res = __m; asm ( "umlal %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "cmn %Q0, %Q1\n\t" "adcs %R0, %R0, %R1\n\t" "adc %Q0, %3, #0" : "=&r" (__res) : "r" (__m), "r" (__n), "r" (__z) : "cc" ); } if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { asm ( "umlal %R0, %Q0, %R1, %Q2\n\t" "umlal %R0, %Q0, %Q1, %R2\n\t" "mov %R0, #0\n\t" "umlal %Q0, %R0, %R1, %R2" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umlal %R0, %Q0, %R2, %Q3\n\t" "umlal %R0, %1, %Q2, %R3\n\t" "mov %R0, #0\n\t" "adds %Q0, %1, %Q0\n\t" "adc %R0, %R0, #0\n\t" "umlal %Q0, %R0, %R2, %R3" : "+&r" (__res), "+&r" (__z) : "r" (__m), "r" (__n) : "cc" ); } __res /= __p; { unsigned int __res0 = __res; unsigned int __b0 = __b; __r -= __res0 * __b0; } dividend = __res; } __r; });
return dividend;
}
extern s64 div_s64_rem(s64 dividend, s32 divisor, s32 *remainder);
extern u64 div64_u64_rem(u64 dividend, u64 divisor, u64 *remainder);
extern u64 div64_u64(u64 dividend, u64 divisor);
extern s64 div64_s64(s64 dividend, s64 divisor);
# 98 "./include/linux/math64.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 div_u64(u64 dividend, u32 divisor)
{
u32 remainder;
return div_u64_rem(dividend, divisor, &remainder);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 div_s64(s64 dividend, s32 divisor)
{
s32 remainder;
return div_s64_rem(dividend, divisor, &remainder);
}
u32 iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remainder);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) u32
__iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remainder)
{
u32 ret = 0;
while (dividend >= divisor) {
asm("" : "+rm"(dividend));
dividend -= divisor;
ret++;
}
*remainder = dividend;
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 mul_u32_u32(u32 a, u32 b)
{
return (u64)a * b;
}
# 166 "./include/linux/math64.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 mul_u64_u32_shr(u64 a, u32 mul, unsigned int shift)
{
u32 ah, al;
u64 ret;
al = a;
ah = a >> 32;
ret = mul_u32_u32(al, mul) >> shift;
if (ah)
ret += mul_u32_u32(ah, mul) << (32 - shift);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 mul_u64_u64_shr(u64 a, u64 b, unsigned int shift)
{
union {
u64 ll;
struct {
u32 low, high;
} l;
} rl, rm, rn, rh, a0, b0;
u64 c;
a0.ll = a;
b0.ll = b;
rl.ll = mul_u32_u32(a0.l.low, b0.l.low);
rm.ll = mul_u32_u32(a0.l.low, b0.l.high);
rn.ll = mul_u32_u32(a0.l.high, b0.l.low);
rh.ll = mul_u32_u32(a0.l.high, b0.l.high);
rl.l.high = c = (u64)rl.l.high + rm.l.low + rn.l.low;
rh.l.low = c = (c >> 32) + rm.l.high + rn.l.high + rh.l.low;
rh.l.high = (c >> 32) + rh.l.high;
if (shift == 0)
return rl.ll;
if (shift < 64)
return (rl.ll >> shift) | (rh.ll << (64 - shift));
return rh.ll >> (shift & 63);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 mul_u64_u32_div(u64 a, u32 mul, u32 divisor)
{
union {
u64 ll;
struct {
u32 low, high;
} l;
} u, rl, rh;
u.ll = a;
rl.ll = mul_u32_u32(u.l.low, mul);
rh.ll = mul_u32_u32(u.l.high, mul) + rl.l.high;
rl.l.high = ({ unsigned int __r, __b = (divisor); if (!__builtin_constant_p(__b) || __b == 0 || (7 < 4 && (__b & (__b - 1)) != 0)) { __r = ({ register unsigned int __base asm("r4") = __b; register unsigned long long __n asm("r0") = rh.ll; register unsigned long long __res asm("r2"); register unsigned int __rem asm("r1"); asm( ".ifnc " "%0" "," "r1" "; " ".ifnc " "%0" "r1" ",fpr11; " ".ifnc " "%0" "r1" ",r11fp; " ".ifnc " "%0" "r1" ",ipr12; " ".ifnc " "%0" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r0" "; " ".ifnc " "%2" "r0" ",fpr11; " ".ifnc " "%2" "r0" ",r11fp; " ".ifnc " "%2" "r0" ",ipr12; " ".ifnc " "%2" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r4" "; " ".ifnc " "%3" "r4" ",fpr11; " ".ifnc " "%3" "r4" ",r11fp; " ".ifnc " "%3" "r4" ",ipr12; " ".ifnc " "%3" "r4" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __do_div64" : "=r" (__rem), "=r" (__res) : "r" (__n), "r" (__base) : "ip", "lr", "cc"); rh.ll = __res; __rem; }); } else if ((__b & (__b - 1)) == 0) { __r = rh.ll; __r &= (__b - 1); rh.ll /= __b; } else { unsigned long long __res, __x, __t, __m, __n = rh.ll; unsigned int __c, __p, __z = 0; __r = __n; __p = 1 << ({ unsigned int __left = (__b), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m = (~0ULL / __b) * __p; __m += (((~0ULL % __b + 1) * __p) + __b - 1) / __b; __x = ~0ULL / __b * __b - 1; __res = (__m & 0xffffffff) * (__x & 0xffffffff); __res >>= 32; __res += (__m & 0xffffffff) * (__x >> 32); __t = __res; __res += (__x & 0xffffffff) * (__m >> 32); __t = (__res < __t) ? (1ULL << 32) : 0; __res = (__res >> 32) + __t; __res += (__m >> 32) * (__x >> 32); __res /= __p; if (~0ULL % (__b / (__b & -__b)) == 0) { __n /= (__b & -__b); __m = ~0ULL / (__b / (__b & -__b)); __p = 1; __c = 1; } else if (__res != __x / __b) { __c = 1; __m = (~0ULL / __b) * __p; __m += ((~0ULL % __b + 1) * __p) / __b; } else { unsigned int __bits = -(__m & -__m); __bits |= __m >> 32; __bits = (~__bits) << 1; if (!__bits) { __p /= (__m & -__m); __m /= (__m & -__m); } else { __p >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); } __c = 0; } if (!__c) { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "=&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { __res = __m; asm ( "umlal %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "cmn %Q0, %Q1\n\t" "adcs %R0, %R0, %R1\n\t" "adc %Q0, %3, #0" : "=&r" (__res) : "r" (__m), "r" (__n), "r" (__z) : "cc" ); } if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { asm ( "umlal %R0, %Q0, %R1, %Q2\n\t" "umlal %R0, %Q0, %Q1, %R2\n\t" "mov %R0, #0\n\t" "umlal %Q0, %R0, %R1, %R2" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umlal %R0, %Q0, %R2, %Q3\n\t" "umlal %R0, %1, %Q2, %R3\n\t" "mov %R0, #0\n\t" "adds %Q0, %1, %Q0\n\t" "adc %R0, %R0, #0\n\t" "umlal %Q0, %R0, %R2, %R3" : "+&r" (__res), "+&r" (__z) : "r" (__m), "r" (__n) : "cc" ); } __res /= __p; { unsigned int __res0 = __res; unsigned int __b0 = __b; __r -= __res0 * __b0; } rh.ll = __res; } __r; });
({ unsigned int __r, __b = (divisor); if (!__builtin_constant_p(__b) || __b == 0 || (7 < 4 && (__b & (__b - 1)) != 0)) { __r = ({ register unsigned int __base asm("r4") = __b; register unsigned long long __n asm("r0") = rl.ll; register unsigned long long __res asm("r2"); register unsigned int __rem asm("r1"); asm( ".ifnc " "%0" "," "r1" "; " ".ifnc " "%0" "r1" ",fpr11; " ".ifnc " "%0" "r1" ",r11fp; " ".ifnc " "%0" "r1" ",ipr12; " ".ifnc " "%0" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r0" "; " ".ifnc " "%2" "r0" ",fpr11; " ".ifnc " "%2" "r0" ",r11fp; " ".ifnc " "%2" "r0" ",ipr12; " ".ifnc " "%2" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r4" "; " ".ifnc " "%3" "r4" ",fpr11; " ".ifnc " "%3" "r4" ",r11fp; " ".ifnc " "%3" "r4" ",ipr12; " ".ifnc " "%3" "r4" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __do_div64" : "=r" (__rem), "=r" (__res) : "r" (__n), "r" (__base) : "ip", "lr", "cc"); rl.ll = __res; __rem; }); } else if ((__b & (__b - 1)) == 0) { __r = rl.ll; __r &= (__b - 1); rl.ll /= __b; } else { unsigned long long __res, __x, __t, __m, __n = rl.ll; unsigned int __c, __p, __z = 0; __r = __n; __p = 1 << ({ unsigned int __left = (__b), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m = (~0ULL / __b) * __p; __m += (((~0ULL % __b + 1) * __p) + __b - 1) / __b; __x = ~0ULL / __b * __b - 1; __res = (__m & 0xffffffff) * (__x & 0xffffffff); __res >>= 32; __res += (__m & 0xffffffff) * (__x >> 32); __t = __res; __res += (__x & 0xffffffff) * (__m >> 32); __t = (__res < __t) ? (1ULL << 32) : 0; __res = (__res >> 32) + __t; __res += (__m >> 32) * (__x >> 32); __res /= __p; if (~0ULL % (__b / (__b & -__b)) == 0) { __n /= (__b & -__b); __m = ~0ULL / (__b / (__b & -__b)); __p = 1; __c = 1; } else if (__res != __x / __b) { __c = 1; __m = (~0ULL / __b) * __p; __m += ((~0ULL % __b + 1) * __p) / __b; } else { unsigned int __bits = -(__m & -__m); __bits |= __m >> 32; __bits = (~__bits) << 1; if (!__bits) { __p /= (__m & -__m); __m /= (__m & -__m); } else { __p >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); } __c = 0; } if (!__c) { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "=&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { __res = __m; asm ( "umlal %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "cmn %Q0, %Q1\n\t" "adcs %R0, %R0, %R1\n\t" "adc %Q0, %3, #0" : "=&r" (__res) : "r" (__m), "r" (__n), "r" (__z) : "cc" ); } if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { asm ( "umlal %R0, %Q0, %R1, %Q2\n\t" "umlal %R0, %Q0, %Q1, %R2\n\t" "mov %R0, #0\n\t" "umlal %Q0, %R0, %R1, %R2" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umlal %R0, %Q0, %R2, %Q3\n\t" "umlal %R0, %1, %Q2, %R3\n\t" "mov %R0, #0\n\t" "adds %Q0, %1, %Q0\n\t" "adc %R0, %R0, #0\n\t" "umlal %Q0, %R0, %R2, %R3" : "+&r" (__res), "+&r" (__z) : "r" (__m), "r" (__n) : "cc" ); } __res /= __p; { unsigned int __res0 = __res; unsigned int __b0 = __b; __r -= __res0 * __b0; } rl.ll = __res; } __r; });
rl.l.high = rh.l.low;
return rl.ll;
}
# 7 "./include/linux/time64.h" 2
typedef __s64 time64_t;
typedef __u64 timeu64_t;
# 19 "./include/linux/time64.h"
struct timespec64 {
time64_t tv_sec;
long tv_nsec;
};
struct itimerspec64 {
struct timespec64 it_interval;
struct timespec64 it_value;
};
# 80 "./include/linux/time64.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec timespec64_to_timespec(const struct timespec64 ts64)
{
struct timespec ret;
ret.tv_sec = (time_t)ts64.tv_sec;
ret.tv_nsec = ts64.tv_nsec;
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec64 timespec_to_timespec64(const struct timespec ts)
{
struct timespec64 ret;
ret.tv_sec = ts.tv_sec;
ret.tv_nsec = ts.tv_nsec;
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct itimerspec itimerspec64_to_itimerspec(struct itimerspec64 *its64)
{
struct itimerspec ret;
ret.it_interval = timespec64_to_timespec(its64->it_interval);
ret.it_value = timespec64_to_timespec(its64->it_value);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct itimerspec64 itimerspec_to_itimerspec64(struct itimerspec *its)
{
struct itimerspec64 ret;
ret.it_interval = timespec_to_timespec64(its->it_interval);
ret.it_value = timespec_to_timespec64(its->it_value);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int timespec64_equal(const struct timespec64 *a,
const struct timespec64 *b)
{
return (a->tv_sec == b->tv_sec) && (a->tv_nsec == b->tv_nsec);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int timespec64_compare(const struct timespec64 *lhs, const struct timespec64 *rhs)
{
if (lhs->tv_sec < rhs->tv_sec)
return -1;
if (lhs->tv_sec > rhs->tv_sec)
return 1;
return lhs->tv_nsec - rhs->tv_nsec;
}
extern void set_normalized_timespec64(struct timespec64 *ts, time64_t sec, s64 nsec);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec64 timespec64_add(struct timespec64 lhs,
struct timespec64 rhs)
{
struct timespec64 ts_delta;
set_normalized_timespec64(&ts_delta, lhs.tv_sec + rhs.tv_sec,
lhs.tv_nsec + rhs.tv_nsec);
return ts_delta;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec64 timespec64_sub(struct timespec64 lhs,
struct timespec64 rhs)
{
struct timespec64 ts_delta;
set_normalized_timespec64(&ts_delta, lhs.tv_sec - rhs.tv_sec,
lhs.tv_nsec - rhs.tv_nsec);
return ts_delta;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool timespec64_valid(const struct timespec64 *ts)
{
if (ts->tv_sec < 0)
return false;
if ((unsigned long)ts->tv_nsec >= 1000000000L)
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool timespec64_valid_strict(const struct timespec64 *ts)
{
if (!timespec64_valid(ts))
return false;
if ((unsigned long long)ts->tv_sec >= (((s64)~((u64)1 << 63)) / 1000000000L))
return false;
return true;
}
# 190 "./include/linux/time64.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 timespec64_to_ns(const struct timespec64 *ts)
{
return ((s64) ts->tv_sec * 1000000000L) + ts->tv_nsec;
}
extern struct timespec64 ns_to_timespec64(const s64 nsec);
# 211 "./include/linux/time64.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void timespec64_add_ns(struct timespec64 *a, u64 ns)
{
a->tv_sec += __iter_div_u64_rem(a->tv_nsec + ns, 1000000000L, &ns);
a->tv_nsec = ns;
}
extern struct timespec64 timespec64_add_safe(const struct timespec64 lhs,
const struct timespec64 rhs);
# 9 "./include/linux/sem.h" 2
# 1 "./include/uapi/linux/sem.h" 1
# 1 "./include/linux/ipc.h" 1
# 1 "./include/linux/uidgid.h" 1
# 16 "./include/linux/uidgid.h"
# 1 "./include/linux/highuid.h" 1
# 35 "./include/linux/highuid.h"
extern int overflowuid;
extern int overflowgid;
extern void __bad_uid(void);
extern void __bad_gid(void);
# 82 "./include/linux/highuid.h"
extern int fs_overflowuid;
extern int fs_overflowgid;
# 17 "./include/linux/uidgid.h" 2
struct user_namespace;
extern struct user_namespace init_user_ns;
typedef struct {
uid_t val;
} kuid_t;
typedef struct {
gid_t val;
} kgid_t;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) uid_t __kuid_val(kuid_t uid)
{
return uid.val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) gid_t __kgid_val(kgid_t gid)
{
return gid.val;
}
# 61 "./include/linux/uidgid.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uid_eq(kuid_t left, kuid_t right)
{
return __kuid_val(left) == __kuid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gid_eq(kgid_t left, kgid_t right)
{
return __kgid_val(left) == __kgid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uid_gt(kuid_t left, kuid_t right)
{
return __kuid_val(left) > __kuid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gid_gt(kgid_t left, kgid_t right)
{
return __kgid_val(left) > __kgid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uid_gte(kuid_t left, kuid_t right)
{
return __kuid_val(left) >= __kuid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gid_gte(kgid_t left, kgid_t right)
{
return __kgid_val(left) >= __kgid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uid_lt(kuid_t left, kuid_t right)
{
return __kuid_val(left) < __kuid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gid_lt(kgid_t left, kgid_t right)
{
return __kgid_val(left) < __kgid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uid_lte(kuid_t left, kuid_t right)
{
return __kuid_val(left) <= __kuid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gid_lte(kgid_t left, kgid_t right)
{
return __kgid_val(left) <= __kgid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uid_valid(kuid_t uid)
{
return __kuid_val(uid) != (uid_t) -1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gid_valid(kgid_t gid)
{
return __kgid_val(gid) != (gid_t) -1;
}
# 143 "./include/linux/uidgid.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kuid_t make_kuid(struct user_namespace *from, uid_t uid)
{
return (kuid_t){ uid };
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kgid_t make_kgid(struct user_namespace *from, gid_t gid)
{
return (kgid_t){ gid };
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) uid_t from_kuid(struct user_namespace *to, kuid_t kuid)
{
return __kuid_val(kuid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) gid_t from_kgid(struct user_namespace *to, kgid_t kgid)
{
return __kgid_val(kgid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) uid_t from_kuid_munged(struct user_namespace *to, kuid_t kuid)
{
uid_t uid = from_kuid(to, kuid);
if (uid == (uid_t)-1)
uid = overflowuid;
return uid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) gid_t from_kgid_munged(struct user_namespace *to, kgid_t kgid)
{
gid_t gid = from_kgid(to, kgid);
if (gid == (gid_t)-1)
gid = overflowgid;
return gid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool kuid_has_mapping(struct user_namespace *ns, kuid_t uid)
{
return uid_valid(uid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool kgid_has_mapping(struct user_namespace *ns, kgid_t gid)
{
return gid_valid(gid);
}
# 7 "./include/linux/ipc.h" 2
# 1 "./include/linux/rhashtable.h" 1
# 22 "./include/linux/rhashtable.h"
# 1 "./include/linux/err.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/errno.h" 1
# 1 "./include/uapi/asm-generic/errno.h" 1
# 1 "./include/uapi/asm-generic/errno-base.h" 1
# 6 "./include/uapi/asm-generic/errno.h" 2
# 2 "./arch/arm/include/generated/uapi/asm/errno.h" 2
# 9 "./include/linux/err.h" 2
# 24 "./include/linux/err.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void * __attribute__((warn_unused_result)) ERR_PTR(long error)
{
return (void *) error;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long __attribute__((warn_unused_result)) PTR_ERR( const void *ptr)
{
return (long) ptr;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool __attribute__((warn_unused_result)) IS_ERR( const void *ptr)
{
return __builtin_expect(!!((unsigned long)(void *)((unsigned long)ptr) >= (unsigned long)-4095), 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool __attribute__((warn_unused_result)) IS_ERR_OR_NULL( const void *ptr)
{
return __builtin_expect(!!(!ptr), 0) || __builtin_expect(!!((unsigned long)(void *)((unsigned long)ptr) >= (unsigned long)-4095), 0);
}
# 51 "./include/linux/err.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void * __attribute__((warn_unused_result)) ERR_CAST( const void *ptr)
{
return (void *) ptr;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) PTR_ERR_OR_ZERO( const void *ptr)
{
if (IS_ERR(ptr))
return PTR_ERR(ptr);
else
return 0;
}
# 23 "./include/linux/rhashtable.h" 2
# 1 "./include/linux/errno.h" 1
# 1 "./include/uapi/linux/errno.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/errno.h" 1
# 2 "./include/uapi/linux/errno.h" 2
# 6 "./include/linux/errno.h" 2
# 24 "./include/linux/rhashtable.h" 2
# 1 "./include/linux/jhash.h" 1
# 27 "./include/linux/jhash.h"
# 1 "./include/linux/unaligned/packed_struct.h" 1
struct __una_u16 { u16 x; } __attribute__((packed));
struct __una_u32 { u32 x; } __attribute__((packed));
struct __una_u64 { u64 x; } __attribute__((packed));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u16 __get_unaligned_cpu16(const void *p)
{
const struct __una_u16 *ptr = (const struct __una_u16 *)p;
return ptr->x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 __get_unaligned_cpu32(const void *p)
{
const struct __una_u32 *ptr = (const struct __una_u32 *)p;
return ptr->x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 __get_unaligned_cpu64(const void *p)
{
const struct __una_u64 *ptr = (const struct __una_u64 *)p;
return ptr->x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __put_unaligned_cpu16(u16 val, void *p)
{
struct __una_u16 *ptr = (struct __una_u16 *)p;
ptr->x = val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __put_unaligned_cpu32(u32 val, void *p)
{
struct __una_u32 *ptr = (struct __una_u32 *)p;
ptr->x = val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __put_unaligned_cpu64(u64 val, void *p)
{
struct __una_u64 *ptr = (struct __una_u64 *)p;
ptr->x = val;
}
# 28 "./include/linux/jhash.h" 2
# 70 "./include/linux/jhash.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 jhash(const void *key, u32 length, u32 initval)
{
u32 a, b, c;
const u8 *k = key;
a = b = c = 0xdeadbeef + length + initval;
while (length > 12) {
a += __get_unaligned_cpu32(k);
b += __get_unaligned_cpu32(k + 4);
c += __get_unaligned_cpu32(k + 8);
{ a -= c; a ^= rol32(c, 4); c += b; b -= a; b ^= rol32(a, 6); a += c; c -= b; c ^= rol32(b, 8); b += a; a -= c; a ^= rol32(c, 16); c += b; b -= a; b ^= rol32(a, 19); a += c; c -= b; c ^= rol32(b, 4); b += a; };
length -= 12;
k += 12;
}
switch (length) {
case 12: c += (u32)k[11]<<24;
case 11: c += (u32)k[10]<<16;
case 10: c += (u32)k[9]<<8;
case 9: c += k[8];
case 8: b += (u32)k[7]<<24;
case 7: b += (u32)k[6]<<16;
case 6: b += (u32)k[5]<<8;
case 5: b += k[4];
case 4: a += (u32)k[3]<<24;
case 3: a += (u32)k[2]<<16;
case 2: a += (u32)k[1]<<8;
case 1: a += k[0];
{ c ^= b; c -= rol32(b, 14); a ^= c; a -= rol32(c, 11); b ^= a; b -= rol32(a, 25); c ^= b; c -= rol32(b, 16); a ^= c; a -= rol32(c, 4); b ^= a; b -= rol32(a, 14); c ^= b; c -= rol32(b, 24); };
case 0:
break;
}
return c;
}
# 116 "./include/linux/jhash.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 jhash2(const u32 *k, u32 length, u32 initval)
{
u32 a, b, c;
a = b = c = 0xdeadbeef + (length<<2) + initval;
while (length > 3) {
a += k[0];
b += k[1];
c += k[2];
{ a -= c; a ^= rol32(c, 4); c += b; b -= a; b ^= rol32(a, 6); a += c; c -= b; c ^= rol32(b, 8); b += a; a -= c; a ^= rol32(c, 16); c += b; b -= a; b ^= rol32(a, 19); a += c; c -= b; c ^= rol32(b, 4); b += a; };
length -= 3;
k += 3;
}
switch (length) {
case 3: c += k[2];
case 2: b += k[1];
case 1: a += k[0];
{ c ^= b; c -= rol32(b, 14); a ^= c; a -= rol32(c, 11); b ^= a; b -= rol32(a, 25); c ^= b; c -= rol32(b, 16); a ^= c; a -= rol32(c, 4); b ^= a; b -= rol32(a, 14); c ^= b; c -= rol32(b, 24); };
case 0:
break;
}
return c;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 __jhash_nwords(u32 a, u32 b, u32 c, u32 initval)
{
a += initval;
b += initval;
c += initval;
{ c ^= b; c -= rol32(b, 14); a ^= c; a -= rol32(c, 11); b ^= a; b -= rol32(a, 25); c ^= b; c -= rol32(b, 16); a ^= c; a -= rol32(c, 4); b ^= a; b -= rol32(a, 14); c ^= b; c -= rol32(b, 24); };
return c;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 jhash_3words(u32 a, u32 b, u32 c, u32 initval)
{
return __jhash_nwords(a, b, c, initval + 0xdeadbeef + (3 << 2));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 jhash_2words(u32 a, u32 b, u32 initval)
{
return __jhash_nwords(a, b, 0, initval + 0xdeadbeef + (2 << 2));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 jhash_1word(u32 a, u32 initval)
{
return __jhash_nwords(a, 0, 0, initval + 0xdeadbeef + (1 << 2));
}
# 25 "./include/linux/rhashtable.h" 2
# 1 "./include/linux/list_nulls.h" 1
# 21 "./include/linux/list_nulls.h"
struct hlist_nulls_head {
struct hlist_nulls_node *first;
};
struct hlist_nulls_node {
struct hlist_nulls_node *next, **pprev;
};
# 43 "./include/linux/list_nulls.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_a_nulls(const struct hlist_nulls_node *ptr)
{
return ((unsigned long)ptr & 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long get_nulls_value(const struct hlist_nulls_node *ptr)
{
return ((unsigned long)ptr) >> 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hlist_nulls_unhashed(const struct hlist_nulls_node *h)
{
return !h->pprev;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hlist_nulls_empty(const struct hlist_nulls_head *h)
{
return is_a_nulls(({ union { typeof(h->first) __val; char __c[1]; } __u; if (1) __read_once_size(&(h->first), __u.__c, sizeof(h->first)); else __read_once_size_nocheck(&(h->first), __u.__c, sizeof(h->first)); do { } while (0); __u.__val; }));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_nulls_add_head(struct hlist_nulls_node *n,
struct hlist_nulls_head *h)
{
struct hlist_nulls_node *first = h->first;
n->next = first;
({ union { typeof(n->pprev) __val; char __c[1]; } __u = { .__val = ( typeof(n->pprev)) (&h->first) }; __write_once_size(&(n->pprev), __u.__c, sizeof(n->pprev)); __u.__val; });
h->first = n;
if (!is_a_nulls(first))
({ union { typeof(first->pprev) __val; char __c[1]; } __u = { .__val = ( typeof(first->pprev)) (&n->next) }; __write_once_size(&(first->pprev), __u.__c, sizeof(first->pprev)); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __hlist_nulls_del(struct hlist_nulls_node *n)
{
struct hlist_nulls_node *next = n->next;
struct hlist_nulls_node **pprev = n->pprev;
({ union { typeof(*pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*pprev)) (next) }; __write_once_size(&(*pprev), __u.__c, sizeof(*pprev)); __u.__val; });
if (!is_a_nulls(next))
({ union { typeof(next->pprev) __val; char __c[1]; } __u = { .__val = ( typeof(next->pprev)) (pprev) }; __write_once_size(&(next->pprev), __u.__c, sizeof(next->pprev)); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_nulls_del(struct hlist_nulls_node *n)
{
__hlist_nulls_del(n);
({ union { typeof(n->pprev) __val; char __c[1]; } __u = { .__val = ( typeof(n->pprev)) (((void *) 0x200 + 0)) }; __write_once_size(&(n->pprev), __u.__c, sizeof(n->pprev)); __u.__val; });
}
# 26 "./include/linux/rhashtable.h" 2
# 1 "./include/linux/workqueue.h" 1
# 1 "./include/linux/timer.h" 1
# 1 "./include/linux/ktime.h" 1
# 24 "./include/linux/ktime.h"
# 1 "./include/linux/time.h" 1
# 1 "./include/linux/seqlock.h" 1
# 48 "./include/linux/seqlock.h"
typedef struct seqcount {
unsigned sequence;
} seqcount_t;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __seqcount_init(seqcount_t *s, const char *name,
struct lock_class_key *key)
{
do { (void)(name); (void)(key); } while (0);
s->sequence = 0;
}
# 108 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned __read_seqcount_begin(const seqcount_t *s)
{
unsigned ret;
repeat:
ret = ({ union { typeof(s->sequence) __val; char __c[1]; } __u; if (1) __read_once_size(&(s->sequence), __u.__c, sizeof(s->sequence)); else __read_once_size_nocheck(&(s->sequence), __u.__c, sizeof(s->sequence)); do { } while (0); __u.__val; });
if (__builtin_expect(!!(ret & 1), 0)) {
__asm__ __volatile__("": : :"memory");
goto repeat;
}
return ret;
}
# 130 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned raw_read_seqcount(const seqcount_t *s)
{
unsigned ret = ({ union { typeof(s->sequence) __val; char __c[1]; } __u; if (1) __read_once_size(&(s->sequence), __u.__c, sizeof(s->sequence)); else __read_once_size_nocheck(&(s->sequence), __u.__c, sizeof(s->sequence)); do { } while (0); __u.__val; });
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return ret;
}
# 146 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned raw_read_seqcount_begin(const seqcount_t *s)
{
unsigned ret = __read_seqcount_begin(s);
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return ret;
}
# 162 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned read_seqcount_begin(const seqcount_t *s)
{
;
return raw_read_seqcount_begin(s);
}
# 182 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned raw_seqcount_begin(const seqcount_t *s)
{
unsigned ret = ({ union { typeof(s->sequence) __val; char __c[1]; } __u; if (1) __read_once_size(&(s->sequence), __u.__c, sizeof(s->sequence)); else __read_once_size_nocheck(&(s->sequence), __u.__c, sizeof(s->sequence)); do { } while (0); __u.__val; });
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return ret & ~1;
}
# 203 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __read_seqcount_retry(const seqcount_t *s, unsigned start)
{
return __builtin_expect(!!(s->sequence != start), 0);
}
# 218 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int read_seqcount_retry(const seqcount_t *s, unsigned start)
{
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return __read_seqcount_retry(s, start);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void raw_write_seqcount_begin(seqcount_t *s)
{
s->sequence++;
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void raw_write_seqcount_end(seqcount_t *s)
{
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
s->sequence++;
}
# 272 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void raw_write_seqcount_barrier(seqcount_t *s)
{
s->sequence++;
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
s->sequence++;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int raw_read_seqcount_latch(seqcount_t *s)
{
int seq = ({ union { typeof(s->sequence) __val; char __c[1]; } __u; if (1) __read_once_size(&(s->sequence), __u.__c, sizeof(s->sequence)); else __read_once_size_nocheck(&(s->sequence), __u.__c, sizeof(s->sequence)); do { } while (0); __u.__val; });
do { } while (0);
return seq;
}
# 364 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void raw_write_seqcount_latch(seqcount_t *s)
{
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
s->sequence++;
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_seqcount_begin_nested(seqcount_t *s, int subclass)
{
raw_write_seqcount_begin(s);
do { } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_seqcount_begin(seqcount_t *s)
{
write_seqcount_begin_nested(s, 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_seqcount_end(seqcount_t *s)
{
do { } while (0);
raw_write_seqcount_end(s);
}
# 399 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_seqcount_invalidate(seqcount_t *s)
{
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
s->sequence+=2;
}
typedef struct {
struct seqcount seqcount;
spinlock_t lock;
} seqlock_t;
# 432 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned read_seqbegin(const seqlock_t *sl)
{
return read_seqcount_begin(&sl->seqcount);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned read_seqretry(const seqlock_t *sl, unsigned start)
{
return read_seqcount_retry(&sl->seqcount, start);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_seqlock(seqlock_t *sl)
{
spin_lock(&sl->lock);
write_seqcount_begin(&sl->seqcount);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_sequnlock(seqlock_t *sl)
{
write_seqcount_end(&sl->seqcount);
spin_unlock(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_seqlock_bh(seqlock_t *sl)
{
spin_lock_bh(&sl->lock);
write_seqcount_begin(&sl->seqcount);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_sequnlock_bh(seqlock_t *sl)
{
write_seqcount_end(&sl->seqcount);
spin_unlock_bh(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_seqlock_irq(seqlock_t *sl)
{
spin_lock_irq(&sl->lock);
write_seqcount_begin(&sl->seqcount);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void write_sequnlock_irq(seqlock_t *sl)
{
write_seqcount_end(&sl->seqcount);
spin_unlock_irq(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __write_seqlock_irqsave(seqlock_t *sl)
{
unsigned long flags;
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = _raw_spin_lock_irqsave(spinlock_check(&sl->lock)); } while (0); } while (0);
write_seqcount_begin(&sl->seqcount);
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
write_sequnlock_irqrestore(seqlock_t *sl, unsigned long flags)
{
write_seqcount_end(&sl->seqcount);
spin_unlock_irqrestore(&sl->lock, flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void read_seqlock_excl(seqlock_t *sl)
{
spin_lock(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void read_sequnlock_excl(seqlock_t *sl)
{
spin_unlock(&sl->lock);
}
# 527 "./include/linux/seqlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void read_seqbegin_or_lock(seqlock_t *lock, int *seq)
{
if (!(*seq & 1))
*seq = read_seqbegin(lock);
else
read_seqlock_excl(lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int need_seqretry(seqlock_t *lock, int seq)
{
return !(seq & 1) && read_seqretry(lock, seq);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void done_seqretry(seqlock_t *lock, int seq)
{
if (seq & 1)
read_sequnlock_excl(lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void read_seqlock_excl_bh(seqlock_t *sl)
{
spin_lock_bh(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void read_sequnlock_excl_bh(seqlock_t *sl)
{
spin_unlock_bh(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void read_seqlock_excl_irq(seqlock_t *sl)
{
spin_lock_irq(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void read_sequnlock_excl_irq(seqlock_t *sl)
{
spin_unlock_irq(&sl->lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __read_seqlock_excl_irqsave(seqlock_t *sl)
{
unsigned long flags;
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = _raw_spin_lock_irqsave(spinlock_check(&sl->lock)); } while (0); } while (0);
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
read_sequnlock_excl_irqrestore(seqlock_t *sl, unsigned long flags)
{
spin_unlock_irqrestore(&sl->lock, flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
read_seqbegin_or_lock_irqsave(seqlock_t *lock, int *seq)
{
unsigned long flags = 0;
if (!(*seq & 1))
*seq = read_seqbegin(lock);
else
do { flags = __read_seqlock_excl_irqsave(lock); } while (0);
return flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
done_seqretry_irqrestore(seqlock_t *lock, int seq, unsigned long flags)
{
if (seq & 1)
read_sequnlock_excl_irqrestore(lock, flags);
}
# 7 "./include/linux/time.h" 2
extern struct timezone sys_tz;
int get_timespec64(struct timespec64 *ts,
const struct timespec *uts);
int put_timespec64(const struct timespec64 *ts,
struct timespec *uts);
int get_itimerspec64(struct itimerspec64 *it,
const struct itimerspec *uit);
int put_itimerspec64(const struct itimerspec64 *it,
struct itimerspec *uit);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int timespec_equal(const struct timespec *a,
const struct timespec *b)
{
return (a->tv_sec == b->tv_sec) && (a->tv_nsec == b->tv_nsec);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int timespec_compare(const struct timespec *lhs, const struct timespec *rhs)
{
if (lhs->tv_sec < rhs->tv_sec)
return -1;
if (lhs->tv_sec > rhs->tv_sec)
return 1;
return lhs->tv_nsec - rhs->tv_nsec;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int timeval_compare(const struct timeval *lhs, const struct timeval *rhs)
{
if (lhs->tv_sec < rhs->tv_sec)
return -1;
if (lhs->tv_sec > rhs->tv_sec)
return 1;
return lhs->tv_usec - rhs->tv_usec;
}
extern time64_t mktime64(const unsigned int year, const unsigned int mon,
const unsigned int day, const unsigned int hour,
const unsigned int min, const unsigned int sec);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long mktime(const unsigned int year,
const unsigned int mon, const unsigned int day,
const unsigned int hour, const unsigned int min,
const unsigned int sec)
{
return mktime64(year, mon, day, hour, min, sec);
}
extern void set_normalized_timespec(struct timespec *ts, time_t sec, s64 nsec);
extern struct timespec timespec_add_safe(const struct timespec lhs,
const struct timespec rhs);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec timespec_add(struct timespec lhs,
struct timespec rhs)
{
struct timespec ts_delta;
set_normalized_timespec(&ts_delta, lhs.tv_sec + rhs.tv_sec,
lhs.tv_nsec + rhs.tv_nsec);
return ts_delta;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec timespec_sub(struct timespec lhs,
struct timespec rhs)
{
struct timespec ts_delta;
set_normalized_timespec(&ts_delta, lhs.tv_sec - rhs.tv_sec,
lhs.tv_nsec - rhs.tv_nsec);
return ts_delta;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool timespec_valid(const struct timespec *ts)
{
if (ts->tv_sec < 0)
return false;
if ((unsigned long)ts->tv_nsec >= 1000000000L)
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool timespec_valid_strict(const struct timespec *ts)
{
if (!timespec_valid(ts))
return false;
if ((unsigned long long)ts->tv_sec >= (((s64)~((u64)1 << 63)) / 1000000000L))
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool timeval_valid(const struct timeval *tv)
{
if (tv->tv_sec < 0)
return false;
if (tv->tv_usec < 0 || tv->tv_usec >= 1000000L)
return false;
return true;
}
extern struct timespec timespec_trunc(struct timespec t, unsigned gran);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool timeval_inject_offset_valid(const struct timeval *tv)
{
if (tv->tv_usec < 0 || tv->tv_usec >= 1000000L)
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool timespec_inject_offset_valid(const struct timespec *ts)
{
if (ts->tv_nsec < 0 || ts->tv_nsec >= 1000000000L)
return false;
return true;
}
# 177 "./include/linux/time.h"
struct itimerval;
extern int do_setitimer(int which, struct itimerval *value,
struct itimerval *ovalue);
extern int do_getitimer(int which, struct itimerval *value);
extern long do_utimes(int dfd, const char *filename, struct timespec64 *times, int flags);
struct tm {
int tm_sec;
int tm_min;
int tm_hour;
int tm_mday;
int tm_mon;
long tm_year;
int tm_wday;
int tm_yday;
};
void time64_to_tm(time64_t totalsecs, int offset, struct tm *result);
# 220 "./include/linux/time.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void time_to_tm(time_t totalsecs, int offset, struct tm *result)
{
time64_to_tm(totalsecs, offset, result);
}
# 232 "./include/linux/time.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 timespec_to_ns(const struct timespec *ts)
{
return ((s64) ts->tv_sec * 1000000000L) + ts->tv_nsec;
}
# 244 "./include/linux/time.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 timeval_to_ns(const struct timeval *tv)
{
return ((s64) tv->tv_sec * 1000000000L) +
tv->tv_usec * 1000L;
}
extern struct timespec ns_to_timespec(const s64 nsec);
extern struct timeval ns_to_timeval(const s64 nsec);
# 274 "./include/linux/time.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void timespec_add_ns(struct timespec *a, u64 ns)
{
a->tv_sec += __iter_div_u64_rem(a->tv_nsec + ns, 1000000000L, &ns);
a->tv_nsec = ns;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool itimerspec64_valid(const struct itimerspec64 *its)
{
if (!timespec64_valid(&(its->it_interval)) ||
!timespec64_valid(&(its->it_value)))
return false;
return true;
}
# 25 "./include/linux/ktime.h" 2
# 1 "./include/linux/jiffies.h" 1
# 10 "./include/linux/jiffies.h"
# 1 "./include/linux/timex.h" 1
# 56 "./include/linux/timex.h"
# 1 "./include/uapi/linux/timex.h" 1
# 64 "./include/uapi/linux/timex.h"
struct timex {
unsigned int modes;
__kernel_long_t offset;
__kernel_long_t freq;
__kernel_long_t maxerror;
__kernel_long_t esterror;
int status;
__kernel_long_t constant;
__kernel_long_t precision;
__kernel_long_t tolerance;
struct timeval time;
__kernel_long_t tick;
__kernel_long_t ppsfreq;
__kernel_long_t jitter;
int shift;
__kernel_long_t stabil;
__kernel_long_t jitcnt;
__kernel_long_t calcnt;
__kernel_long_t errcnt;
__kernel_long_t stbcnt;
int tai;
int :32; int :32; int :32; int :32;
int :32; int :32; int :32; int :32;
int :32; int :32; int :32;
};
# 57 "./include/linux/timex.h" 2
# 1 "./include/uapi/linux/param.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/param.h" 1
# 1 "./include/asm-generic/param.h" 1
# 1 "./include/uapi/asm-generic/param.h" 1
# 6 "./include/asm-generic/param.h" 2
# 2 "./arch/arm/include/generated/uapi/asm/param.h" 2
# 6 "./include/uapi/linux/param.h" 2
# 64 "./include/linux/timex.h" 2
# 1 "./arch/arm/include/asm/timex.h" 1
# 15 "./arch/arm/include/asm/timex.h"
typedef unsigned long cycles_t;
# 66 "./include/linux/timex.h" 2
# 139 "./include/linux/timex.h"
extern unsigned long tick_usec;
extern unsigned long tick_nsec;
# 154 "./include/linux/timex.h"
extern int do_adjtimex(struct timex *);
extern void hardpps(const struct timespec64 *, const struct timespec64 *);
int read_current_timer(unsigned long *timer_val);
void ntp_notify_cmos_timer(void);
# 11 "./include/linux/jiffies.h" 2
# 1 "./arch/arm/include/generated/uapi/asm/param.h" 1
# 12 "./include/linux/jiffies.h" 2
# 1 "./include/generated/timeconst.h" 1
# 13 "./include/linux/jiffies.h" 2
# 60 "./include/linux/jiffies.h"
extern int register_refined_jiffies(long clock_tick_rate);
# 80 "./include/linux/jiffies.h"
extern u64 __attribute__((__aligned__((1 << 6)), __section__(".data..cacheline_aligned"))) jiffies_64;
extern unsigned long volatile __attribute__((__aligned__((1 << 6)), __section__(".data..cacheline_aligned"))) jiffies;
u64 get_jiffies_64(void);
# 190 "./include/linux/jiffies.h"
extern unsigned long preset_lpj;
# 291 "./include/linux/jiffies.h"
extern unsigned int jiffies_to_msecs(const unsigned long j);
extern unsigned int jiffies_to_usecs(const unsigned long j);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 jiffies_to_nsecs(const unsigned long j)
{
return (u64)jiffies_to_usecs(j) * 1000L;
}
extern u64 jiffies64_to_nsecs(u64 j);
extern unsigned long __msecs_to_jiffies(const unsigned int m);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long _msecs_to_jiffies(const unsigned int m)
{
return (m + (1000L / 100) - 1) / (1000L / 100);
}
# 363 "./include/linux/jiffies.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long msecs_to_jiffies(const unsigned int m)
{
if (__builtin_constant_p(m)) {
if ((int)m < 0)
return ((((long)(~0UL>>1)) >> 1)-1);
return _msecs_to_jiffies(m);
} else {
return __msecs_to_jiffies(m);
}
}
extern unsigned long __usecs_to_jiffies(const unsigned int u);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long _usecs_to_jiffies(const unsigned int u)
{
return (u + (1000000L / 100) - 1) / (1000000L / 100);
}
# 410 "./include/linux/jiffies.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long usecs_to_jiffies(const unsigned int u)
{
if (__builtin_constant_p(u)) {
if (u > jiffies_to_usecs(((((long)(~0UL>>1)) >> 1)-1)))
return ((((long)(~0UL>>1)) >> 1)-1);
return _usecs_to_jiffies(u);
} else {
return __usecs_to_jiffies(u);
}
}
extern unsigned long timespec64_to_jiffies(const struct timespec64 *value);
extern void jiffies_to_timespec64(const unsigned long jiffies,
struct timespec64 *value);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long timespec_to_jiffies(const struct timespec *value)
{
struct timespec64 ts = timespec_to_timespec64(*value);
return timespec64_to_jiffies(&ts);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void jiffies_to_timespec(const unsigned long jiffies,
struct timespec *value)
{
struct timespec64 ts;
jiffies_to_timespec64(jiffies, &ts);
*value = timespec64_to_timespec(ts);
}
extern unsigned long timeval_to_jiffies(const struct timeval *value);
extern void jiffies_to_timeval(const unsigned long jiffies,
struct timeval *value);
extern clock_t jiffies_to_clock_t(unsigned long x);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) clock_t jiffies_delta_to_clock_t(long delta)
{
return jiffies_to_clock_t(({ typeof(0L) __UNIQUE_ID_max1_10 = (0L); typeof(delta) __UNIQUE_ID_max2_11 = (delta); (void) (&__UNIQUE_ID_max1_10 == &__UNIQUE_ID_max2_11); __UNIQUE_ID_max1_10 > __UNIQUE_ID_max2_11 ? __UNIQUE_ID_max1_10 : __UNIQUE_ID_max2_11; }));
}
extern unsigned long clock_t_to_jiffies(unsigned long x);
extern u64 jiffies_64_to_clock_t(u64 x);
extern u64 nsec_to_clock_t(u64 x);
extern u64 nsecs_to_jiffies64(u64 n);
extern unsigned long nsecs_to_jiffies(u64 n);
# 26 "./include/linux/ktime.h" 2
typedef s64 ktime_t;
# 37 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_set(const s64 secs, const unsigned long nsecs)
{
if (__builtin_expect(!!(secs >= (((s64)~((u64)1 << 63)) / 1000000000L)), 0))
return ((s64)~((u64)1 << 63));
return secs * 1000000000L + (s64)nsecs;
}
# 70 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t timespec_to_ktime(struct timespec ts)
{
return ktime_set(ts.tv_sec, ts.tv_nsec);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t timespec64_to_ktime(struct timespec64 ts)
{
return ktime_set(ts.tv_sec, ts.tv_nsec);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t timeval_to_ktime(struct timeval tv)
{
return ktime_set(tv.tv_sec, tv.tv_usec * 1000L);
}
# 109 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int ktime_compare(const ktime_t cmp1, const ktime_t cmp2)
{
if (cmp1 < cmp2)
return -1;
if (cmp1 > cmp2)
return 1;
return 0;
}
# 125 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool ktime_after(const ktime_t cmp1, const ktime_t cmp2)
{
return ktime_compare(cmp1, cmp2) > 0;
}
# 137 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool ktime_before(const ktime_t cmp1, const ktime_t cmp2)
{
return ktime_compare(cmp1, cmp2) < 0;
}
extern s64 __ktime_divns(const ktime_t kt, s64 div);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 ktime_divns(const ktime_t kt, s64 div)
{
do { if (__builtin_expect(!!(div < 0), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/ktime.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "150" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
if (__builtin_constant_p(div) && !(div >> 32)) {
s64 ns = kt;
u64 tmp = ns < 0 ? -ns : ns;
({ unsigned int __r, __b = (div); if (!__builtin_constant_p(__b) || __b == 0 || (7 < 4 && (__b & (__b - 1)) != 0)) { __r = ({ register unsigned int __base asm("r4") = __b; register unsigned long long __n asm("r0") = tmp; register unsigned long long __res asm("r2"); register unsigned int __rem asm("r1"); asm( ".ifnc " "%0" "," "r1" "; " ".ifnc " "%0" "r1" ",fpr11; " ".ifnc " "%0" "r1" ",r11fp; " ".ifnc " "%0" "r1" ",ipr12; " ".ifnc " "%0" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r0" "; " ".ifnc " "%2" "r0" ",fpr11; " ".ifnc " "%2" "r0" ",r11fp; " ".ifnc " "%2" "r0" ",ipr12; " ".ifnc " "%2" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r4" "; " ".ifnc " "%3" "r4" ",fpr11; " ".ifnc " "%3" "r4" ",r11fp; " ".ifnc " "%3" "r4" ",ipr12; " ".ifnc " "%3" "r4" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __do_div64" : "=r" (__rem), "=r" (__res) : "r" (__n), "r" (__base) : "ip", "lr", "cc"); tmp = __res; __rem; }); } else if ((__b & (__b - 1)) == 0) { __r = tmp; __r &= (__b - 1); tmp /= __b; } else { unsigned long long __res, __x, __t, __m, __n = tmp; unsigned int __c, __p, __z = 0; __r = __n; __p = 1 << ({ unsigned int __left = (__b), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m = (~0ULL / __b) * __p; __m += (((~0ULL % __b + 1) * __p) + __b - 1) / __b; __x = ~0ULL / __b * __b - 1; __res = (__m & 0xffffffff) * (__x & 0xffffffff); __res >>= 32; __res += (__m & 0xffffffff) * (__x >> 32); __t = __res; __res += (__x & 0xffffffff) * (__m >> 32); __t = (__res < __t) ? (1ULL << 32) : 0; __res = (__res >> 32) + __t; __res += (__m >> 32) * (__x >> 32); __res /= __p; if (~0ULL % (__b / (__b & -__b)) == 0) { __n /= (__b & -__b); __m = ~0ULL / (__b / (__b & -__b)); __p = 1; __c = 1; } else if (__res != __x / __b) { __c = 1; __m = (~0ULL / __b) * __p; __m += ((~0ULL % __b + 1) * __p) / __b; } else { unsigned int __bits = -(__m & -__m); __bits |= __m >> 32; __bits = (~__bits) << 1; if (!__bits) { __p /= (__m & -__m); __m /= (__m & -__m); } else { __p >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); } __c = 0; } if (!__c) { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "=&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { __res = __m; asm ( "umlal %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "cmn %Q0, %Q1\n\t" "adcs %R0, %R0, %R1\n\t" "adc %Q0, %3, #0" : "=&r" (__res) : "r" (__m), "r" (__n), "r" (__z) : "cc" ); } if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { asm ( "umlal %R0, %Q0, %R1, %Q2\n\t" "umlal %R0, %Q0, %Q1, %R2\n\t" "mov %R0, #0\n\t" "umlal %Q0, %R0, %R1, %R2" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umlal %R0, %Q0, %R2, %Q3\n\t" "umlal %R0, %1, %Q2, %R3\n\t" "mov %R0, #0\n\t" "adds %Q0, %1, %Q0\n\t" "adc %R0, %R0, #0\n\t" "umlal %Q0, %R0, %R2, %R3" : "+&r" (__res), "+&r" (__z) : "r" (__m), "r" (__n) : "cc" ); } __res /= __p; { unsigned int __res0 = __res; unsigned int __b0 = __b; __r -= __res0 * __b0; } tmp = __res; } __r; });
return ns < 0 ? -tmp : tmp;
} else {
return __ktime_divns(kt, div);
}
}
# 173 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 ktime_to_us(const ktime_t kt)
{
return ktime_divns(kt, 1000L);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 ktime_to_ms(const ktime_t kt)
{
return ktime_divns(kt, 1000000L);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 ktime_us_delta(const ktime_t later, const ktime_t earlier)
{
return ktime_to_us(((later) - (earlier)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 ktime_ms_delta(const ktime_t later, const ktime_t earlier)
{
return ktime_to_ms(((later) - (earlier)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_add_us(const ktime_t kt, const u64 usec)
{
return ((kt) + (usec * 1000L));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_add_ms(const ktime_t kt, const u64 msec)
{
return ((kt) + (msec * 1000000L));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_sub_us(const ktime_t kt, const u64 usec)
{
return ((kt) - (usec * 1000L));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_sub_ms(const ktime_t kt, const u64 msec)
{
return ((kt) - (msec * 1000000L));
}
extern ktime_t ktime_add_safe(const ktime_t lhs, const ktime_t rhs);
# 223 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result)) bool ktime_to_timespec_cond(const ktime_t kt,
struct timespec *ts)
{
if (kt) {
*ts = ns_to_timespec((kt));
return true;
} else {
return false;
}
}
# 242 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result)) bool ktime_to_timespec64_cond(const ktime_t kt,
struct timespec64 *ts)
{
if (kt) {
*ts = ns_to_timespec64((kt));
return true;
} else {
return false;
}
}
# 262 "./include/linux/ktime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ns_to_ktime(u64 ns)
{
return ns;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ms_to_ktime(u64 ms)
{
return ms * 1000000L;
}
# 1 "./include/linux/timekeeping.h" 1
void timekeeping_init(void);
extern int timekeeping_suspended;
extern void update_process_times(int user);
extern void xtime_update(unsigned long ticks);
extern void do_gettimeofday(struct timeval *tv);
extern int do_settimeofday64(const struct timespec64 *ts);
extern int do_sys_settimeofday64(const struct timespec64 *tv,
const struct timezone *tz);
unsigned long get_seconds(void);
struct timespec64 current_kernel_time64(void);
struct timespec __current_kernel_time(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec current_kernel_time(void)
{
struct timespec64 now = current_kernel_time64();
return timespec64_to_timespec(now);
}
struct timespec64 get_monotonic_coarse64(void);
extern void getrawmonotonic64(struct timespec64 *ts);
extern void ktime_get_ts64(struct timespec64 *ts);
extern time64_t ktime_get_seconds(void);
extern time64_t ktime_get_real_seconds(void);
extern int __getnstimeofday64(struct timespec64 *tv);
extern void getnstimeofday64(struct timespec64 *tv);
extern void getboottime64(struct timespec64 *ts);
# 98 "./include/linux/timekeeping.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int do_settimeofday(const struct timespec *ts)
{
struct timespec64 ts64;
ts64 = timespec_to_timespec64(*ts);
return do_settimeofday64(&ts64);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __getnstimeofday(struct timespec *ts)
{
struct timespec64 ts64;
int ret = __getnstimeofday64(&ts64);
*ts = timespec64_to_timespec(ts64);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void getnstimeofday(struct timespec *ts)
{
struct timespec64 ts64;
getnstimeofday64(&ts64);
*ts = timespec64_to_timespec(ts64);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ktime_get_ts(struct timespec *ts)
{
struct timespec64 ts64;
ktime_get_ts64(&ts64);
*ts = timespec64_to_timespec(ts64);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ktime_get_real_ts(struct timespec *ts)
{
struct timespec64 ts64;
getnstimeofday64(&ts64);
*ts = timespec64_to_timespec(ts64);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void getrawmonotonic(struct timespec *ts)
{
struct timespec64 ts64;
getrawmonotonic64(&ts64);
*ts = timespec64_to_timespec(ts64);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct timespec get_monotonic_coarse(void)
{
return timespec64_to_timespec(get_monotonic_coarse64());
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void getboottime(struct timespec *ts)
{
struct timespec64 ts64;
getboottime64(&ts64);
*ts = timespec64_to_timespec(ts64);
}
# 167 "./include/linux/timekeeping.h"
enum tk_offsets {
TK_OFFS_REAL,
TK_OFFS_BOOT,
TK_OFFS_TAI,
TK_OFFS_MAX,
};
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs);
extern ktime_t ktime_get_raw(void);
extern u32 ktime_get_resolution_ns(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_get_real(void)
{
return ktime_get_with_offset(TK_OFFS_REAL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_get_boottime(void)
{
return ktime_get_with_offset(TK_OFFS_BOOT);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_get_clocktai(void)
{
return ktime_get_with_offset(TK_OFFS_TAI);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t ktime_mono_to_real(ktime_t mono)
{
return ktime_mono_to_any(mono, TK_OFFS_REAL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 ktime_get_ns(void)
{
return (ktime_get());
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 ktime_get_real_ns(void)
{
return (ktime_get_real());
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 ktime_get_boot_ns(void)
{
return (ktime_get_boottime());
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 ktime_get_tai_ns(void)
{
return (ktime_get_clocktai());
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 ktime_get_raw_ns(void)
{
return (ktime_get_raw());
}
extern u64 ktime_get_mono_fast_ns(void);
extern u64 ktime_get_raw_fast_ns(void);
extern u64 ktime_get_boot_fast_ns(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void get_monotonic_boottime(struct timespec *ts)
{
*ts = ns_to_timespec((ktime_get_boottime()));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void get_monotonic_boottime64(struct timespec64 *ts)
{
*ts = ns_to_timespec64((ktime_get_boottime()));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void timekeeping_clocktai(struct timespec *ts)
{
*ts = ns_to_timespec((ktime_get_clocktai()));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void timekeeping_clocktai64(struct timespec64 *ts)
{
*ts = ns_to_timespec64((ktime_get_clocktai()));
}
extern bool timekeeping_rtc_skipsuspend(void);
extern bool timekeeping_rtc_skipresume(void);
extern void timekeeping_inject_sleeptime64(struct timespec64 *delta);
extern void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw,
struct timespec64 *ts_real);
# 290 "./include/linux/timekeeping.h"
struct system_time_snapshot {
u64 cycles;
ktime_t real;
ktime_t raw;
unsigned int clock_was_set_seq;
u8 cs_was_changed_seq;
};
# 305 "./include/linux/timekeeping.h"
struct system_device_crosststamp {
ktime_t device;
ktime_t sys_realtime;
ktime_t sys_monoraw;
};
# 318 "./include/linux/timekeeping.h"
struct system_counterval_t {
u64 cycles;
struct clocksource *cs;
};
extern int get_device_system_crosststamp(
int (*get_time_fn)(ktime_t *device_time,
struct system_counterval_t *system_counterval,
void *ctx),
void *ctx,
struct system_time_snapshot *history,
struct system_device_crosststamp *xtstamp);
extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
extern int persistent_clock_is_local;
extern void read_persistent_clock(struct timespec *ts);
extern void read_persistent_clock64(struct timespec64 *ts);
extern void read_boot_clock64(struct timespec64 *ts);
extern int update_persistent_clock(struct timespec now);
extern int update_persistent_clock64(struct timespec64 now);
# 273 "./include/linux/ktime.h" 2
# 7 "./include/linux/timer.h" 2
# 1 "./include/linux/debugobjects.h" 1
enum debug_obj_state {
ODEBUG_STATE_NONE,
ODEBUG_STATE_INIT,
ODEBUG_STATE_INACTIVE,
ODEBUG_STATE_ACTIVE,
ODEBUG_STATE_DESTROYED,
ODEBUG_STATE_NOTAVAILABLE,
ODEBUG_STATE_MAX,
};
struct debug_obj_descr;
# 28 "./include/linux/debugobjects.h"
struct debug_obj {
struct hlist_node node;
enum debug_obj_state state;
unsigned int astate;
void *object;
struct debug_obj_descr *descr;
};
# 55 "./include/linux/debugobjects.h"
struct debug_obj_descr {
const char *name;
void *(*debug_hint)(void *addr);
bool (*is_static_object)(void *addr);
bool (*fixup_init)(void *addr, enum debug_obj_state state);
bool (*fixup_activate)(void *addr, enum debug_obj_state state);
bool (*fixup_destroy)(void *addr, enum debug_obj_state state);
bool (*fixup_free)(void *addr, enum debug_obj_state state);
bool (*fixup_assert_init)(void *addr, enum debug_obj_state state);
};
# 88 "./include/linux/debugobjects.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_object_init (void *addr, struct debug_obj_descr *descr) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_object_init_on_stack(void *addr, struct debug_obj_descr *descr) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
debug_object_activate (void *addr, struct debug_obj_descr *descr) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_object_deactivate(void *addr, struct debug_obj_descr *descr) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_object_destroy (void *addr, struct debug_obj_descr *descr) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_object_free (void *addr, struct debug_obj_descr *descr) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_object_assert_init(void *addr, struct debug_obj_descr *descr) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void debug_objects_early_init(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void debug_objects_mem_init(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_check_no_obj_freed(const void *address, unsigned long size) { }
# 9 "./include/linux/timer.h" 2
struct tvec_base;
struct timer_list {
struct hlist_node entry;
unsigned long expires;
void (*function)(unsigned long);
unsigned long data;
u32 flags;
};
# 96 "./include/linux/timer.h"
void init_timer_key(struct timer_list *timer, unsigned int flags,
const char *name, struct lock_class_key *key);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void destroy_timer_on_stack(struct timer_list *timer) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_timer_on_stack_key(struct timer_list *timer,
unsigned int flags, const char *name,
struct lock_class_key *key)
{
init_timer_key(timer, flags, name, key);
}
# 202 "./include/linux/timer.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void timer_setup(struct timer_list *timer,
void (*callback)(struct timer_list *),
unsigned int flags)
{
do { init_timer_key(((timer)), ((flags)), ((void *)0), ((void *)0)); (timer)->function = ((void (*)(unsigned long))callback); (timer)->data = ((unsigned long)timer); } while (0);
}
# 224 "./include/linux/timer.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int timer_pending(const struct timer_list * timer)
{
return timer->entry.pprev != ((void *)0);
}
extern void add_timer_on(struct timer_list *timer, int cpu);
extern int del_timer(struct timer_list * timer);
extern int mod_timer(struct timer_list *timer, unsigned long expires);
extern int mod_timer_pending(struct timer_list *timer, unsigned long expires);
extern void add_timer(struct timer_list *timer);
extern int try_to_del_timer_sync(struct timer_list *timer);
extern int del_timer_sync(struct timer_list *timer);
extern void init_timers(void);
extern void run_local_timers(void);
struct hrtimer;
extern enum hrtimer_restart it_real_fn(struct hrtimer *);
struct ctl_table;
extern unsigned int sysctl_timer_migration;
int timer_migration_handler(struct ctl_table *table, int write,
void *buffer, size_t *lenp,
loff_t *ppos);
unsigned long __round_jiffies(unsigned long j, int cpu);
unsigned long __round_jiffies_relative(unsigned long j, int cpu);
unsigned long round_jiffies(unsigned long j);
unsigned long round_jiffies_relative(unsigned long j);
unsigned long __round_jiffies_up(unsigned long j, int cpu);
unsigned long __round_jiffies_up_relative(unsigned long j, int cpu);
unsigned long round_jiffies_up(unsigned long j);
unsigned long round_jiffies_up_relative(unsigned long j);
int timers_prepare_cpu(unsigned int cpu);
int timers_dead_cpu(unsigned int cpu);
# 10 "./include/linux/workqueue.h" 2
struct workqueue_struct;
struct work_struct;
typedef void (*work_func_t)(struct work_struct *work);
void delayed_work_timer_fn(unsigned long __data);
enum {
WORK_STRUCT_PENDING_BIT = 0,
WORK_STRUCT_DELAYED_BIT = 1,
WORK_STRUCT_PWQ_BIT = 2,
WORK_STRUCT_LINKED_BIT = 3,
WORK_STRUCT_COLOR_SHIFT = 4,
WORK_STRUCT_COLOR_BITS = 4,
WORK_STRUCT_PENDING = 1 << WORK_STRUCT_PENDING_BIT,
WORK_STRUCT_DELAYED = 1 << WORK_STRUCT_DELAYED_BIT,
WORK_STRUCT_PWQ = 1 << WORK_STRUCT_PWQ_BIT,
WORK_STRUCT_LINKED = 1 << WORK_STRUCT_LINKED_BIT,
WORK_STRUCT_STATIC = 0,
WORK_NR_COLORS = (1 << WORK_STRUCT_COLOR_BITS) - 1,
WORK_NO_COLOR = WORK_NR_COLORS,
WORK_CPU_UNBOUND = 4,
WORK_STRUCT_FLAG_BITS = WORK_STRUCT_COLOR_SHIFT +
WORK_STRUCT_COLOR_BITS,
WORK_OFFQ_FLAG_BASE = WORK_STRUCT_COLOR_SHIFT,
__WORK_OFFQ_CANCELING = WORK_OFFQ_FLAG_BASE,
WORK_OFFQ_CANCELING = (1 << __WORK_OFFQ_CANCELING),
WORK_OFFQ_FLAG_BITS = 1,
WORK_OFFQ_POOL_SHIFT = WORK_OFFQ_FLAG_BASE + WORK_OFFQ_FLAG_BITS,
WORK_OFFQ_LEFT = 32 - WORK_OFFQ_POOL_SHIFT,
WORK_OFFQ_POOL_BITS = WORK_OFFQ_LEFT <= 31 ? WORK_OFFQ_LEFT : 31,
WORK_OFFQ_POOL_NONE = (1LU << WORK_OFFQ_POOL_BITS) - 1,
WORK_STRUCT_FLAG_MASK = (1UL << WORK_STRUCT_FLAG_BITS) - 1,
WORK_STRUCT_WQ_DATA_MASK = ~WORK_STRUCT_FLAG_MASK,
WORK_STRUCT_NO_POOL = (unsigned long)WORK_OFFQ_POOL_NONE << WORK_OFFQ_POOL_SHIFT,
WORK_BUSY_PENDING = 1 << 0,
WORK_BUSY_RUNNING = 1 << 1,
WORKER_DESC_LEN = 24,
};
struct work_struct {
atomic_long_t data;
struct list_head entry;
work_func_t func;
};
struct delayed_work {
struct work_struct work;
struct timer_list timer;
struct workqueue_struct *wq;
int cpu;
};
struct workqueue_attrs {
int nice;
cpumask_var_t cpumask;
# 146 "./include/linux/workqueue.h"
bool no_numa;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct delayed_work *to_delayed_work(struct work_struct *work)
{
return ({ void *__mptr = (void *)(work); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(work)), typeof(((struct delayed_work *)0)->work)) && !__builtin_types_compatible_p(typeof(*(work)), typeof(void)))); extern void __compiletime_assert_12(void) ; if (__cond) __compiletime_assert_12(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct delayed_work *)(__mptr - __builtin_offsetof(struct delayed_work, work))); });
}
struct execute_work {
struct work_struct work;
};
# 202 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __init_work(struct work_struct *work, int onstack) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void destroy_work_on_stack(struct work_struct *work) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void destroy_delayed_work_on_stack(struct delayed_work *work) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int work_static(struct work_struct *work) { return 0; }
# 290 "./include/linux/workqueue.h"
enum {
WQ_UNBOUND = 1 << 1,
WQ_FREEZABLE = 1 << 2,
WQ_MEM_RECLAIM = 1 << 3,
WQ_HIGHPRI = 1 << 4,
WQ_CPU_INTENSIVE = 1 << 5,
WQ_SYSFS = 1 << 6,
# 323 "./include/linux/workqueue.h"
WQ_POWER_EFFICIENT = 1 << 7,
__WQ_DRAINING = 1 << 16,
__WQ_ORDERED = 1 << 17,
__WQ_LEGACY = 1 << 18,
__WQ_ORDERED_EXPLICIT = 1 << 19,
WQ_MAX_ACTIVE = 512,
WQ_MAX_UNBOUND_PER_CPU = 4,
WQ_DFL_ACTIVE = WQ_MAX_ACTIVE / 2,
};
# 367 "./include/linux/workqueue.h"
extern struct workqueue_struct *system_wq;
extern struct workqueue_struct *system_highpri_wq;
extern struct workqueue_struct *system_long_wq;
extern struct workqueue_struct *system_unbound_wq;
extern struct workqueue_struct *system_freezable_wq;
extern struct workqueue_struct *system_power_efficient_wq;
extern struct workqueue_struct *system_freezable_power_efficient_wq;
extern struct workqueue_struct *
__alloc_workqueue_key(const char *fmt, unsigned int flags, int max_active,
struct lock_class_key *key, const char *lock_name, ...) __attribute__((format(printf, 1, 6)));
# 438 "./include/linux/workqueue.h"
extern void destroy_workqueue(struct workqueue_struct *wq);
struct workqueue_attrs *alloc_workqueue_attrs(gfp_t gfp_mask);
void free_workqueue_attrs(struct workqueue_attrs *attrs);
int apply_workqueue_attrs(struct workqueue_struct *wq,
const struct workqueue_attrs *attrs);
int workqueue_set_unbound_cpumask(cpumask_var_t cpumask);
extern bool queue_work_on(int cpu, struct workqueue_struct *wq,
struct work_struct *work);
extern bool queue_delayed_work_on(int cpu, struct workqueue_struct *wq,
struct delayed_work *work, unsigned long delay);
extern bool mod_delayed_work_on(int cpu, struct workqueue_struct *wq,
struct delayed_work *dwork, unsigned long delay);
extern void flush_workqueue(struct workqueue_struct *wq);
extern void drain_workqueue(struct workqueue_struct *wq);
extern int schedule_on_each_cpu(work_func_t func);
int execute_in_process_context(work_func_t fn, struct execute_work *);
extern bool flush_work(struct work_struct *work);
extern bool cancel_work(struct work_struct *work);
extern bool cancel_work_sync(struct work_struct *work);
extern bool flush_delayed_work(struct delayed_work *dwork);
extern bool cancel_delayed_work(struct delayed_work *dwork);
extern bool cancel_delayed_work_sync(struct delayed_work *dwork);
extern void workqueue_set_max_active(struct workqueue_struct *wq,
int max_active);
extern struct work_struct *current_work(void);
extern bool current_is_workqueue_rescuer(void);
extern bool workqueue_congested(int cpu, struct workqueue_struct *wq);
extern unsigned int work_busy(struct work_struct *work);
extern __attribute__((format(printf, 1, 2))) void set_worker_desc(const char *fmt, ...);
extern void print_worker_info(const char *log_lvl, struct task_struct *task);
extern void show_workqueue_state(void);
# 488 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool queue_work(struct workqueue_struct *wq,
struct work_struct *work)
{
return queue_work_on(WORK_CPU_UNBOUND, wq, work);
}
# 502 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool queue_delayed_work(struct workqueue_struct *wq,
struct delayed_work *dwork,
unsigned long delay)
{
return queue_delayed_work_on(WORK_CPU_UNBOUND, wq, dwork, delay);
}
# 517 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool mod_delayed_work(struct workqueue_struct *wq,
struct delayed_work *dwork,
unsigned long delay)
{
return mod_delayed_work_on(WORK_CPU_UNBOUND, wq, dwork, delay);
}
# 531 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool schedule_work_on(int cpu, struct work_struct *work)
{
return queue_work_on(cpu, system_wq, work);
}
# 547 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool schedule_work(struct work_struct *work)
{
return queue_work(system_wq, work);
}
# 576 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void flush_scheduled_work(void)
{
flush_workqueue(system_wq);
}
# 590 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool schedule_delayed_work_on(int cpu, struct delayed_work *dwork,
unsigned long delay)
{
return queue_delayed_work_on(cpu, system_wq, dwork, delay);
}
# 604 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool schedule_delayed_work(struct delayed_work *dwork,
unsigned long delay)
{
return queue_delayed_work(system_wq, dwork, delay);
}
# 620 "./include/linux/workqueue.h"
long work_on_cpu(int cpu, long (*fn)(void *), void *arg);
long work_on_cpu_safe(int cpu, long (*fn)(void *), void *arg);
extern void freeze_workqueues_begin(void);
extern bool freeze_workqueues_busy(void);
extern void thaw_workqueues(void);
int workqueue_sysfs_register(struct workqueue_struct *wq);
# 640 "./include/linux/workqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void wq_watchdog_touch(int cpu) { }
int workqueue_prepare_cpu(unsigned int cpu);
int workqueue_online_cpu(unsigned int cpu);
int workqueue_offline_cpu(unsigned int cpu);
int __attribute__ ((__section__(".init.text"))) workqueue_init_early(void);
int __attribute__ ((__section__(".init.text"))) workqueue_init(void);
# 27 "./include/linux/rhashtable.h" 2
# 1 "./include/linux/mutex.h" 1
# 14 "./include/linux/mutex.h"
# 1 "./arch/arm/include/generated/asm/current.h" 1
# 15 "./include/linux/mutex.h" 2
# 1 "./include/linux/osq_lock.h" 1
struct optimistic_spin_node {
struct optimistic_spin_node *next, *prev;
int locked;
int cpu;
};
struct optimistic_spin_queue {
atomic_t tail;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void osq_lock_init(struct optimistic_spin_queue *lock)
{
({ union { typeof(((&lock->tail)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&lock->tail)->counter))) (((0))) }; __write_once_size(&(((&lock->tail)->counter)), __u.__c, sizeof(((&lock->tail)->counter))); __u.__val; });
}
extern bool osq_lock(struct optimistic_spin_queue *lock);
extern void osq_unlock(struct optimistic_spin_queue *lock);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool osq_is_locked(struct optimistic_spin_queue *lock)
{
return ({ union { typeof((&lock->tail)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&lock->tail)->counter), __u.__c, sizeof((&lock->tail)->counter)); else __read_once_size_nocheck(&((&lock->tail)->counter), __u.__c, sizeof((&lock->tail)->counter)); do { } while (0); __u.__val; }) != (0);
}
# 22 "./include/linux/mutex.h" 2
# 1 "./include/linux/debug_locks.h" 1
struct task_struct;
extern int debug_locks;
extern int debug_locks_silent;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __debug_locks_off(void)
{
return ({ typeof(({ (__typeof__(*(&debug_locks)))__xchg((unsigned long)(0), (&debug_locks), sizeof(*(&debug_locks))); })) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = ({ (__typeof__(*(&debug_locks)))__xchg((unsigned long)(0), (&debug_locks), sizeof(*(&debug_locks))); }); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
}
extern int debug_locks_off(void);
# 49 "./include/linux/debug_locks.h"
struct task_struct;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void debug_show_all_locks(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void debug_show_held_locks(struct task_struct *task)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_check_no_locks_freed(const void *from, unsigned long len)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
debug_check_no_locks_held(void)
{
}
# 23 "./include/linux/mutex.h" 2
struct ww_acquire_ctx;
# 54 "./include/linux/mutex.h"
struct mutex {
atomic_long_t owner;
spinlock_t wait_lock;
struct optimistic_spin_queue osq;
struct list_head wait_list;
void *magic;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct task_struct *__mutex_owner(struct mutex *lock)
{
return (struct task_struct *)(atomic_long_read(&lock->owner) & ~0x07);
}
struct mutex_waiter {
struct list_head list;
struct task_struct *task;
struct ww_acquire_ctx *ww_ctx;
void *magic;
};
extern void mutex_destroy(struct mutex *lock);
# 134 "./include/linux/mutex.h"
extern void __mutex_init(struct mutex *lock, const char *name,
struct lock_class_key *key);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mutex_is_locked(struct mutex *lock)
{
return __mutex_owner(lock) != ((void *)0);
}
# 177 "./include/linux/mutex.h"
extern void mutex_lock(struct mutex *lock);
extern int __attribute__((warn_unused_result)) mutex_lock_interruptible(struct mutex *lock);
extern int __attribute__((warn_unused_result)) mutex_lock_killable(struct mutex *lock);
extern void mutex_lock_io(struct mutex *lock);
# 195 "./include/linux/mutex.h"
extern int mutex_trylock(struct mutex *lock);
extern void mutex_unlock(struct mutex *lock);
extern int atomic_dec_and_mutex_lock(atomic_t *cnt, struct mutex *lock);
enum mutex_trylock_recursive_enum {
MUTEX_TRYLOCK_FAILED = 0,
MUTEX_TRYLOCK_SUCCESS = 1,
MUTEX_TRYLOCK_RECURSIVE,
};
# 222 "./include/linux/mutex.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result)) enum mutex_trylock_recursive_enum
mutex_trylock_recursive(struct mutex *lock)
{
if (__builtin_expect(!!(__mutex_owner(lock) == (current_thread_info()->task)), 0))
return MUTEX_TRYLOCK_RECURSIVE;
return mutex_trylock(lock);
}
# 28 "./include/linux/rhashtable.h" 2
# 67 "./include/linux/rhashtable.h"
struct rhash_head {
struct rhash_head *next;
};
struct rhlist_head {
struct rhash_head rhead;
struct rhlist_head *next;
};
# 90 "./include/linux/rhashtable.h"
struct bucket_table {
unsigned int size;
unsigned int nest;
unsigned int rehash;
u32 hash_rnd;
unsigned int locks_mask;
spinlock_t *locks;
struct list_head walkers;
struct callback_head rcu;
struct bucket_table *future_tbl;
struct rhash_head *buckets[] __attribute__((__aligned__((1 << 6))));
};
struct rhashtable_compare_arg {
struct rhashtable *ht;
const void *key;
};
typedef u32 (*rht_hashfn_t)(const void *data, u32 len, u32 seed);
typedef u32 (*rht_obj_hashfn_t)(const void *data, u32 len, u32 seed);
typedef int (*rht_obj_cmpfn_t)(struct rhashtable_compare_arg *arg,
const void *obj);
struct rhashtable;
# 137 "./include/linux/rhashtable.h"
struct rhashtable_params {
u16 nelem_hint;
u16 key_len;
u16 key_offset;
u16 head_offset;
unsigned int max_size;
u16 min_size;
bool automatic_shrinking;
u8 locks_mul;
u32 nulls_base;
rht_hashfn_t hashfn;
rht_obj_hashfn_t obj_hashfn;
rht_obj_cmpfn_t obj_cmpfn;
};
# 164 "./include/linux/rhashtable.h"
struct rhashtable {
struct bucket_table *tbl;
unsigned int key_len;
unsigned int max_elems;
struct rhashtable_params p;
bool rhlist;
struct work_struct run_work;
struct mutex mutex;
spinlock_t lock;
atomic_t nelems;
};
struct rhltable {
struct rhashtable ht;
};
struct rhashtable_walker {
struct list_head list;
struct bucket_table *tbl;
};
# 203 "./include/linux/rhashtable.h"
struct rhashtable_iter {
struct rhashtable *ht;
struct rhash_head *p;
struct rhlist_head *list;
struct rhashtable_walker walker;
unsigned int slot;
unsigned int skip;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long rht_marker(const struct rhashtable *ht, u32 hash)
{
return (1UL | (((long)ht->p.nulls_base + hash) << 1));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rht_is_a_nulls(const struct rhash_head *ptr)
{
return ((unsigned long) ptr & 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long rht_get_nulls_value(const struct rhash_head *ptr)
{
return ((unsigned long) ptr) >> 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *rht_obj(const struct rhashtable *ht,
const struct rhash_head *he)
{
return (char *)he - ht->p.head_offset;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int rht_bucket_index(const struct bucket_table *tbl,
unsigned int hash)
{
return (hash >> (4 + 1)) & (tbl->size - 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int rht_key_hashfn(
struct rhashtable *ht, const struct bucket_table *tbl,
const void *key, const struct rhashtable_params params)
{
unsigned int hash;
if (!__builtin_constant_p(params.key_len))
hash = ht->p.hashfn(key, ht->key_len, tbl->hash_rnd);
else if (params.key_len) {
unsigned int key_len = params.key_len;
if (params.hashfn)
hash = params.hashfn(key, key_len, tbl->hash_rnd);
else if (key_len & (sizeof(u32) - 1))
hash = jhash(key, key_len, tbl->hash_rnd);
else
hash = jhash2(key, key_len / sizeof(u32),
tbl->hash_rnd);
} else {
unsigned int key_len = ht->p.key_len;
if (params.hashfn)
hash = params.hashfn(key, key_len, tbl->hash_rnd);
else
hash = jhash(key, key_len, tbl->hash_rnd);
}
return rht_bucket_index(tbl, hash);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int rht_head_hashfn(
struct rhashtable *ht, const struct bucket_table *tbl,
const struct rhash_head *he, const struct rhashtable_params params)
{
const char *ptr = rht_obj(ht, he);
return __builtin_expect(!!(params.obj_hashfn), 1) ?
rht_bucket_index(tbl, params.obj_hashfn(ptr, params.key_len ?:
ht->p.key_len,
tbl->hash_rnd)) :
rht_key_hashfn(ht, tbl, ptr + params.key_offset, params);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rht_grow_above_75(const struct rhashtable *ht,
const struct bucket_table *tbl)
{
return ({ union { typeof((&ht->nelems)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); else __read_once_size_nocheck(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); do { } while (0); __u.__val; }) > (tbl->size / 4 * 3) &&
(!ht->p.max_size || tbl->size < ht->p.max_size);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rht_shrink_below_30(const struct rhashtable *ht,
const struct bucket_table *tbl)
{
return ({ union { typeof((&ht->nelems)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); else __read_once_size_nocheck(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); do { } while (0); __u.__val; }) < (tbl->size * 3 / 10) &&
tbl->size > ht->p.min_size;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rht_grow_above_100(const struct rhashtable *ht,
const struct bucket_table *tbl)
{
return ({ union { typeof((&ht->nelems)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); else __read_once_size_nocheck(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); do { } while (0); __u.__val; }) > tbl->size &&
(!ht->p.max_size || tbl->size < ht->p.max_size);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rht_grow_above_max(const struct rhashtable *ht,
const struct bucket_table *tbl)
{
return ({ union { typeof((&ht->nelems)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); else __read_once_size_nocheck(&((&ht->nelems)->counter), __u.__c, sizeof((&ht->nelems)->counter)); do { } while (0); __u.__val; }) >= ht->max_elems;
}
# 348 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *rht_bucket_lock(const struct bucket_table *tbl,
unsigned int hash)
{
return &tbl->locks[hash & tbl->locks_mask];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int lockdep_rht_mutex_is_held(struct rhashtable *ht)
{
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int lockdep_rht_bucket_is_held(const struct bucket_table *tbl,
u32 hash)
{
return 1;
}
int rhashtable_init(struct rhashtable *ht,
const struct rhashtable_params *params);
int rhltable_init(struct rhltable *hlt,
const struct rhashtable_params *params);
void *rhashtable_insert_slow(struct rhashtable *ht, const void *key,
struct rhash_head *obj);
void rhashtable_walk_enter(struct rhashtable *ht,
struct rhashtable_iter *iter);
void rhashtable_walk_exit(struct rhashtable_iter *iter);
int rhashtable_walk_start(struct rhashtable_iter *iter) ;
void *rhashtable_walk_next(struct rhashtable_iter *iter);
void rhashtable_walk_stop(struct rhashtable_iter *iter) ;
void rhashtable_free_and_destroy(struct rhashtable *ht,
void (*free_fn)(void *ptr, void *arg),
void *arg);
void rhashtable_destroy(struct rhashtable *ht);
struct rhash_head **rht_bucket_nested(const struct bucket_table *tbl,
unsigned int hash);
struct rhash_head **rht_bucket_nested_insert(struct rhashtable *ht,
struct bucket_table *tbl,
unsigned int hash);
# 411 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct rhash_head *const *rht_bucket(
const struct bucket_table *tbl, unsigned int hash)
{
return __builtin_expect(!!(tbl->nest), 0) ? rht_bucket_nested(tbl, hash) :
&tbl->buckets[hash];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct rhash_head **rht_bucket_var(
struct bucket_table *tbl, unsigned int hash)
{
return __builtin_expect(!!(tbl->nest), 0) ? rht_bucket_nested(tbl, hash) :
&tbl->buckets[hash];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct rhash_head **rht_bucket_insert(
struct rhashtable *ht, struct bucket_table *tbl, unsigned int hash)
{
return __builtin_expect(!!(tbl->nest), 0) ? rht_bucket_nested_insert(ht, tbl, hash) :
&tbl->buckets[hash];
}
# 590 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhashtable_compare(struct rhashtable_compare_arg *arg,
const void *obj)
{
struct rhashtable *ht = arg->ht;
const char *ptr = obj;
return memcmp(ptr + ht->p.key_offset, arg->key, ht->p.key_len);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct rhash_head *__rhashtable_lookup(
struct rhashtable *ht, const void *key,
const struct rhashtable_params params)
{
struct rhashtable_compare_arg arg = {
.ht = ht,
.key = key,
};
struct bucket_table *tbl;
struct rhash_head *he;
unsigned int hash;
tbl = ({ typeof(*(ht->tbl)) *________p1 = (typeof(*(ht->tbl)) *)({ union { typeof((ht->tbl)) __val; char __c[1]; } __u; if (1) __read_once_size(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); else __read_once_size_nocheck(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(ht->tbl)) *)(________p1)); });
restart:
hash = rht_key_hashfn(ht, tbl, key, params);
for (({__asm__ __volatile__("": : :"memory"); }), he = ({ typeof(*(*rht_bucket(tbl, hash))) *________p1 = (typeof(*(*rht_bucket(tbl, hash))) *)({ union { typeof((*rht_bucket(tbl, hash))) __val; char __c[1]; } __u; if (1) __read_once_size(&((*rht_bucket(tbl, hash))), __u.__c, sizeof((*rht_bucket(tbl, hash)))); else __read_once_size_nocheck(&((*rht_bucket(tbl, hash))), __u.__c, sizeof((*rht_bucket(tbl, hash)))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(*rht_bucket(tbl, hash))) *)(________p1)); }); !rht_is_a_nulls(he); he = ({ typeof(he->next) ________p1 = ({ union { typeof(he->next) __val; char __c[1]; } __u; if (1) __read_once_size(&(he->next), __u.__c, sizeof(he->next)); else __read_once_size_nocheck(&(he->next), __u.__c, sizeof(he->next)); do { } while (0); __u.__val; }); ((typeof(*he->next) *)(________p1)); })) {
if (params.obj_cmpfn ?
params.obj_cmpfn(&arg, rht_obj(ht, he)) :
rhashtable_compare(&arg, rht_obj(ht, he)))
continue;
return he;
}
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
tbl = ({ typeof(*(tbl->future_tbl)) *________p1 = (typeof(*(tbl->future_tbl)) *)({ union { typeof((tbl->future_tbl)) __val; char __c[1]; } __u; if (1) __read_once_size(&((tbl->future_tbl)), __u.__c, sizeof((tbl->future_tbl))); else __read_once_size_nocheck(&((tbl->future_tbl)), __u.__c, sizeof((tbl->future_tbl))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(tbl->future_tbl)) *)(________p1)); });
if (__builtin_expect(!!(tbl), 0))
goto restart;
return ((void *)0);
}
# 646 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *rhashtable_lookup(
struct rhashtable *ht, const void *key,
const struct rhashtable_params params)
{
struct rhash_head *he = __rhashtable_lookup(ht, key, params);
return he ? rht_obj(ht, he) : ((void *)0);
}
# 669 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *rhashtable_lookup_fast(
struct rhashtable *ht, const void *key,
const struct rhashtable_params params)
{
void *obj;
rcu_read_lock();
obj = rhashtable_lookup(ht, key, params);
rcu_read_unlock();
return obj;
}
# 696 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct rhlist_head *rhltable_lookup(
struct rhltable *hlt, const void *key,
const struct rhashtable_params params)
{
struct rhash_head *he = __rhashtable_lookup(&hlt->ht, key, params);
return he ? ({ void *__mptr = (void *)(he); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(he)), typeof(((struct rhlist_head *)0)->rhead)) && !__builtin_types_compatible_p(typeof(*(he)), typeof(void)))); extern void __compiletime_assert_13(void) ; if (__cond) __compiletime_assert_13(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct rhlist_head *)(__mptr - __builtin_offsetof(struct rhlist_head, rhead))); }) : ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *__rhashtable_insert_fast(
struct rhashtable *ht, const void *key, struct rhash_head *obj,
const struct rhashtable_params params, bool rhlist)
{
struct rhashtable_compare_arg arg = {
.ht = ht,
.key = key,
};
struct rhash_head **pprev;
struct bucket_table *tbl;
struct rhash_head *head;
spinlock_t *lock;
unsigned int hash;
int elasticity;
void *data;
rcu_read_lock();
tbl = ({ typeof(*(ht->tbl)) *________p1 = (typeof(*(ht->tbl)) *)({ union { typeof((ht->tbl)) __val; char __c[1]; } __u; if (1) __read_once_size(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); else __read_once_size_nocheck(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(ht->tbl)) *)(________p1)); });
hash = rht_head_hashfn(ht, tbl, obj, params);
lock = rht_bucket_lock(tbl, hash);
spin_lock_bh(lock);
if (__builtin_expect(!!(({ do { } while (0); ; ((typeof(*(tbl->future_tbl)) *)((tbl->future_tbl))); })), 0)) {
slow_path:
spin_unlock_bh(lock);
rcu_read_unlock();
return rhashtable_insert_slow(ht, key, obj);
}
elasticity = 16u;
pprev = rht_bucket_insert(ht, tbl, hash);
data = ERR_PTR(-12);
if (!pprev)
goto out;
for (head = ({ do { } while (0); ; ((typeof(*(*pprev)) *)((*pprev))); }); !rht_is_a_nulls(head); head = ({ do { } while (0); ; ((typeof(*((head)->next)) *)(((head)->next))); })) {
struct rhlist_head *plist;
struct rhlist_head *list;
elasticity--;
if (!key ||
(params.obj_cmpfn ?
params.obj_cmpfn(&arg, rht_obj(ht, head)) :
rhashtable_compare(&arg, rht_obj(ht, head)))) {
pprev = &head->next;
continue;
}
data = rht_obj(ht, head);
if (!rhlist)
goto out;
list = ({ void *__mptr = (void *)(obj); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(obj)), typeof(((struct rhlist_head *)0)->rhead)) && !__builtin_types_compatible_p(typeof(*(obj)), typeof(void)))); extern void __compiletime_assert_14(void) ; if (__cond) __compiletime_assert_14(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct rhlist_head *)(__mptr - __builtin_offsetof(struct rhlist_head, rhead))); });
plist = ({ void *__mptr = (void *)(head); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(head)), typeof(((struct rhlist_head *)0)->rhead)) && !__builtin_types_compatible_p(typeof(*(head)), typeof(void)))); extern void __compiletime_assert_15(void) ; if (__cond) __compiletime_assert_15(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct rhlist_head *)(__mptr - __builtin_offsetof(struct rhlist_head, rhead))); });
do { ; ({ union { typeof(list->next) __val; char __c[1]; } __u = { .__val = ( typeof(list->next)) ((typeof(*(plist)) *)(plist)) }; __write_once_size(&(list->next), __u.__c, sizeof(list->next)); __u.__val; }); } while (0);
head = ({ do { } while (0); ; ((typeof(*(head->next)) *)((head->next))); });
do { ; ({ union { typeof(list->rhead.next) __val; char __c[1]; } __u = { .__val = ( typeof(list->rhead.next)) ((typeof(*(head)) *)(head)) }; __write_once_size(&(list->rhead.next), __u.__c, sizeof(list->rhead.next)); __u.__val; }); } while (0);
({ uintptr_t _r_a_p__v = (uintptr_t)(obj); if (__builtin_constant_p(obj) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((*pprev)) __val; char __c[1]; } __u = { .__val = ( typeof((*pprev))) ((typeof(*pprev))(_r_a_p__v)) }; __write_once_size(&((*pprev)), __u.__c, sizeof((*pprev))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&*pprev) == sizeof(char) || sizeof(*&*pprev) == sizeof(short) || sizeof(*&*pprev) == sizeof(int) || sizeof(*&*pprev) == sizeof(long))); extern void __compiletime_assert_16(void) ; if (__cond) __compiletime_assert_16(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&*pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*&*pprev)) ((typeof(*((typeof(*pprev))_r_a_p__v)) *)((typeof(*pprev))_r_a_p__v)) }; __write_once_size(&(*&*pprev), __u.__c, sizeof(*&*pprev)); __u.__val; }); } while (0); _r_a_p__v; });
goto good;
}
if (elasticity <= 0)
goto slow_path;
data = ERR_PTR(-7);
if (__builtin_expect(!!(rht_grow_above_max(ht, tbl)), 0))
goto out;
if (__builtin_expect(!!(rht_grow_above_100(ht, tbl)), 0))
goto slow_path;
head = ({ do { } while (0); ; ((typeof(*(*pprev)) *)((*pprev))); });
do { ; ({ union { typeof(obj->next) __val; char __c[1]; } __u = { .__val = ( typeof(obj->next)) ((typeof(*(head)) *)(head)) }; __write_once_size(&(obj->next), __u.__c, sizeof(obj->next)); __u.__val; }); } while (0);
if (rhlist) {
struct rhlist_head *list;
list = ({ void *__mptr = (void *)(obj); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(obj)), typeof(((struct rhlist_head *)0)->rhead)) && !__builtin_types_compatible_p(typeof(*(obj)), typeof(void)))); extern void __compiletime_assert_17(void) ; if (__cond) __compiletime_assert_17(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct rhlist_head *)(__mptr - __builtin_offsetof(struct rhlist_head, rhead))); });
do { ; ({ union { typeof(list->next) __val; char __c[1]; } __u = { .__val = ( typeof(list->next)) ((typeof(*(((void *)0))) *)(((void *)0))) }; __write_once_size(&(list->next), __u.__c, sizeof(list->next)); __u.__val; }); } while (0);
}
({ uintptr_t _r_a_p__v = (uintptr_t)(obj); if (__builtin_constant_p(obj) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((*pprev)) __val; char __c[1]; } __u = { .__val = ( typeof((*pprev))) ((typeof(*pprev))(_r_a_p__v)) }; __write_once_size(&((*pprev)), __u.__c, sizeof((*pprev))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&*pprev) == sizeof(char) || sizeof(*&*pprev) == sizeof(short) || sizeof(*&*pprev) == sizeof(int) || sizeof(*&*pprev) == sizeof(long))); extern void __compiletime_assert_18(void) ; if (__cond) __compiletime_assert_18(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&*pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*&*pprev)) ((typeof(*((typeof(*pprev))_r_a_p__v)) *)((typeof(*pprev))_r_a_p__v)) }; __write_once_size(&(*&*pprev), __u.__c, sizeof(*&*pprev)); __u.__val; }); } while (0); _r_a_p__v; });
atomic_add(1, &ht->nelems);
if (rht_grow_above_75(ht, tbl))
schedule_work(&ht->run_work);
good:
data = ((void *)0);
out:
spin_unlock_bh(lock);
rcu_read_unlock();
return data;
}
# 827 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhashtable_insert_fast(
struct rhashtable *ht, struct rhash_head *obj,
const struct rhashtable_params params)
{
void *ret;
ret = __rhashtable_insert_fast(ht, ((void *)0), obj, params, false);
if (IS_ERR(ret))
return PTR_ERR(ret);
return ret == ((void *)0) ? 0 : -17;
}
# 857 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhltable_insert_key(
struct rhltable *hlt, const void *key, struct rhlist_head *list,
const struct rhashtable_params params)
{
return PTR_ERR(__rhashtable_insert_fast(&hlt->ht, key, &list->rhead,
params, true));
}
# 881 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhltable_insert(
struct rhltable *hlt, struct rhlist_head *list,
const struct rhashtable_params params)
{
const char *key = rht_obj(&hlt->ht, &list->rhead);
key += params.key_offset;
return rhltable_insert_key(hlt, key, list, params);
}
# 913 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhashtable_lookup_insert_fast(
struct rhashtable *ht, struct rhash_head *obj,
const struct rhashtable_params params)
{
const char *key = rht_obj(ht, obj);
void *ret;
do { if (__builtin_expect(!!(ht->p.obj_hashfn), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/rhashtable.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "920" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
ret = __rhashtable_insert_fast(ht, key + ht->p.key_offset, obj, params,
false);
if (IS_ERR(ret))
return PTR_ERR(ret);
return ret == ((void *)0) ? 0 : -17;
}
# 940 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *rhashtable_lookup_get_insert_fast(
struct rhashtable *ht, struct rhash_head *obj,
const struct rhashtable_params params)
{
const char *key = rht_obj(ht, obj);
do { if (__builtin_expect(!!(ht->p.obj_hashfn), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/rhashtable.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "946" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
return __rhashtable_insert_fast(ht, key + ht->p.key_offset, obj, params,
false);
}
# 974 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhashtable_lookup_insert_key(
struct rhashtable *ht, const void *key, struct rhash_head *obj,
const struct rhashtable_params params)
{
void *ret;
do { if (__builtin_expect(!!(!ht->p.obj_hashfn || !key), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/rhashtable.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "980" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
ret = __rhashtable_insert_fast(ht, key, obj, params, false);
if (IS_ERR(ret))
return PTR_ERR(ret);
return ret == ((void *)0) ? 0 : -17;
}
# 1000 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *rhashtable_lookup_get_insert_key(
struct rhashtable *ht, const void *key, struct rhash_head *obj,
const struct rhashtable_params params)
{
do { if (__builtin_expect(!!(!ht->p.obj_hashfn || !key), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/rhashtable.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "1004" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
return __rhashtable_insert_fast(ht, key, obj, params, false);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __rhashtable_remove_fast_one(
struct rhashtable *ht, struct bucket_table *tbl,
struct rhash_head *obj, const struct rhashtable_params params,
bool rhlist)
{
struct rhash_head **pprev;
struct rhash_head *he;
spinlock_t * lock;
unsigned int hash;
int err = -2;
hash = rht_head_hashfn(ht, tbl, obj, params);
lock = rht_bucket_lock(tbl, hash);
spin_lock_bh(lock);
pprev = rht_bucket_var(tbl, hash);
for (he = ({ do { } while (0); ; ((typeof(*(*pprev)) *)((*pprev))); }); !rht_is_a_nulls(he); he = ({ do { } while (0); ; ((typeof(*((he)->next)) *)(((he)->next))); })) {
struct rhlist_head *list;
list = ({ void *__mptr = (void *)(he); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(he)), typeof(((struct rhlist_head *)0)->rhead)) && !__builtin_types_compatible_p(typeof(*(he)), typeof(void)))); extern void __compiletime_assert_19(void) ; if (__cond) __compiletime_assert_19(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct rhlist_head *)(__mptr - __builtin_offsetof(struct rhlist_head, rhead))); });
if (he != obj) {
struct rhlist_head **lpprev;
pprev = &he->next;
if (!rhlist)
continue;
do {
lpprev = &list->next;
list = ({ do { } while (0); ; ((typeof(*(list->next)) *)((list->next))); });
} while (list && obj != &list->rhead);
if (!list)
continue;
list = ({ do { } while (0); ; ((typeof(*(list->next)) *)((list->next))); });
do { ; ({ union { typeof(*lpprev) __val; char __c[1]; } __u = { .__val = ( typeof(*lpprev)) ((typeof(*(list)) *)(list)) }; __write_once_size(&(*lpprev), __u.__c, sizeof(*lpprev)); __u.__val; }); } while (0);
err = 0;
break;
}
obj = ({ do { } while (0); ; ((typeof(*(obj->next)) *)((obj->next))); });
err = 1;
if (rhlist) {
list = ({ do { } while (0); ; ((typeof(*(list->next)) *)((list->next))); });
if (list) {
do { ; ({ union { typeof(list->rhead.next) __val; char __c[1]; } __u = { .__val = ( typeof(list->rhead.next)) ((typeof(*(obj)) *)(obj)) }; __write_once_size(&(list->rhead.next), __u.__c, sizeof(list->rhead.next)); __u.__val; }); } while (0);
obj = &list->rhead;
err = 0;
}
}
({ uintptr_t _r_a_p__v = (uintptr_t)(obj); if (__builtin_constant_p(obj) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((*pprev)) __val; char __c[1]; } __u = { .__val = ( typeof((*pprev))) ((typeof(*pprev))(_r_a_p__v)) }; __write_once_size(&((*pprev)), __u.__c, sizeof((*pprev))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&*pprev) == sizeof(char) || sizeof(*&*pprev) == sizeof(short) || sizeof(*&*pprev) == sizeof(int) || sizeof(*&*pprev) == sizeof(long))); extern void __compiletime_assert_20(void) ; if (__cond) __compiletime_assert_20(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&*pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*&*pprev)) ((typeof(*((typeof(*pprev))_r_a_p__v)) *)((typeof(*pprev))_r_a_p__v)) }; __write_once_size(&(*&*pprev), __u.__c, sizeof(*&*pprev)); __u.__val; }); } while (0); _r_a_p__v; });
break;
}
spin_unlock_bh(lock);
if (err > 0) {
atomic_sub(1, &ht->nelems);
if (__builtin_expect(!!(ht->p.automatic_shrinking && rht_shrink_below_30(ht, tbl)), 0))
schedule_work(&ht->run_work);
err = 0;
}
return err;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __rhashtable_remove_fast(
struct rhashtable *ht, struct rhash_head *obj,
const struct rhashtable_params params, bool rhlist)
{
struct bucket_table *tbl;
int err;
rcu_read_lock();
tbl = ({ typeof(*(ht->tbl)) *________p1 = (typeof(*(ht->tbl)) *)({ union { typeof((ht->tbl)) __val; char __c[1]; } __u; if (1) __read_once_size(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); else __read_once_size_nocheck(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(ht->tbl)) *)(________p1)); });
while ((err = __rhashtable_remove_fast_one(ht, tbl, obj, params,
rhlist)) &&
(tbl = ({ typeof(*(tbl->future_tbl)) *________p1 = (typeof(*(tbl->future_tbl)) *)({ union { typeof((tbl->future_tbl)) __val; char __c[1]; } __u; if (1) __read_once_size(&((tbl->future_tbl)), __u.__c, sizeof((tbl->future_tbl))); else __read_once_size_nocheck(&((tbl->future_tbl)), __u.__c, sizeof((tbl->future_tbl))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(tbl->future_tbl)) *)(________p1)); })))
;
rcu_read_unlock();
return err;
}
# 1126 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhashtable_remove_fast(
struct rhashtable *ht, struct rhash_head *obj,
const struct rhashtable_params params)
{
return __rhashtable_remove_fast(ht, obj, params, false);
}
# 1148 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhltable_remove(
struct rhltable *hlt, struct rhlist_head *list,
const struct rhashtable_params params)
{
return __rhashtable_remove_fast(&hlt->ht, &list->rhead, params, true);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __rhashtable_replace_fast(
struct rhashtable *ht, struct bucket_table *tbl,
struct rhash_head *obj_old, struct rhash_head *obj_new,
const struct rhashtable_params params)
{
struct rhash_head **pprev;
struct rhash_head *he;
spinlock_t *lock;
unsigned int hash;
int err = -2;
hash = rht_head_hashfn(ht, tbl, obj_old, params);
if (hash != rht_head_hashfn(ht, tbl, obj_new, params))
return -22;
lock = rht_bucket_lock(tbl, hash);
spin_lock_bh(lock);
pprev = rht_bucket_var(tbl, hash);
for (he = ({ do { } while (0); ; ((typeof(*(*pprev)) *)((*pprev))); }); !rht_is_a_nulls(he); he = ({ do { } while (0); ; ((typeof(*((he)->next)) *)(((he)->next))); })) {
if (he != obj_old) {
pprev = &he->next;
continue;
}
({ uintptr_t _r_a_p__v = (uintptr_t)(obj_old->next); if (__builtin_constant_p(obj_old->next) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((obj_new->next)) __val; char __c[1]; } __u = { .__val = ( typeof((obj_new->next))) ((typeof(obj_new->next))(_r_a_p__v)) }; __write_once_size(&((obj_new->next)), __u.__c, sizeof((obj_new->next))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&obj_new->next) == sizeof(char) || sizeof(*&obj_new->next) == sizeof(short) || sizeof(*&obj_new->next) == sizeof(int) || sizeof(*&obj_new->next) == sizeof(long))); extern void __compiletime_assert_21(void) ; if (__cond) __compiletime_assert_21(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&obj_new->next) __val; char __c[1]; } __u = { .__val = ( typeof(*&obj_new->next)) ((typeof(*((typeof(obj_new->next))_r_a_p__v)) *)((typeof(obj_new->next))_r_a_p__v)) }; __write_once_size(&(*&obj_new->next), __u.__c, sizeof(*&obj_new->next)); __u.__val; }); } while (0); _r_a_p__v; });
({ uintptr_t _r_a_p__v = (uintptr_t)(obj_new); if (__builtin_constant_p(obj_new) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((*pprev)) __val; char __c[1]; } __u = { .__val = ( typeof((*pprev))) ((typeof(*pprev))(_r_a_p__v)) }; __write_once_size(&((*pprev)), __u.__c, sizeof((*pprev))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&*pprev) == sizeof(char) || sizeof(*&*pprev) == sizeof(short) || sizeof(*&*pprev) == sizeof(int) || sizeof(*&*pprev) == sizeof(long))); extern void __compiletime_assert_22(void) ; if (__cond) __compiletime_assert_22(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&*pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*&*pprev)) ((typeof(*((typeof(*pprev))_r_a_p__v)) *)((typeof(*pprev))_r_a_p__v)) }; __write_once_size(&(*&*pprev), __u.__c, sizeof(*&*pprev)); __u.__val; }); } while (0); _r_a_p__v; });
err = 0;
break;
}
spin_unlock_bh(lock);
return err;
}
# 1210 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhashtable_replace_fast(
struct rhashtable *ht, struct rhash_head *obj_old,
struct rhash_head *obj_new,
const struct rhashtable_params params)
{
struct bucket_table *tbl;
int err;
rcu_read_lock();
tbl = ({ typeof(*(ht->tbl)) *________p1 = (typeof(*(ht->tbl)) *)({ union { typeof((ht->tbl)) __val; char __c[1]; } __u; if (1) __read_once_size(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); else __read_once_size_nocheck(&((ht->tbl)), __u.__c, sizeof((ht->tbl))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(ht->tbl)) *)(________p1)); });
while ((err = __rhashtable_replace_fast(ht, tbl, obj_old,
obj_new, params)) &&
(tbl = ({ typeof(*(tbl->future_tbl)) *________p1 = (typeof(*(tbl->future_tbl)) *)({ union { typeof((tbl->future_tbl)) __val; char __c[1]; } __u; if (1) __read_once_size(&((tbl->future_tbl)), __u.__c, sizeof((tbl->future_tbl))); else __read_once_size_nocheck(&((tbl->future_tbl)), __u.__c, sizeof((tbl->future_tbl))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(tbl->future_tbl)) *)(________p1)); })))
;
rcu_read_unlock();
return err;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rhashtable_walk_init(struct rhashtable *ht,
struct rhashtable_iter *iter, gfp_t gfp)
{
rhashtable_walk_enter(ht, iter);
return 0;
}
# 1265 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rhltable_walk_enter(struct rhltable *hlt,
struct rhashtable_iter *iter)
{
return rhashtable_walk_enter(&hlt->ht, iter);
}
# 1279 "./include/linux/rhashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rhltable_free_and_destroy(struct rhltable *hlt,
void (*free_fn)(void *ptr,
void *arg),
void *arg)
{
return rhashtable_free_and_destroy(&hlt->ht, free_fn, arg);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rhltable_destroy(struct rhltable *hlt)
{
return rhltable_free_and_destroy(hlt, ((void *)0), ((void *)0));
}
# 8 "./include/linux/ipc.h" 2
# 1 "./include/uapi/linux/ipc.h" 1
# 10 "./include/uapi/linux/ipc.h"
struct ipc_perm
{
__kernel_key_t key;
__kernel_uid_t uid;
__kernel_gid_t gid;
__kernel_uid_t cuid;
__kernel_gid_t cgid;
__kernel_mode_t mode;
unsigned short seq;
};
# 1 "./arch/arm/include/generated/uapi/asm/ipcbuf.h" 1
# 1 "./include/uapi/asm-generic/ipcbuf.h" 1
# 20 "./include/uapi/asm-generic/ipcbuf.h"
struct ipc64_perm {
__kernel_key_t key;
__kernel_uid32_t uid;
__kernel_gid32_t gid;
__kernel_uid32_t cuid;
__kernel_gid32_t cgid;
__kernel_mode_t mode;
unsigned char __pad1[4 - sizeof(__kernel_mode_t)];
unsigned short seq;
unsigned short __pad2;
__kernel_ulong_t __unused1;
__kernel_ulong_t __unused2;
};
# 2 "./arch/arm/include/generated/uapi/asm/ipcbuf.h" 2
# 23 "./include/uapi/linux/ipc.h" 2
# 58 "./include/uapi/linux/ipc.h"
struct ipc_kludge {
struct msgbuf *msgp;
long msgtyp;
};
# 9 "./include/linux/ipc.h" 2
# 1 "./include/linux/refcount.h" 1
# 18 "./include/linux/refcount.h"
typedef struct refcount_struct {
atomic_t refs;
} refcount_t;
# 29 "./include/linux/refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void refcount_set(refcount_t *r, unsigned int n)
{
({ union { typeof(((&r->refs)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&r->refs)->counter))) ((n)) }; __write_once_size(&(((&r->refs)->counter)), __u.__c, sizeof(((&r->refs)->counter))); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int refcount_read(const refcount_t *r)
{
return ({ union { typeof((&r->refs)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&r->refs)->counter), __u.__c, sizeof((&r->refs)->counter)); else __read_once_size_nocheck(&((&r->refs)->counter), __u.__c, sizeof((&r->refs)->counter)); do { } while (0); __u.__val; });
}
extern __attribute__((warn_unused_result)) bool refcount_add_not_zero(unsigned int i, refcount_t *r);
extern void refcount_add(unsigned int i, refcount_t *r);
extern __attribute__((warn_unused_result)) bool refcount_inc_not_zero(refcount_t *r);
extern void refcount_inc(refcount_t *r);
extern __attribute__((warn_unused_result)) bool refcount_sub_and_test(unsigned int i, refcount_t *r);
extern __attribute__((warn_unused_result)) bool refcount_dec_and_test(refcount_t *r);
extern void refcount_dec(refcount_t *r);
# 97 "./include/linux/refcount.h"
extern __attribute__((warn_unused_result)) bool refcount_dec_if_one(refcount_t *r);
extern __attribute__((warn_unused_result)) bool refcount_dec_not_one(refcount_t *r);
extern __attribute__((warn_unused_result)) bool refcount_dec_and_mutex_lock(refcount_t *r, struct mutex *lock);
extern __attribute__((warn_unused_result)) bool refcount_dec_and_lock(refcount_t *r, spinlock_t *lock);
# 10 "./include/linux/ipc.h" 2
struct kern_ipc_perm {
spinlock_t lock;
bool deleted;
int id;
key_t key;
kuid_t uid;
kgid_t gid;
kuid_t cuid;
kgid_t cgid;
umode_t mode;
unsigned long seq;
void *security;
struct rhash_head khtnode;
struct callback_head rcu;
refcount_t refcount;
} __attribute__((__aligned__((1 << 6)))) ;
# 6 "./include/uapi/linux/sem.h" 2
# 24 "./include/uapi/linux/sem.h"
struct semid_ds {
struct ipc_perm sem_perm;
__kernel_time_t sem_otime;
__kernel_time_t sem_ctime;
struct sem *sem_base;
struct sem_queue *sem_pending;
struct sem_queue **sem_pending_last;
struct sem_undo *undo;
unsigned short sem_nsems;
};
# 1 "./arch/arm/include/generated/uapi/asm/sembuf.h" 1
# 1 "./include/uapi/asm-generic/sembuf.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/bitsperlong.h" 1
# 6 "./include/uapi/asm-generic/sembuf.h" 2
# 24 "./include/uapi/asm-generic/sembuf.h"
struct semid64_ds {
struct ipc64_perm sem_perm;
__kernel_time_t sem_otime;
unsigned long __unused1;
__kernel_time_t sem_ctime;
unsigned long __unused2;
unsigned long sem_nsems;
unsigned long __unused3;
unsigned long __unused4;
};
# 2 "./arch/arm/include/generated/uapi/asm/sembuf.h" 2
# 37 "./include/uapi/linux/sem.h" 2
struct sembuf {
unsigned short sem_num;
short sem_op;
short sem_flg;
};
union semun {
int val;
struct semid_ds *buf;
unsigned short *array;
struct seminfo *__buf;
void *__pad;
};
struct seminfo {
int semmap;
int semmni;
int semmns;
int semmnu;
int semmsl;
int semopm;
int semume;
int semusz;
int semvmx;
int semaem;
};
# 10 "./include/linux/sem.h" 2
struct task_struct;
struct sem {
int semval;
int sempid;
spinlock_t lock;
struct list_head pending_alter;
struct list_head pending_const;
time_t sem_otime;
} __attribute__((__aligned__((1 << 6))));
struct sem_array {
struct kern_ipc_perm sem_perm;
time64_t sem_ctime;
struct list_head pending_alter;
struct list_head pending_const;
struct list_head list_id;
int sem_nsems;
int complex_count;
unsigned int use_global_lock;
struct sem sems[];
} ;
# 59 "./include/linux/sem.h"
struct sysv_sem {
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int copy_semundo(unsigned long clone_flags, struct task_struct *tsk)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void exit_sem(struct task_struct *tsk)
{
return;
}
# 16 "./include/linux/sched.h" 2
# 1 "./include/linux/shm.h" 1
# 1 "./include/uapi/linux/shm.h" 1
# 1 "./include/uapi/asm-generic/hugetlb_encode.h" 1
# 8 "./include/uapi/linux/shm.h" 2
# 28 "./include/uapi/linux/shm.h"
struct shmid_ds {
struct ipc_perm shm_perm;
int shm_segsz;
__kernel_time_t shm_atime;
__kernel_time_t shm_dtime;
__kernel_time_t shm_ctime;
__kernel_ipc_pid_t shm_cpid;
__kernel_ipc_pid_t shm_lpid;
unsigned short shm_nattch;
unsigned short shm_unused;
void *shm_unused2;
void *shm_unused3;
};
# 1 "./arch/arm/include/generated/uapi/asm/shmbuf.h" 1
# 1 "./include/uapi/asm-generic/shmbuf.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/bitsperlong.h" 1
# 6 "./include/uapi/asm-generic/shmbuf.h" 2
# 26 "./include/uapi/asm-generic/shmbuf.h"
struct shmid64_ds {
struct ipc64_perm shm_perm;
size_t shm_segsz;
__kernel_time_t shm_atime;
unsigned long __unused1;
__kernel_time_t shm_dtime;
unsigned long __unused2;
__kernel_time_t shm_ctime;
unsigned long __unused3;
__kernel_pid_t shm_cpid;
__kernel_pid_t shm_lpid;
__kernel_ulong_t shm_nattch;
__kernel_ulong_t __unused4;
__kernel_ulong_t __unused5;
};
struct shminfo64 {
__kernel_ulong_t shmmax;
__kernel_ulong_t shmmin;
__kernel_ulong_t shmmni;
__kernel_ulong_t shmseg;
__kernel_ulong_t shmall;
__kernel_ulong_t __unused1;
__kernel_ulong_t __unused2;
__kernel_ulong_t __unused3;
__kernel_ulong_t __unused4;
};
# 2 "./arch/arm/include/generated/uapi/asm/shmbuf.h" 2
# 44 "./include/uapi/linux/shm.h" 2
# 90 "./include/uapi/linux/shm.h"
struct shminfo {
int shmmax;
int shmmin;
int shmmni;
int shmseg;
int shmall;
};
struct shm_info {
int used_ids;
__kernel_ulong_t shm_tot;
__kernel_ulong_t shm_rss;
__kernel_ulong_t shm_swp;
__kernel_ulong_t swap_attempts;
__kernel_ulong_t swap_successes;
};
# 8 "./include/linux/shm.h" 2
# 1 "./arch/arm/include/asm/shmparam.h" 1
# 9 "./include/linux/shm.h" 2
struct shmid_kernel
{
struct kern_ipc_perm shm_perm;
struct file *shm_file;
unsigned long shm_nattch;
unsigned long shm_segsz;
time64_t shm_atim;
time64_t shm_dtim;
time64_t shm_ctim;
pid_t shm_cprid;
pid_t shm_lprid;
struct user_struct *mlock_user;
struct task_struct *shm_creator;
struct list_head shm_clist;
} ;
# 43 "./include/linux/shm.h"
struct sysv_shm {
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long do_shmat(int shmid, char *shmaddr,
int shmflg, unsigned long *addr,
unsigned long shmlba)
{
return -38;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_file_shm_hugepages(struct file *file)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void exit_shm(struct task_struct *task)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void shm_init_task(struct task_struct *task)
{
}
# 17 "./include/linux/sched.h" 2
# 1 "./include/linux/kcov.h" 1
# 1 "./include/uapi/linux/kcov.h" 1
# 11 "./include/uapi/linux/kcov.h"
struct kcov_remote_arg {
__u32 trace_mode;
__u32 area_size;
__u32 num_handles;
__u64 __attribute__((aligned(8))) common_handle;
__u64 __attribute__((aligned(8))) handles[0];
};
# 26 "./include/uapi/linux/kcov.h"
enum {
# 35 "./include/uapi/linux/kcov.h"
KCOV_TRACE_PC = 0,
KCOV_TRACE_CMP = 1,
};
# 56 "./include/uapi/linux/kcov.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __u64 kcov_remote_handle(__u64 subsys, __u64 inst)
{
if (subsys & ~(0xffull << 56) || inst & ~(0xffffffffull))
return 0;
return subsys | inst;
}
# 6 "./include/linux/kcov.h" 2
struct task_struct;
# 57 "./include/linux/kcov.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_task_init(struct task_struct *t) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_task_exit(struct task_struct *t) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_prepare_switch(struct task_struct *t) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_finish_switch(struct task_struct *t) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_remote_start(u64 handle) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_remote_stop(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 kcov_common_handle(void)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_remote_start_common(u64 id) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kcov_remote_start_usb(u64 id) {}
# 18 "./include/linux/sched.h" 2
# 1 "./include/linux/plist.h" 1
# 81 "./include/linux/plist.h"
struct plist_head {
struct list_head node_list;
};
struct plist_node {
int prio;
struct list_head prio_list;
struct list_head node_list;
};
# 123 "./include/linux/plist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
plist_head_init(struct plist_head *head)
{
INIT_LIST_HEAD(&head->node_list);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void plist_node_init(struct plist_node *node, int prio)
{
node->prio = prio;
INIT_LIST_HEAD(&node->prio_list);
INIT_LIST_HEAD(&node->node_list);
}
extern void plist_add(struct plist_node *node, struct plist_head *head);
extern void plist_del(struct plist_node *node, struct plist_head *head);
extern void plist_requeue(struct plist_node *node, struct plist_head *head);
# 212 "./include/linux/plist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int plist_head_empty(const struct plist_head *head)
{
return list_empty(&head->node_list);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int plist_node_empty(const struct plist_node *node)
{
return list_empty(&node->node_list);
}
# 282 "./include/linux/plist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct plist_node *plist_first(const struct plist_head *head)
{
return ({ void *__mptr = (void *)(head->node_list.next); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(head->node_list.next)), typeof(((struct plist_node *)0)->node_list)) && !__builtin_types_compatible_p(typeof(*(head->node_list.next)), typeof(void)))); extern void __compiletime_assert_23(void) ; if (__cond) __compiletime_assert_23(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct plist_node *)(__mptr - __builtin_offsetof(struct plist_node, node_list))); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct plist_node *plist_last(const struct plist_head *head)
{
return ({ void *__mptr = (void *)(head->node_list.prev); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(head->node_list.prev)), typeof(((struct plist_node *)0)->node_list)) && !__builtin_types_compatible_p(typeof(*(head->node_list.prev)), typeof(void)))); extern void __compiletime_assert_24(void) ; if (__cond) __compiletime_assert_24(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct plist_node *)(__mptr - __builtin_offsetof(struct plist_node, node_list))); });
}
# 20 "./include/linux/sched.h" 2
# 1 "./include/linux/hrtimer.h" 1
# 18 "./include/linux/hrtimer.h"
# 1 "./include/linux/rbtree.h" 1
# 36 "./include/linux/rbtree.h"
struct rb_node {
unsigned long __rb_parent_color;
struct rb_node *rb_right;
struct rb_node *rb_left;
} __attribute__((aligned(sizeof(long))));
struct rb_root {
struct rb_node *rb_node;
};
# 57 "./include/linux/rbtree.h"
struct rb_root_cached {
struct rb_root rb_root;
struct rb_node *rb_leftmost;
};
# 77 "./include/linux/rbtree.h"
extern void rb_insert_color(struct rb_node *, struct rb_root *);
extern void rb_erase(struct rb_node *, struct rb_root *);
extern struct rb_node *rb_next(const struct rb_node *);
extern struct rb_node *rb_prev(const struct rb_node *);
extern struct rb_node *rb_first(const struct rb_root *);
extern struct rb_node *rb_last(const struct rb_root *);
extern void rb_insert_color_cached(struct rb_node *,
struct rb_root_cached *, bool);
extern void rb_erase_cached(struct rb_node *node, struct rb_root_cached *);
extern struct rb_node *rb_first_postorder(const struct rb_root *);
extern struct rb_node *rb_next_postorder(const struct rb_node *);
extern void rb_replace_node(struct rb_node *victim, struct rb_node *new,
struct rb_root *root);
extern void rb_replace_node_rcu(struct rb_node *victim, struct rb_node *new,
struct rb_root *root);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rb_link_node(struct rb_node *node, struct rb_node *parent,
struct rb_node **rb_link)
{
node->__rb_parent_color = (unsigned long)parent;
node->rb_left = node->rb_right = ((void *)0);
*rb_link = node;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rb_link_node_rcu(struct rb_node *node, struct rb_node *parent,
struct rb_node **rb_link)
{
node->__rb_parent_color = (unsigned long)parent;
node->rb_left = node->rb_right = ((void *)0);
({ uintptr_t _r_a_p__v = (uintptr_t)(node); if (__builtin_constant_p(node) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((*rb_link)) __val; char __c[1]; } __u = { .__val = ( typeof((*rb_link))) ((typeof(*rb_link))(_r_a_p__v)) }; __write_once_size(&((*rb_link)), __u.__c, sizeof((*rb_link))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&*rb_link) == sizeof(char) || sizeof(*&*rb_link) == sizeof(short) || sizeof(*&*rb_link) == sizeof(int) || sizeof(*&*rb_link) == sizeof(long))); extern void __compiletime_assert_25(void) ; if (__cond) __compiletime_assert_25(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&*rb_link) __val; char __c[1]; } __u = { .__val = ( typeof(*&*rb_link)) ((typeof(*((typeof(*rb_link))_r_a_p__v)) *)((typeof(*rb_link))_r_a_p__v)) }; __write_once_size(&(*&*rb_link), __u.__c, sizeof(*&*rb_link)); __u.__val; }); } while (0); _r_a_p__v; });
}
# 19 "./include/linux/hrtimer.h" 2
# 1 "./include/linux/percpu.h" 1
# 1 "./include/linux/mmdebug.h" 1
struct page;
struct vm_area_struct;
struct mm_struct;
extern void dump_page(struct page *page, const char *reason);
extern void __dump_page(struct page *page, const char *reason);
void dump_vma(const struct vm_area_struct *vma);
void dump_mm(const struct mm_struct *mm);
# 6 "./include/linux/percpu.h" 2
# 1 "./include/linux/smp.h" 1
# 15 "./include/linux/smp.h"
# 1 "./include/linux/llist.h" 1
# 66 "./include/linux/llist.h"
struct llist_head {
struct llist_node *first;
};
struct llist_node {
struct llist_node *next;
};
# 81 "./include/linux/llist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_llist_head(struct llist_head *list)
{
list->first = ((void *)0);
}
# 199 "./include/linux/llist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool llist_empty(const struct llist_head *head)
{
return (*({ __attribute__((unused)) typeof(head->first) __var = ( typeof(head->first)) 0; (volatile typeof(head->first) *)&(head->first); })) == ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct llist_node *llist_next(struct llist_node *node)
{
return node->next;
}
extern bool llist_add_batch(struct llist_node *new_first,
struct llist_node *new_last,
struct llist_head *head);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool llist_add(struct llist_node *new, struct llist_head *head)
{
return llist_add_batch(new, new, head);
}
# 232 "./include/linux/llist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct llist_node *llist_del_all(struct llist_head *head)
{
return ({ typeof(({ (__typeof__(*(&head->first)))__xchg((unsigned long)(((void *)0)), (&head->first), sizeof(*(&head->first))); })) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = ({ (__typeof__(*(&head->first)))__xchg((unsigned long)(((void *)0)), (&head->first), sizeof(*(&head->first))); }); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
}
extern struct llist_node *llist_del_first(struct llist_head *head);
struct llist_node *llist_reverse_order(struct llist_node *head);
# 16 "./include/linux/smp.h" 2
typedef void (*smp_call_func_t)(void *info);
struct __call_single_data {
struct llist_node llist;
smp_call_func_t func;
void *info;
unsigned int flags;
};
typedef struct __call_single_data call_single_data_t
__attribute__((aligned(sizeof(struct __call_single_data))));
extern unsigned int total_cpus;
int smp_call_function_single(int cpuid, smp_call_func_t func, void *info,
int wait);
int on_each_cpu(smp_call_func_t func, void *info, int wait);
void on_each_cpu_mask(const struct cpumask *mask, smp_call_func_t func,
void *info, bool wait);
void on_each_cpu_cond(bool (*cond_func)(int cpu, void *info),
smp_call_func_t func, void *info, bool wait,
gfp_t gfp_flags);
int smp_call_function_single_async(int cpu, call_single_data_t *csd);
# 1 "./arch/arm/include/asm/smp.h" 1
# 23 "./arch/arm/include/asm/smp.h"
struct seq_file;
extern void show_ipi_list(struct seq_file *, int);
void do_IPI(int ipinr, struct pt_regs *regs);
void handle_IPI(int ipinr, struct pt_regs *regs);
extern void smp_init_cpus(void);
extern void set_smp_cross_call(void (*)(const struct cpumask *, unsigned int));
void secondary_start_kernel(void);
struct secondary_data {
union {
unsigned long mpu_rgn_szr;
u64 pgdir;
};
unsigned long swapper_pg_dir;
void *stack;
};
extern struct secondary_data secondary_data;
extern volatile int pen_release;
extern void secondary_startup(void);
extern void secondary_startup_arm(void);
extern int __cpu_disable(void);
extern void __cpu_die(unsigned int cpu);
extern void arch_send_call_function_single_ipi(int cpu);
extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
extern void arch_send_wakeup_ipi_mask(const struct cpumask *mask);
extern int register_ipi_completion(struct completion *completion, int cpu);
struct smp_operations {
void (*smp_init_cpus)(void);
void (*smp_prepare_cpus)(unsigned int max_cpus);
void (*smp_secondary_init)(unsigned int cpu);
int (*smp_boot_secondary)(unsigned int cpu, struct task_struct *idle);
int (*cpu_kill)(unsigned int cpu);
void (*cpu_die)(unsigned int cpu);
bool (*cpu_can_disable)(unsigned int cpu);
int (*cpu_disable)(unsigned int cpu);
};
struct of_cpu_method {
const char *method;
const struct smp_operations *ops;
};
# 125 "./arch/arm/include/asm/smp.h"
extern void smp_set_ops(const struct smp_operations *);
# 65 "./include/linux/smp.h" 2
# 74 "./include/linux/smp.h"
extern void smp_send_stop(void);
extern void smp_send_reschedule(int cpu);
extern void smp_prepare_cpus(unsigned int max_cpus);
extern int __cpu_up(unsigned int cpunum, struct task_struct *tidle);
extern void smp_cpus_done(unsigned int max_cpus);
int smp_call_function(smp_call_func_t func, void *info, int wait);
void smp_call_function_many(const struct cpumask *mask,
smp_call_func_t func, void *info, bool wait);
int smp_call_function_any(const struct cpumask *mask,
smp_call_func_t func, void *info, int wait);
void kick_all_cpus_sync(void);
void wake_up_all_idle_cpus(void);
void __attribute__ ((__section__(".init.text"))) call_function_init(void);
void generic_smp_call_function_single_interrupt(void);
void smp_prepare_boot_cpu(void);
extern unsigned int setup_max_cpus;
extern void __attribute__ ((__section__(".init.text"))) setup_nr_cpu_ids(void);
extern void __attribute__ ((__section__(".init.text"))) smp_init(void);
extern int __boot_cpu_id;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_boot_cpu_id(void)
{
return __boot_cpu_id;
}
# 209 "./include/linux/smp.h"
extern void arch_disable_smp_support(void);
extern void arch_enable_nonboot_cpus_begin(void);
extern void arch_enable_nonboot_cpus_end(void);
void smp_setup_processor_id(void);
int smp_call_on_cpu(unsigned int cpu, int (*func)(void *), void *par,
bool phys);
int smpcfd_prepare_cpu(unsigned int cpu);
int smpcfd_dead_cpu(unsigned int cpu);
int smpcfd_dying_cpu(unsigned int cpu);
# 8 "./include/linux/percpu.h" 2
# 1 "./arch/arm/include/asm/percpu.h" 1
# 24 "./arch/arm/include/asm/percpu.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_my_cpu_offset(unsigned long off)
{
asm volatile("mcr p15, 0, %0, c13, c0, 4" : : "r" (off) : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __my_cpu_offset(void)
{
unsigned long off;
asm("mrc p15, 0, %0, c13, c0, 4" : "=r" (off)
: "Q" (*(const unsigned long *)current_stack_pointer));
return off;
}
# 1 "./include/asm-generic/percpu.h" 1
# 1 "./include/linux/percpu-defs.h" 1
# 295 "./include/linux/percpu-defs.h"
extern void __bad_size_call_parameter(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __this_cpu_preempt_check(const char *op) { }
# 8 "./include/asm-generic/percpu.h" 2
# 19 "./include/asm-generic/percpu.h"
extern unsigned long __per_cpu_offset[4];
# 51 "./arch/arm/include/asm/percpu.h" 2
# 14 "./include/linux/percpu.h" 2
# 71 "./include/linux/percpu.h"
extern void *pcpu_base_addr;
extern const unsigned long *pcpu_unit_offsets;
struct pcpu_group_info {
int nr_units;
unsigned long base_offset;
unsigned int *cpu_map;
};
struct pcpu_alloc_info {
size_t static_size;
size_t reserved_size;
size_t dyn_size;
size_t unit_size;
size_t atom_size;
size_t alloc_size;
size_t __ai_size;
int nr_groups;
struct pcpu_group_info groups[];
};
enum pcpu_fc {
PCPU_FC_AUTO,
PCPU_FC_EMBED,
PCPU_FC_PAGE,
PCPU_FC_NR,
};
extern const char * const pcpu_fc_names[PCPU_FC_NR];
extern enum pcpu_fc pcpu_chosen_fc;
typedef void * (*pcpu_fc_alloc_fn_t)(unsigned int cpu, size_t size,
size_t align);
typedef void (*pcpu_fc_free_fn_t)(void *ptr, size_t size);
typedef void (*pcpu_fc_populate_pte_fn_t)(unsigned long addr);
typedef int (pcpu_fc_cpu_distance_fn_t)(unsigned int from, unsigned int to);
extern struct pcpu_alloc_info * __attribute__ ((__section__(".init.text"))) pcpu_alloc_alloc_info(int nr_groups,
int nr_units);
extern void __attribute__ ((__section__(".init.text"))) pcpu_free_alloc_info(struct pcpu_alloc_info *ai);
extern int __attribute__ ((__section__(".init.text"))) pcpu_setup_first_chunk(const struct pcpu_alloc_info *ai,
void *base_addr);
# 132 "./include/linux/percpu.h"
extern void *__alloc_reserved_percpu(size_t size, size_t align);
extern bool __is_kernel_percpu_address(unsigned long addr, unsigned long *can_addr);
extern bool is_kernel_percpu_address(unsigned long addr);
extern void __attribute__ ((__section__(".init.text"))) setup_per_cpu_areas(void);
extern void *__alloc_percpu_gfp(size_t size, size_t align, gfp_t gfp);
extern void *__alloc_percpu(size_t size, size_t align);
extern void free_percpu(void *__pdata);
extern phys_addr_t per_cpu_ptr_to_phys(void *addr);
# 23 "./include/linux/hrtimer.h" 2
# 1 "./include/linux/timerqueue.h" 1
struct timerqueue_node {
struct rb_node node;
ktime_t expires;
};
struct timerqueue_head {
struct rb_root head;
struct timerqueue_node *next;
};
extern bool timerqueue_add(struct timerqueue_head *head,
struct timerqueue_node *node);
extern bool timerqueue_del(struct timerqueue_head *head,
struct timerqueue_node *node);
extern struct timerqueue_node *timerqueue_iterate_next(
struct timerqueue_node *node);
# 35 "./include/linux/timerqueue.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
struct timerqueue_node *timerqueue_getnext(struct timerqueue_head *head)
{
return head->next;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void timerqueue_init(struct timerqueue_node *node)
{
((&node->node)->__rb_parent_color = (unsigned long)(&node->node));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void timerqueue_init_head(struct timerqueue_head *head)
{
head->head = (struct rb_root) { ((void *)0), };
head->next = ((void *)0);
}
# 25 "./include/linux/hrtimer.h" 2
struct hrtimer_clock_base;
struct hrtimer_cpu_base;
enum hrtimer_mode {
HRTIMER_MODE_ABS = 0x0,
HRTIMER_MODE_REL = 0x1,
HRTIMER_MODE_PINNED = 0x02,
HRTIMER_MODE_ABS_PINNED = 0x02,
HRTIMER_MODE_REL_PINNED = 0x03,
};
enum hrtimer_restart {
HRTIMER_NORESTART,
HRTIMER_RESTART,
};
# 93 "./include/linux/hrtimer.h"
struct hrtimer {
struct timerqueue_node node;
ktime_t _softexpires;
enum hrtimer_restart (*function)(struct hrtimer *);
struct hrtimer_clock_base *base;
u8 state;
u8 is_rel;
};
# 109 "./include/linux/hrtimer.h"
struct hrtimer_sleeper {
struct hrtimer timer;
struct task_struct *task;
};
# 130 "./include/linux/hrtimer.h"
struct hrtimer_clock_base {
struct hrtimer_cpu_base *cpu_base;
int index;
clockid_t clockid;
struct timerqueue_head active;
ktime_t (*get_time)(void);
ktime_t offset;
} __attribute__((__aligned__(32)));
enum hrtimer_base_type {
HRTIMER_BASE_MONOTONIC,
HRTIMER_BASE_REALTIME,
HRTIMER_BASE_BOOTTIME,
HRTIMER_BASE_TAI,
HRTIMER_MAX_CLOCK_BASES,
};
# 174 "./include/linux/hrtimer.h"
struct hrtimer_cpu_base {
raw_spinlock_t lock;
seqcount_t seq;
struct hrtimer *running;
unsigned int cpu;
unsigned int active_bases;
unsigned int clock_was_set_seq;
bool migration_enabled;
bool nohz_active;
unsigned int in_hrtirq : 1,
hres_active : 1,
hang_detected : 1;
ktime_t expires_next;
struct hrtimer *next_timer;
unsigned int nr_events;
unsigned int nr_retries;
unsigned int nr_hangs;
unsigned int max_hang_time;
struct hrtimer_clock_base clock_base[HRTIMER_MAX_CLOCK_BASES];
} __attribute__((__aligned__((1 << 6))));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_set_expires(struct hrtimer *timer, ktime_t time)
{
do { bool __cond = !(!(sizeof(struct hrtimer_clock_base) > 32)); extern void __compiletime_assert_26(void) ; if (__cond) __compiletime_assert_26(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
timer->node.expires = time;
timer->_softexpires = time;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_set_expires_range(struct hrtimer *timer, ktime_t time, ktime_t delta)
{
timer->_softexpires = time;
timer->node.expires = ktime_add_safe(time, delta);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_set_expires_range_ns(struct hrtimer *timer, ktime_t time, u64 delta)
{
timer->_softexpires = time;
timer->node.expires = ktime_add_safe(time, ns_to_ktime(delta));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_set_expires_tv64(struct hrtimer *timer, s64 tv64)
{
timer->node.expires = tv64;
timer->_softexpires = tv64;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_add_expires(struct hrtimer *timer, ktime_t time)
{
timer->node.expires = ktime_add_safe(timer->node.expires, time);
timer->_softexpires = ktime_add_safe(timer->_softexpires, time);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_add_expires_ns(struct hrtimer *timer, u64 ns)
{
timer->node.expires = ((timer->node.expires) + (ns));
timer->_softexpires = ((timer->_softexpires) + (ns));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t hrtimer_get_expires(const struct hrtimer *timer)
{
return timer->node.expires;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t hrtimer_get_softexpires(const struct hrtimer *timer)
{
return timer->_softexpires;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 hrtimer_get_expires_tv64(const struct hrtimer *timer)
{
return timer->node.expires;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 hrtimer_get_softexpires_tv64(const struct hrtimer *timer)
{
return timer->_softexpires;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 hrtimer_get_expires_ns(const struct hrtimer *timer)
{
return (timer->node.expires);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t hrtimer_expires_remaining(const struct hrtimer *timer)
{
return ((timer->node.expires) - (timer->base->get_time()));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t hrtimer_cb_get_time(struct hrtimer *timer)
{
return timer->base->get_time();
}
struct clock_event_device;
extern void hrtimer_interrupt(struct clock_event_device *dev);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hrtimer_is_hres_active(struct hrtimer *timer)
{
return timer->base->cpu_base->hres_active;
}
# 290 "./include/linux/hrtimer.h"
extern void clock_was_set_delayed(void);
extern unsigned int hrtimer_resolution;
# 310 "./include/linux/hrtimer.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t
__hrtimer_expires_remaining_adjusted(const struct hrtimer *timer, ktime_t now)
{
ktime_t rem = ((timer->node.expires) - (now));
if (0 && timer->is_rel)
rem -= hrtimer_resolution;
return rem;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t
hrtimer_expires_remaining_adjusted(const struct hrtimer *timer)
{
return __hrtimer_expires_remaining_adjusted(timer,
timer->base->get_time());
}
extern void clock_was_set(void);
extern void timerfd_clock_was_set(void);
extern void hrtimers_resume(void);
extern __attribute__((section(".data..percpu" ""))) __typeof__(struct tick_device) tick_cpu_device;
extern void hrtimer_init(struct hrtimer *timer, clockid_t which_clock,
enum hrtimer_mode mode);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_init_on_stack(struct hrtimer *timer,
clockid_t which_clock,
enum hrtimer_mode mode)
{
hrtimer_init(timer, which_clock, mode);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void destroy_hrtimer_on_stack(struct hrtimer *timer) { }
extern void hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
u64 range_ns, const enum hrtimer_mode mode);
# 374 "./include/linux/hrtimer.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_start(struct hrtimer *timer, ktime_t tim,
const enum hrtimer_mode mode)
{
hrtimer_start_range_ns(timer, tim, 0, mode);
}
extern int hrtimer_cancel(struct hrtimer *timer);
extern int hrtimer_try_to_cancel(struct hrtimer *timer);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_start_expires(struct hrtimer *timer,
enum hrtimer_mode mode)
{
u64 delta;
ktime_t soft, hard;
soft = hrtimer_get_softexpires(timer);
hard = hrtimer_get_expires(timer);
delta = (((hard) - (soft)));
hrtimer_start_range_ns(timer, soft, delta, mode);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hrtimer_restart(struct hrtimer *timer)
{
hrtimer_start_expires(timer, HRTIMER_MODE_ABS);
}
extern ktime_t __hrtimer_get_remaining(const struct hrtimer *timer, bool adjust);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ktime_t hrtimer_get_remaining(const struct hrtimer *timer)
{
return __hrtimer_get_remaining(timer, false);
}
extern u64 hrtimer_get_next_event(void);
extern u64 hrtimer_next_event_without(const struct hrtimer *exclude);
extern bool hrtimer_active(const struct hrtimer *timer);
# 420 "./include/linux/hrtimer.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool hrtimer_is_queued(struct hrtimer *timer)
{
return !!(({ union { typeof(timer->state) __val; char __c[1]; } __u; if (1) __read_once_size(&(timer->state), __u.__c, sizeof(timer->state)); else __read_once_size_nocheck(&(timer->state), __u.__c, sizeof(timer->state)); do { } while (0); __u.__val; }) & 0x01);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hrtimer_callback_running(struct hrtimer *timer)
{
return timer->base->cpu_base->running == timer;
}
extern u64
hrtimer_forward(struct hrtimer *timer, ktime_t now, ktime_t interval);
# 455 "./include/linux/hrtimer.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 hrtimer_forward_now(struct hrtimer *timer,
ktime_t interval)
{
return hrtimer_forward(timer, timer->base->get_time(), interval);
}
extern int nanosleep_copyout(struct restart_block *, struct timespec64 *);
extern long hrtimer_nanosleep(const struct timespec64 *rqtp,
const enum hrtimer_mode mode,
const clockid_t clockid);
extern void hrtimer_init_sleeper(struct hrtimer_sleeper *sl,
struct task_struct *tsk);
extern int schedule_hrtimeout_range(ktime_t *expires, u64 delta,
const enum hrtimer_mode mode);
extern int schedule_hrtimeout_range_clock(ktime_t *expires,
u64 delta,
const enum hrtimer_mode mode,
int clock);
extern int schedule_hrtimeout(ktime_t *expires, const enum hrtimer_mode mode);
extern void hrtimer_run_queues(void);
extern void __attribute__ ((__section__(".init.text"))) hrtimers_init(void);
extern void sysrq_timer_list_show(void);
int hrtimers_prepare_cpu(unsigned int cpu);
int hrtimers_dead_cpu(unsigned int cpu);
# 21 "./include/linux/sched.h" 2
# 1 "./include/linux/seccomp.h" 1
# 1 "./include/uapi/linux/seccomp.h" 1
# 56 "./include/uapi/linux/seccomp.h"
struct seccomp_data {
int nr;
__u32 arch;
__u64 instruction_pointer;
__u64 args[6];
};
# 6 "./include/linux/seccomp.h" 2
# 1 "./arch/arm/include/generated/asm/seccomp.h" 1
# 1 "./include/asm-generic/seccomp.h" 1
# 14 "./include/asm-generic/seccomp.h"
# 1 "./include/uapi/linux/unistd.h" 1
# 1 "./arch/arm/include/asm/unistd.h" 1
# 16 "./arch/arm/include/asm/unistd.h"
# 1 "./arch/arm/include/uapi/asm/unistd.h" 1
# 21 "./arch/arm/include/uapi/asm/unistd.h"
# 1 "./arch/arm/include/generated/uapi/asm/unistd-eabi.h" 1
# 22 "./arch/arm/include/uapi/asm/unistd.h" 2
# 1 "./arch/arm/include/generated/uapi/asm/unistd-common.h" 1
# 28 "./arch/arm/include/uapi/asm/unistd.h" 2
# 17 "./arch/arm/include/asm/unistd.h" 2
# 1 "./arch/arm/include/generated/asm/unistd-nr.h" 1
# 18 "./arch/arm/include/asm/unistd.h" 2
# 9 "./include/uapi/linux/unistd.h" 2
# 15 "./include/asm-generic/seccomp.h" 2
# 2 "./arch/arm/include/generated/asm/seccomp.h" 2
# 15 "./include/linux/seccomp.h" 2
struct seccomp_filter;
# 28 "./include/linux/seccomp.h"
struct seccomp {
int mode;
struct seccomp_filter *filter;
};
extern int __secure_computing(const struct seccomp_data *sd);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int secure_computing(const struct seccomp_data *sd)
{
if (__builtin_expect(!!(test_ti_thread_flag(current_thread_info(), 7)), 0))
return __secure_computing(sd);
return 0;
}
extern long prctl_get_seccomp(void);
extern long prctl_set_seccomp(unsigned long, char *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int seccomp_mode(struct seccomp *s)
{
return s->mode;
}
# 83 "./include/linux/seccomp.h"
extern void put_seccomp_filter(struct task_struct *tsk);
extern void get_seccomp_filter(struct task_struct *tsk);
# 100 "./include/linux/seccomp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long seccomp_get_filter(struct task_struct *task,
unsigned long n, void *data)
{
return -22;
}
# 22 "./include/linux/sched.h" 2
# 1 "./include/linux/nodemask.h" 1
# 96 "./include/linux/nodemask.h"
# 1 "./include/linux/numa.h" 1
# 97 "./include/linux/nodemask.h" 2
typedef struct { unsigned long bits[((((1 << 0)) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8)))]; } nodemask_t;
extern nodemask_t _unused_nodemask_arg_;
# 119 "./include/linux/nodemask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __node_set(int node, volatile nodemask_t *dstp)
{
_set_bit(node,dstp->bits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __node_clear(int node, volatile nodemask_t *dstp)
{
_clear_bit(node,dstp->bits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_setall(nodemask_t *dstp, unsigned int nbits)
{
bitmap_fill(dstp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_clear(nodemask_t *dstp, unsigned int nbits)
{
bitmap_zero(dstp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __node_test_and_set(int node, nodemask_t *addr)
{
return _test_and_set_bit(node,addr->bits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_and(nodemask_t *dstp, const nodemask_t *src1p,
const nodemask_t *src2p, unsigned int nbits)
{
bitmap_and(dstp->bits, src1p->bits, src2p->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_or(nodemask_t *dstp, const nodemask_t *src1p,
const nodemask_t *src2p, unsigned int nbits)
{
bitmap_or(dstp->bits, src1p->bits, src2p->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_xor(nodemask_t *dstp, const nodemask_t *src1p,
const nodemask_t *src2p, unsigned int nbits)
{
bitmap_xor(dstp->bits, src1p->bits, src2p->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_andnot(nodemask_t *dstp, const nodemask_t *src1p,
const nodemask_t *src2p, unsigned int nbits)
{
bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_complement(nodemask_t *dstp,
const nodemask_t *srcp, unsigned int nbits)
{
bitmap_complement(dstp->bits, srcp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodes_equal(const nodemask_t *src1p,
const nodemask_t *src2p, unsigned int nbits)
{
return bitmap_equal(src1p->bits, src2p->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodes_intersects(const nodemask_t *src1p,
const nodemask_t *src2p, unsigned int nbits)
{
return bitmap_intersects(src1p->bits, src2p->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodes_subset(const nodemask_t *src1p,
const nodemask_t *src2p, unsigned int nbits)
{
return bitmap_subset(src1p->bits, src2p->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodes_empty(const nodemask_t *srcp, unsigned int nbits)
{
return bitmap_empty(srcp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodes_full(const nodemask_t *srcp, unsigned int nbits)
{
return bitmap_full(srcp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodes_weight(const nodemask_t *srcp, unsigned int nbits)
{
return bitmap_weight(srcp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_shift_right(nodemask_t *dstp,
const nodemask_t *srcp, int n, int nbits)
{
bitmap_shift_right(dstp->bits, srcp->bits, n, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_shift_left(nodemask_t *dstp,
const nodemask_t *srcp, int n, int nbits)
{
bitmap_shift_left(dstp->bits, srcp->bits, n, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __first_node(const nodemask_t *srcp)
{
return ({ int __UNIQUE_ID_min1_27 = ((1 << 0)); int __UNIQUE_ID_min2_28 = (_find_first_bit_le(srcp->bits,(1 << 0))); (void) (&__UNIQUE_ID_min1_27 == &__UNIQUE_ID_min2_28); __UNIQUE_ID_min1_27 < __UNIQUE_ID_min2_28 ? __UNIQUE_ID_min1_27 : __UNIQUE_ID_min2_28; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __next_node(int n, const nodemask_t *srcp)
{
return ({ int __UNIQUE_ID_min1_29 = ((1 << 0)); int __UNIQUE_ID_min2_30 = (_find_next_bit_le(srcp->bits,(1 << 0),n+1)); (void) (&__UNIQUE_ID_min1_29 == &__UNIQUE_ID_min2_30); __UNIQUE_ID_min1_29 < __UNIQUE_ID_min2_30 ? __UNIQUE_ID_min1_29 : __UNIQUE_ID_min2_30; });
}
int __next_node_in(int node, const nodemask_t *srcp);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_nodemask_of_node(nodemask_t *mask, int node)
{
__nodes_clear(&(*mask), (1 << 0));
__node_set((node), &(*mask));
}
# 290 "./include/linux/nodemask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __first_unset_node(const nodemask_t *maskp)
{
return ({ int __UNIQUE_ID_min1_31 = ((1 << 0)); int __UNIQUE_ID_min2_32 = (_find_first_zero_bit_le(maskp->bits,(1 << 0))); (void) (&__UNIQUE_ID_min1_31 == &__UNIQUE_ID_min2_32); __UNIQUE_ID_min1_31 < __UNIQUE_ID_min2_32 ? __UNIQUE_ID_min1_31 : __UNIQUE_ID_min2_32; });
}
# 324 "./include/linux/nodemask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodemask_parse_user(const char *buf, int len,
nodemask_t *dstp, int nbits)
{
return bitmap_parse_user(buf, len, dstp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __nodelist_parse(const char *buf, nodemask_t *dstp, int nbits)
{
return bitmap_parselist(buf, dstp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __node_remap(int oldbit,
const nodemask_t *oldp, const nodemask_t *newp, int nbits)
{
return bitmap_bitremap(oldbit, oldp->bits, newp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_remap(nodemask_t *dstp, const nodemask_t *srcp,
const nodemask_t *oldp, const nodemask_t *newp, int nbits)
{
bitmap_remap(dstp->bits, srcp->bits, oldp->bits, newp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_onto(nodemask_t *dstp, const nodemask_t *origp,
const nodemask_t *relmapp, int nbits)
{
bitmap_onto(dstp->bits, origp->bits, relmapp->bits, nbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __nodes_fold(nodemask_t *dstp, const nodemask_t *origp,
int sz, int nbits)
{
bitmap_fold(dstp->bits, origp->bits, sz, nbits);
}
# 382 "./include/linux/nodemask.h"
enum node_states {
N_POSSIBLE,
N_ONLINE,
N_NORMAL_MEMORY,
N_HIGH_MEMORY,
N_MEMORY,
N_CPU,
NR_NODE_STATES
};
extern nodemask_t node_states[NR_NODE_STATES];
# 455 "./include/linux/nodemask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int node_state(int node, enum node_states state)
{
return node == 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void node_set_state(int node, enum node_states state)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void node_clear_state(int node, enum node_states state)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int num_node_state(enum node_states state)
{
return 1;
}
# 490 "./include/linux/nodemask.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int node_random(const nodemask_t *mask)
{
return 0;
}
# 522 "./include/linux/nodemask.h"
struct nodemask_scratch {
nodemask_t mask1;
nodemask_t mask2;
};
# 23 "./include/linux/sched.h" 2
# 1 "./include/linux/resource.h" 1
# 1 "./include/uapi/linux/resource.h" 1
# 24 "./include/uapi/linux/resource.h"
struct rusage {
struct timeval ru_utime;
struct timeval ru_stime;
__kernel_long_t ru_maxrss;
__kernel_long_t ru_ixrss;
__kernel_long_t ru_idrss;
__kernel_long_t ru_isrss;
__kernel_long_t ru_minflt;
__kernel_long_t ru_majflt;
__kernel_long_t ru_nswap;
__kernel_long_t ru_inblock;
__kernel_long_t ru_oublock;
__kernel_long_t ru_msgsnd;
__kernel_long_t ru_msgrcv;
__kernel_long_t ru_nsignals;
__kernel_long_t ru_nvcsw;
__kernel_long_t ru_nivcsw;
};
struct rlimit {
__kernel_ulong_t rlim_cur;
__kernel_ulong_t rlim_max;
};
struct rlimit64 {
__u64 rlim_cur;
__u64 rlim_max;
};
# 78 "./include/uapi/linux/resource.h"
# 1 "./arch/arm/include/generated/uapi/asm/resource.h" 1
# 1 "./include/asm-generic/resource.h" 1
# 1 "./include/uapi/asm-generic/resource.h" 1
# 6 "./include/asm-generic/resource.h" 2
# 2 "./arch/arm/include/generated/uapi/asm/resource.h" 2
# 79 "./include/uapi/linux/resource.h" 2
# 6 "./include/linux/resource.h" 2
struct task_struct;
void getrusage(struct task_struct *p, int who, struct rusage *ru);
int do_prlimit(struct task_struct *tsk, unsigned int resource,
struct rlimit *new_rlim, struct rlimit *old_rlim);
# 25 "./include/linux/sched.h" 2
# 1 "./include/linux/latencytop.h" 1
# 14 "./include/linux/latencytop.h"
struct task_struct;
# 46 "./include/linux/latencytop.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
account_scheduler_latency(struct task_struct *task, int usecs, int inter)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_all_latency_tracing(struct task_struct *p)
{
}
# 26 "./include/linux/sched.h" 2
# 1 "./include/linux/sched/prio.h" 1
# 48 "./include/linux/sched/prio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long nice_to_rlimit(long nice)
{
return (19 - nice + 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long rlimit_to_nice(long prio)
{
return (19 - prio + 1);
}
# 27 "./include/linux/sched.h" 2
# 1 "./include/linux/signal_types.h" 1
# 10 "./include/linux/signal_types.h"
# 1 "./include/uapi/linux/signal.h" 1
# 1 "./arch/arm/include/asm/signal.h" 1
# 1 "./arch/arm/include/uapi/asm/signal.h" 1
struct siginfo;
# 94 "./arch/arm/include/uapi/asm/signal.h"
# 1 "./include/uapi/asm-generic/signal-defs.h" 1
# 18 "./include/uapi/asm-generic/signal-defs.h"
typedef void __signalfn_t(int);
typedef __signalfn_t *__sighandler_t;
typedef void __restorefn_t(void);
typedef __restorefn_t *__sigrestore_t;
# 95 "./arch/arm/include/uapi/asm/signal.h" 2
# 114 "./arch/arm/include/uapi/asm/signal.h"
typedef struct sigaltstack {
void *ss_sp;
int ss_flags;
size_t ss_size;
} stack_t;
# 6 "./arch/arm/include/asm/signal.h" 2
typedef unsigned long old_sigset_t;
typedef struct {
unsigned long sig[(64 / 32)];
} sigset_t;
# 1 "./arch/arm/include/uapi/asm/sigcontext.h" 1
# 10 "./arch/arm/include/uapi/asm/sigcontext.h"
struct sigcontext {
unsigned long trap_no;
unsigned long error_code;
unsigned long oldmask;
unsigned long arm_r0;
unsigned long arm_r1;
unsigned long arm_r2;
unsigned long arm_r3;
unsigned long arm_r4;
unsigned long arm_r5;
unsigned long arm_r6;
unsigned long arm_r7;
unsigned long arm_r8;
unsigned long arm_r9;
unsigned long arm_r10;
unsigned long arm_fp;
unsigned long arm_ip;
unsigned long arm_sp;
unsigned long arm_lr;
unsigned long arm_pc;
unsigned long arm_cpsr;
unsigned long fault_address;
};
# 23 "./arch/arm/include/asm/signal.h" 2
# 6 "./include/uapi/linux/signal.h" 2
# 1 "./arch/arm/include/generated/uapi/asm/siginfo.h" 1
# 1 "./include/uapi/asm-generic/siginfo.h" 1
typedef union sigval {
int sival_int;
void *sival_ptr;
} sigval_t;
# 49 "./include/uapi/asm-generic/siginfo.h"
typedef struct siginfo {
int si_signo;
int si_errno;
int si_code;
union {
int _pad[((128 - (3 * sizeof(int))) / sizeof(int))];
struct {
__kernel_pid_t _pid;
__kernel_uid32_t _uid;
} _kill;
struct {
__kernel_timer_t _tid;
int _overrun;
char _pad[sizeof( __kernel_uid32_t) - sizeof(int)];
sigval_t _sigval;
int _sys_private;
} _timer;
struct {
__kernel_pid_t _pid;
__kernel_uid32_t _uid;
sigval_t _sigval;
} _rt;
struct {
__kernel_pid_t _pid;
__kernel_uid32_t _uid;
int _status;
__kernel_clock_t _utime;
__kernel_clock_t _stime;
} _sigchld;
struct {
void *_addr;
short _addr_lsb;
union {
struct {
void *_lower;
void *_upper;
} _addr_bnd;
__u32 _pkey;
};
} _sigfault;
struct {
long _band;
int _fd;
} _sigpoll;
struct {
void *_call_addr;
int _syscall;
unsigned int _arch;
} _sigsys;
} _sifields;
} siginfo_t;
# 281 "./include/uapi/asm-generic/siginfo.h"
typedef struct sigevent {
sigval_t sigev_value;
int sigev_signo;
int sigev_notify;
union {
int _pad[((64 - (sizeof(int) * 2 + sizeof(sigval_t))) / sizeof(int))];
int _tid;
struct {
void (*_function)(sigval_t);
void *_attribute;
} _sigev_thread;
} _sigev_un;
} sigevent_t;
# 2 "./arch/arm/include/generated/uapi/asm/siginfo.h" 2
# 7 "./include/uapi/linux/signal.h" 2
# 11 "./include/linux/signal_types.h" 2
struct sigqueue {
struct list_head list;
int flags;
siginfo_t info;
struct user_struct *user;
};
struct sigpending {
struct list_head list;
sigset_t signal;
};
struct sigaction {
__sighandler_t sa_handler;
unsigned long sa_flags;
__sigrestore_t sa_restorer;
sigset_t sa_mask;
};
struct k_sigaction {
struct sigaction sa;
};
struct old_sigaction {
__sighandler_t sa_handler;
old_sigset_t sa_mask;
unsigned long sa_flags;
__sigrestore_t sa_restorer;
};
struct ksignal {
struct k_sigaction ka;
siginfo_t info;
int sig;
};
# 28 "./include/linux/sched.h" 2
# 1 "./include/linux/mm_types_task.h" 1
# 34 "./include/linux/mm_types_task.h"
struct vmacache {
u64 seqnum;
struct vm_area_struct *vmas[(1U << 2)];
};
enum {
MM_FILEPAGES,
MM_ANONPAGES,
MM_SWAPENTS,
MM_SHMEMPAGES,
NR_MM_COUNTERS
};
struct task_rss_stat {
int events;
int count[NR_MM_COUNTERS];
};
struct mm_rss_stat {
atomic_long_t count[NR_MM_COUNTERS];
};
struct page_frag {
struct page *page;
__u16 offset;
__u16 size;
};
struct tlbflush_unmap_batch {
# 93 "./include/linux/mm_types_task.h"
};
# 29 "./include/linux/sched.h" 2
# 1 "./include/linux/task_io_accounting.h" 1
# 12 "./include/linux/task_io_accounting.h"
struct task_io_accounting {
u64 rchar;
u64 wchar;
u64 syscr;
u64 syscw;
u64 syscfs;
u64 read_bytes;
u64 write_bytes;
# 46 "./include/linux/task_io_accounting.h"
u64 cancelled_write_bytes;
};
# 30 "./include/linux/sched.h" 2
struct audit_context;
struct backing_dev_info;
struct bio_list;
struct blk_plug;
struct cfs_rq;
struct fs_struct;
struct futex_pi_state;
struct io_context;
struct mempolicy;
struct nameidata;
struct nsproxy;
struct perf_event_context;
struct pid_namespace;
struct pipe_inode_info;
struct rcu_node;
struct reclaim_state;
struct robust_list_head;
struct sched_attr;
struct sched_param;
struct seq_file;
struct sighand_struct;
struct signal_struct;
struct task_delay_info;
struct task_group;
# 209 "./include/linux/sched.h"
enum task_event {
PUT_PREV_TASK = 0,
PICK_NEXT_TASK = 1,
TASK_WAKE = 2,
TASK_MIGRATE = 3,
TASK_UPDATE = 4,
IRQ_UPDATE = 5,
};
extern cpumask_var_t cpu_isolated_map;
extern void scheduler_tick(void);
extern long schedule_timeout(long timeout);
extern long schedule_timeout_interruptible(long timeout);
extern long schedule_timeout_killable(long timeout);
extern long schedule_timeout_uninterruptible(long timeout);
extern long schedule_timeout_idle(long timeout);
void schedule(void);
extern void schedule_preempt_disabled(void);
extern int __attribute__((warn_unused_result)) io_schedule_prepare(void);
extern void io_schedule_finish(int token);
extern long io_schedule_timeout(long timeout);
extern void io_schedule(void);
# 246 "./include/linux/sched.h"
struct prev_cputime {
u64 utime;
u64 stime;
raw_spinlock_t lock;
};
# 264 "./include/linux/sched.h"
struct task_cputime {
u64 utime;
u64 stime;
unsigned long long sum_exec_runtime;
};
enum vtime_state {
VTIME_INACTIVE = 0,
VTIME_USER,
VTIME_SYS,
};
struct vtime {
seqcount_t seqcount;
unsigned long long starttime;
enum vtime_state state;
u64 utime;
u64 stime;
u64 gtime;
};
struct sched_info {
unsigned long pcount;
unsigned long long run_delay;
unsigned long long last_arrival;
unsigned long long last_queued;
};
# 324 "./include/linux/sched.h"
struct load_weight {
unsigned long weight;
u32 inv_weight;
};
# 351 "./include/linux/sched.h"
struct util_est {
unsigned int enqueued;
unsigned int ewma;
};
# 409 "./include/linux/sched.h"
struct sched_avg {
u64 last_update_time;
u64 load_sum;
u32 util_sum;
u32 period_contrib;
unsigned long load_avg;
unsigned long util_avg;
struct util_est util_est;
};
struct sched_statistics {
u64 wait_start;
u64 wait_max;
u64 wait_count;
u64 wait_sum;
u64 iowait_count;
u64 iowait_sum;
u64 sleep_start;
u64 sleep_max;
s64 sum_sleep_runtime;
u64 block_start;
u64 block_max;
u64 exec_max;
u64 slice_max;
u64 nr_migrations_cold;
u64 nr_failed_migrations_affine;
u64 nr_failed_migrations_running;
u64 nr_failed_migrations_hot;
u64 nr_forced_migrations;
u64 nr_wakeups;
u64 nr_wakeups_sync;
u64 nr_wakeups_migrate;
u64 nr_wakeups_local;
u64 nr_wakeups_remote;
u64 nr_wakeups_affine;
u64 nr_wakeups_affine_attempts;
u64 nr_wakeups_passive;
u64 nr_wakeups_idle;
};
struct sched_entity {
struct load_weight load;
struct rb_node run_node;
struct list_head group_node;
unsigned int on_rq;
u64 exec_start;
u64 sum_exec_runtime;
u64 vruntime;
u64 prev_sum_exec_runtime;
u64 nr_migrations;
struct sched_statistics statistics;
int depth;
struct sched_entity *parent;
struct cfs_rq *cfs_rq;
struct cfs_rq *my_q;
# 487 "./include/linux/sched.h"
struct sched_avg avg __attribute__((__aligned__((1 << 6))));
};
# 526 "./include/linux/sched.h"
struct sched_rt_entity {
struct list_head run_list;
unsigned long timeout;
unsigned long watchdog_stamp;
unsigned int time_slice;
unsigned short on_rq;
unsigned short on_list;
struct sched_rt_entity *back;
} ;
struct sched_dl_entity {
struct rb_node rb_node;
u64 dl_runtime;
u64 dl_deadline;
u64 dl_period;
u64 dl_bw;
u64 dl_density;
s64 runtime;
u64 deadline;
unsigned int flags;
# 588 "./include/linux/sched.h"
int dl_throttled;
int dl_boosted;
int dl_yielded;
int dl_non_contending;
struct hrtimer dl_timer;
# 606 "./include/linux/sched.h"
struct hrtimer inactive_timer;
};
union rcu_special {
struct {
u8 blocked;
u8 need_qs;
u8 exp_need_qs;
u8 pad;
} b;
u32 s;
};
enum perf_event_task_context {
perf_invalid_context = -1,
perf_hw_context = 0,
perf_sw_context,
perf_nr_task_contexts,
};
struct wake_q_node {
struct wake_q_node *next;
};
struct task_struct {
# 641 "./include/linux/sched.h"
volatile long state;
void *stack;
atomic_t usage;
unsigned int flags;
unsigned int ptrace;
struct llist_node wake_entry;
int on_cpu;
unsigned int wakee_flips;
unsigned long wakee_flip_decay_ts;
struct task_struct *last_wakee;
int wake_cpu;
int on_rq;
int prio;
int static_prio;
int normal_prio;
unsigned int rt_priority;
const struct sched_class *sched_class;
struct sched_entity se;
struct sched_rt_entity rt;
# 689 "./include/linux/sched.h"
struct task_group *sched_task_group;
struct sched_dl_entity dl;
# 702 "./include/linux/sched.h"
unsigned int policy;
int nr_cpus_allowed;
cpumask_t cpus_allowed;
int rcu_read_lock_nesting;
union rcu_special rcu_read_unlock_special;
struct list_head rcu_node_entry;
struct rcu_node *rcu_blocked_node;
unsigned long rcu_tasks_nvcsw;
u8 rcu_tasks_holdout;
u8 rcu_tasks_idx;
int rcu_tasks_idle_cpu;
struct list_head rcu_tasks_holdout_list;
struct sched_info sched_info;
struct list_head tasks;
struct plist_node pushable_tasks;
struct rb_node pushable_dl_tasks;
struct mm_struct *mm;
struct mm_struct *active_mm;
struct vmacache vmacache;
struct task_rss_stat rss_stat;
int exit_state;
int exit_code;
int exit_signal;
int pdeath_signal;
unsigned long jobctl;
unsigned int personality;
unsigned sched_reset_on_fork:1;
unsigned sched_contributes_to_load:1;
unsigned sched_migrated:1;
unsigned sched_remote_wakeup:1;
unsigned sched_psi_wake_requeue:1;
unsigned :0;
unsigned in_execve:1;
unsigned in_iowait:1;
unsigned memcg_may_oom:1;
unsigned memcg_kmem_skip_account:1;
unsigned brk_randomized:1;
unsigned no_cgroup_migration:1;
unsigned long atomic_flags;
struct restart_block restart_block;
pid_t pid;
pid_t tgid;
unsigned long stack_canary;
# 801 "./include/linux/sched.h"
struct task_struct *real_parent;
struct task_struct *parent;
struct list_head children;
struct list_head sibling;
struct task_struct *group_leader;
struct list_head ptraced;
struct list_head ptrace_entry;
struct pid_link pids[PIDTYPE_MAX];
struct list_head thread_group;
struct list_head thread_node;
struct completion *vfork_done;
int *set_child_tid;
int *clear_child_tid;
u64 utime;
u64 stime;
u64 gtime;
u64 *time_in_state;
unsigned int max_state;
struct prev_cputime prev_cputime;
# 855 "./include/linux/sched.h"
unsigned long nvcsw;
unsigned long nivcsw;
u64 start_time;
u64 real_start_time;
unsigned long min_flt;
unsigned long maj_flt;
struct task_cputime cputime_expires;
struct list_head cpu_timers[3];
const struct cred *ptracer_cred;
const struct cred *real_cred;
const struct cred *cred;
# 891 "./include/linux/sched.h"
char comm[16];
struct nameidata *nameidata;
# 903 "./include/linux/sched.h"
struct fs_struct *fs;
struct files_struct *files;
struct nsproxy *nsproxy;
struct signal_struct *signal;
struct sighand_struct *sighand;
sigset_t blocked;
sigset_t real_blocked;
sigset_t saved_sigmask;
struct sigpending pending;
unsigned long sas_ss_sp;
size_t sas_ss_size;
unsigned int sas_ss_flags;
struct callback_head *task_works;
struct audit_context *audit_context;
kuid_t loginuid;
unsigned int sessionid;
struct seccomp seccomp;
u64 parent_exec_id;
u64 self_exec_id;
spinlock_t alloc_lock;
raw_spinlock_t pi_lock;
struct wake_q_node wake_q;
struct rb_root_cached pi_waiters;
struct task_struct *pi_top_task;
struct rt_mutex_waiter *pi_blocked_on;
struct mutex_waiter *blocked_on;
# 998 "./include/linux/sched.h"
void *journal_info;
struct bio_list *bio_list;
struct blk_plug *plug;
struct reclaim_state *reclaim_state;
struct backing_dev_info *backing_dev_info;
struct io_context *io_context;
unsigned long ptrace_message;
siginfo_t *last_siginfo;
struct task_io_accounting ioac;
unsigned int psi_flags;
u64 acct_rss_mem1;
u64 acct_vm_mem1;
u64 acct_timexpd;
nodemask_t mems_allowed;
seqcount_t mems_allowed_seq;
int cpuset_mem_spread_rotor;
int cpuset_slab_spread_rotor;
struct css_set *cgroups;
struct list_head cg_list;
struct robust_list_head *robust_list;
struct list_head pi_state_list;
struct futex_pi_state *pi_state_cache;
struct mutex futex_exit_mutex;
unsigned int futex_state;
struct perf_event_context *perf_event_ctxp[perf_nr_task_contexts];
struct mutex perf_event_mutex;
struct list_head perf_event_list;
# 1117 "./include/linux/sched.h"
struct tlbflush_unmap_batch tlb_ubc;
struct callback_head rcu;
struct pipe_inode_info *splice_pipe;
struct page_frag task_frag;
struct task_delay_info *delays;
# 1138 "./include/linux/sched.h"
int nr_dirtied;
int nr_dirtied_pause;
unsigned long dirty_paused_when;
# 1151 "./include/linux/sched.h"
u64 timer_slack_ns;
u64 default_timer_slack_ns;
int curr_ret_stack;
struct ftrace_ret_stack *ret_stack;
unsigned long long ftrace_timestamp;
atomic_t trace_overrun;
atomic_t tracing_graph_pause;
unsigned long trace;
unsigned long trace_recursion;
# 1209 "./include/linux/sched.h"
struct mem_cgroup *memcg_in_oom;
gfp_t memcg_oom_gfp_mask;
int memcg_oom_order;
unsigned int memcg_nr_pages_over_high;
struct uprobe_task *utask;
# 1227 "./include/linux/sched.h"
int pagefault_disabled;
struct task_struct *oom_reaper_list;
# 1243 "./include/linux/sched.h"
void *security;
# 1253 "./include/linux/sched.h"
struct thread_struct thread;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pid *task_pid(struct task_struct *task)
{
return task->pids[PIDTYPE_PID].pid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pid *task_tgid(struct task_struct *task)
{
return task->group_leader->pids[PIDTYPE_PID].pid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pid *task_pgrp(struct task_struct *task)
{
return task->group_leader->pids[PIDTYPE_PGID].pid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pid *task_session(struct task_struct *task)
{
return task->group_leader->pids[PIDTYPE_SID].pid;
}
# 1299 "./include/linux/sched.h"
pid_t __task_pid_nr_ns(struct task_struct *task, enum pid_type type, struct pid_namespace *ns);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_pid_nr(struct task_struct *tsk)
{
return tsk->pid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_pid_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)
{
return __task_pid_nr_ns(tsk, PIDTYPE_PID, ns);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_pid_vnr(struct task_struct *tsk)
{
return __task_pid_nr_ns(tsk, PIDTYPE_PID, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_tgid_nr(struct task_struct *tsk)
{
return tsk->tgid;
}
# 1332 "./include/linux/sched.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pid_alive(const struct task_struct *p)
{
return p->pids[PIDTYPE_PID].pid != ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_pgrp_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)
{
return __task_pid_nr_ns(tsk, PIDTYPE_PGID, ns);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_pgrp_vnr(struct task_struct *tsk)
{
return __task_pid_nr_ns(tsk, PIDTYPE_PGID, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_session_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)
{
return __task_pid_nr_ns(tsk, PIDTYPE_SID, ns);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_session_vnr(struct task_struct *tsk)
{
return __task_pid_nr_ns(tsk, PIDTYPE_SID, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_tgid_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)
{
return __task_pid_nr_ns(tsk, __PIDTYPE_TGID, ns);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_tgid_vnr(struct task_struct *tsk)
{
return __task_pid_nr_ns(tsk, __PIDTYPE_TGID, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_ppid_nr_ns(const struct task_struct *tsk, struct pid_namespace *ns)
{
pid_t pid = 0;
rcu_read_lock();
if (pid_alive(tsk))
pid = task_tgid_nr_ns(({ typeof(*(tsk->real_parent)) *________p1 = (typeof(*(tsk->real_parent)) *)({ union { typeof((tsk->real_parent)) __val; char __c[1]; } __u; if (1) __read_once_size(&((tsk->real_parent)), __u.__c, sizeof((tsk->real_parent))); else __read_once_size_nocheck(&((tsk->real_parent)), __u.__c, sizeof((tsk->real_parent))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(tsk->real_parent)) *)(________p1)); }), ns);
rcu_read_unlock();
return pid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_ppid_nr(const struct task_struct *tsk)
{
return task_ppid_nr_ns(tsk, &init_pid_ns);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pid_t task_pgrp_nr(struct task_struct *tsk)
{
return task_pgrp_nr_ns(tsk, &init_pid_ns);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int __get_task_state(struct task_struct *tsk)
{
unsigned int tsk_state = ({ union { typeof(tsk->state) __val; char __c[1]; } __u; if (1) __read_once_size(&(tsk->state), __u.__c, sizeof(tsk->state)); else __read_once_size_nocheck(&(tsk->state), __u.__c, sizeof(tsk->state)); do { } while (0); __u.__val; });
unsigned int state = (tsk_state | tsk->exit_state) & (0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040);
do { bool __cond = !(!(((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) == 0 || ((((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) - 1)) != 0))); extern void __compiletime_assert_33(void) ; if (__cond) __compiletime_assert_33(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
if (tsk_state == (0x0002 | 0x0400))
state = ((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1);
return fls(state);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) char __task_state_to_char(unsigned int state)
{
static const char state_char[] = "RSDTtXZPI";
do { bool __cond = !(!(1 + ( __builtin_constant_p((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) ? ( __builtin_constant_p((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) ? ( ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) < 2 ? 0 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 63) ? 63 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 62) ? 62 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 61) ? 61 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 60) ? 60 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 59) ? 59 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 58) ? 58 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 57) ? 57 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 56) ? 56 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 55) ? 55 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 54) ? 54 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 53) ? 53 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 52) ? 52 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 51) ? 51 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 50) ? 50 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 49) ? 49 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 48) ? 48 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 47) ? 47 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 46) ? 46 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 45) ? 45 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 44) ? 44 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 43) ? 43 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 42) ? 42 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 41) ? 41 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 40) ? 40 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 39) ? 39 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 38) ? 38 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 37) ? 37 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 36) ? 36 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 35) ? 35 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 34) ? 34 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 33) ? 33 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 32) ? 32 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 31) ? 31 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 30) ? 30 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 29) ? 29 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 28) ? 28 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 27) ? 27 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 26) ? 26 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 25) ? 25 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 24) ? 24 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 23) ? 23 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 22) ? 22 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 21) ? 21 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 20) ? 20 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 19) ? 19 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 18) ? 18 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 17) ? 17 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 16) ? 16 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 15) ? 15 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 14) ? 14 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 13) ? 13 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 12) ? 12 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 11) ? 11 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 10) ? 10 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 9) ? 9 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 8) ? 8 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 7) ? 7 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 6) ? 6 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 5) ? 5 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 4) ? 4 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 3) ? 3 : ((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) & (1ULL << 2) ? 2 : 1) : -1) : (sizeof((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) <= 4) ? __ilog2_u32((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) : __ilog2_u64((((0x0000 | 0x0001 | 0x0002 | 0x0004 | 0x0008 | 0x0010 | 0x0020 | 0x0040) + 1) << 1)) ) != sizeof(state_char) - 1)); extern void __compiletime_assert_34(void) ; if (__cond) __compiletime_assert_34(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return state_char[state];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) char task_state_to_char(struct task_struct *tsk)
{
return __task_state_to_char(__get_task_state(tsk));
}
# 1430 "./include/linux/sched.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_global_init(struct task_struct *tsk)
{
return task_tgid_nr(tsk) == 1;
}
extern struct pid *cad_pid;
# 1497 "./include/linux/sched.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_percpu_thread(void)
{
return ((current_thread_info()->task)->flags & 0x04000000) &&
((current_thread_info()->task)->nr_cpus_allowed == 1);
}
# 1529 "./include/linux/sched.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_no_new_privs(struct task_struct *p) { return test_bit(0, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_no_new_privs(struct task_struct *p) { _set_bit(0,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_spread_page(struct task_struct *p) { return test_bit(1, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_spread_page(struct task_struct *p) { _set_bit(1,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_clear_spread_page(struct task_struct *p) { _clear_bit(1,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_spread_slab(struct task_struct *p) { return test_bit(2, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_spread_slab(struct task_struct *p) { _set_bit(2,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_clear_spread_slab(struct task_struct *p) { _clear_bit(2,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_spec_ssb_disable(struct task_struct *p) { return test_bit(3, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_spec_ssb_disable(struct task_struct *p) { _set_bit(3,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_clear_spec_ssb_disable(struct task_struct *p) { _clear_bit(3,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_spec_ssb_force_disable(struct task_struct *p) { return test_bit(4, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_spec_ssb_force_disable(struct task_struct *p) { _set_bit(4,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_lmk_waiting(struct task_struct *p) { return test_bit(7, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_lmk_waiting(struct task_struct *p) { _set_bit(7,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_spec_ib_disable(struct task_struct *p) { return test_bit(5, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_spec_ib_disable(struct task_struct *p) { _set_bit(5,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_clear_spec_ib_disable(struct task_struct *p) { _clear_bit(5,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool task_spec_ib_force_disable(struct task_struct *p) { return test_bit(6, &p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void task_set_spec_ib_force_disable(struct task_struct *p) { _set_bit(6,&p->atomic_flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
current_restore_flags(unsigned long orig_flags, unsigned long flags)
{
(current_thread_info()->task)->flags &= ~flags;
(current_thread_info()->task)->flags |= orig_flags & flags;
}
extern int cpuset_cpumask_can_shrink(const struct cpumask *cur, const struct cpumask *trial);
extern int task_can_attach(struct task_struct *p, const struct cpumask *cs_cpus_allowed);
extern void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask);
extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask);
# 1585 "./include/linux/sched.h"
extern int yield_to(struct task_struct *p, bool preempt);
extern void set_user_nice(struct task_struct *p, long nice);
extern int task_prio(const struct task_struct *p);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int task_nice(const struct task_struct *p)
{
return (((p)->static_prio) - (100 + (19 - -20 + 1) / 2));
}
extern int can_nice(const struct task_struct *p, const int nice);
extern int task_curr(const struct task_struct *p);
extern int idle_cpu(int cpu);
extern int sched_setscheduler(struct task_struct *, int, const struct sched_param *);
extern int sched_setscheduler_nocheck(struct task_struct *, int, const struct sched_param *);
extern int sched_setattr(struct task_struct *, const struct sched_attr *);
extern struct task_struct *idle_task(int cpu);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_idle_task(const struct task_struct *p)
{
return !!(p->flags & 0x00000002);
}
extern struct task_struct *curr_task(int cpu);
extern void ia64_set_curr_task(int cpu, struct task_struct *p);
void yield(void);
union thread_union {
struct thread_info thread_info;
unsigned long stack[(((1UL) << 12) << 1)/sizeof(long)];
};
# 1651 "./include/linux/sched.h"
extern struct task_struct *find_task_by_vpid(pid_t nr);
extern struct task_struct *find_task_by_pid_ns(pid_t nr, struct pid_namespace *ns);
extern int wake_up_state(struct task_struct *tsk, unsigned int state);
extern int wake_up_process(struct task_struct *tsk);
extern void wake_up_new_task(struct task_struct *tsk);
extern void kick_process(struct task_struct *tsk);
extern void __set_task_comm(struct task_struct *tsk, const char *from, bool exec);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_task_comm(struct task_struct *tsk, const char *from)
{
__set_task_comm(tsk, from, false);
}
extern char *__get_task_comm(char *to, size_t len, struct task_struct *tsk);
void scheduler_ipi(void);
extern unsigned long wait_task_inactive(struct task_struct *, long match_state);
# 1692 "./include/linux/sched.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_tsk_thread_flag(struct task_struct *tsk, int flag)
{
set_ti_thread_flag(((struct thread_info *)(tsk)->stack), flag);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_tsk_thread_flag(struct task_struct *tsk, int flag)
{
clear_ti_thread_flag(((struct thread_info *)(tsk)->stack), flag);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void update_tsk_thread_flag(struct task_struct *tsk, int flag,
bool value)
{
update_ti_thread_flag(((struct thread_info *)(tsk)->stack), flag, value);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_and_set_tsk_thread_flag(struct task_struct *tsk, int flag)
{
return test_and_set_ti_thread_flag(((struct thread_info *)(tsk)->stack), flag);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_and_clear_tsk_thread_flag(struct task_struct *tsk, int flag)
{
return test_and_clear_ti_thread_flag(((struct thread_info *)(tsk)->stack), flag);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_tsk_thread_flag(struct task_struct *tsk, int flag)
{
return test_ti_thread_flag(((struct thread_info *)(tsk)->stack), flag);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_tsk_need_resched(struct task_struct *tsk)
{
set_tsk_thread_flag(tsk,1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_tsk_need_resched(struct task_struct *tsk)
{
clear_tsk_thread_flag(tsk,1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int test_tsk_need_resched(struct task_struct *tsk)
{
return __builtin_expect(!!(test_tsk_thread_flag(tsk,1)), 0);
}
# 1748 "./include/linux/sched.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int _cond_resched(void) { return 0; }
extern int __cond_resched_lock(spinlock_t *lock);
extern int __cond_resched_softirq(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void cond_resched_rcu(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int spin_needbreak(spinlock_t *lock)
{
return spin_is_contended(lock);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) bool need_resched(void)
{
return __builtin_expect(!!(test_ti_thread_flag(current_thread_info(), 1)), 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int task_cpu(const struct task_struct *p)
{
return ((struct thread_info *)(p)->stack)->cpu;
}
extern void set_task_cpu(struct task_struct *p, unsigned int cpu);
# 1839 "./include/linux/sched.h"
extern long sched_setaffinity(pid_t pid, const struct cpumask *new_mask);
extern long sched_getaffinity(pid_t pid, struct cpumask *mask);
# 6 "./include/linux/blkdev.h" 2
# 1 "./include/linux/sched/clock.h" 1
# 15 "./include/linux/sched/clock.h"
extern unsigned long long __attribute__((no_instrument_function)) sched_clock(void);
extern u64 running_clock(void);
extern u64 sched_clock_cpu(int cpu);
extern void sched_clock_init(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sched_clock_tick(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_sched_clock_stable(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sched_clock_idle_sleep_event(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sched_clock_idle_wakeup_event(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 cpu_clock(int cpu)
{
return sched_clock();
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 local_clock(void)
{
return sched_clock();
}
# 97 "./include/linux/sched/clock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void enable_sched_clock_irqtime(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void disable_sched_clock_irqtime(void) {}
# 7 "./include/linux/blkdev.h" 2
# 1 "./include/uapi/linux/major.h" 1
# 11 "./include/linux/blkdev.h" 2
# 1 "./include/linux/genhd.h" 1
# 14 "./include/linux/genhd.h"
# 1 "./include/linux/kdev_t.h" 1
# 1 "./include/uapi/linux/kdev_t.h" 1
# 6 "./include/linux/kdev_t.h" 2
# 24 "./include/linux/kdev_t.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool old_valid_dev(dev_t dev)
{
return ((unsigned int) ((dev) >> 20)) < 256 && ((unsigned int) ((dev) & ((1U << 20) - 1))) < 256;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u16 old_encode_dev(dev_t dev)
{
return (((unsigned int) ((dev) >> 20)) << 8) | ((unsigned int) ((dev) & ((1U << 20) - 1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) dev_t old_decode_dev(u16 val)
{
return ((((val >> 8) & 255) << 20) | (val & 255));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 new_encode_dev(dev_t dev)
{
unsigned major = ((unsigned int) ((dev) >> 20));
unsigned minor = ((unsigned int) ((dev) & ((1U << 20) - 1)));
return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) dev_t new_decode_dev(u32 dev)
{
unsigned major = (dev & 0xfff00) >> 8;
unsigned minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);
return (((major) << 20) | (minor));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u64 huge_encode_dev(dev_t dev)
{
return new_encode_dev(dev);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) dev_t huge_decode_dev(u64 dev)
{
return new_decode_dev(dev);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int sysv_valid_dev(dev_t dev)
{
return ((unsigned int) ((dev) >> 20)) < (1<<14) && ((unsigned int) ((dev) & ((1U << 20) - 1))) < (1<<18);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 sysv_encode_dev(dev_t dev)
{
return ((unsigned int) ((dev) & ((1U << 20) - 1))) | (((unsigned int) ((dev) >> 20)) << 18);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned sysv_major(u32 dev)
{
return (dev >> 18) & 0x3fff;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned sysv_minor(u32 dev)
{
return dev & 0x3ffff;
}
# 15 "./include/linux/genhd.h" 2
# 1 "./include/linux/slab.h" 1
# 15 "./include/linux/slab.h"
# 1 "./include/linux/gfp.h" 1
# 1 "./include/linux/mmzone.h" 1
# 18 "./include/linux/mmzone.h"
# 1 "./include/linux/pageblock-flags.h" 1
# 29 "./include/linux/pageblock-flags.h"
enum pageblock_bits {
PB_migrate,
PB_migrate_end = PB_migrate + 3 - 1,
PB_migrate_skip,
NR_PAGEBLOCK_BITS
};
# 66 "./include/linux/pageblock-flags.h"
struct page;
unsigned long get_pfnblock_flags_mask(struct page *page,
unsigned long pfn,
unsigned long end_bitidx,
unsigned long mask);
void set_pfnblock_flags_mask(struct page *page,
unsigned long flags,
unsigned long pfn,
unsigned long end_bitidx,
unsigned long mask);
# 19 "./include/linux/mmzone.h" 2
# 1 "./include/linux/page-flags-layout.h" 1
# 1 "./include/generated/bounds.h" 1
# 7 "./include/linux/page-flags-layout.h" 2
# 20 "./include/linux/mmzone.h" 2
# 39 "./include/linux/mmzone.h"
enum migratetype {
MIGRATE_UNMOVABLE,
MIGRATE_MOVABLE,
MIGRATE_RECLAIMABLE,
MIGRATE_PCPTYPES,
MIGRATE_HIGHATOMIC = MIGRATE_PCPTYPES,
# 59 "./include/linux/mmzone.h"
MIGRATE_CMA,
MIGRATE_ISOLATE,
MIGRATE_TYPES
};
extern char * const migratetype_names[MIGRATE_TYPES];
# 78 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_migrate_movable(int mt)
{
return __builtin_expect(!!((mt) == MIGRATE_CMA), 0) || mt == MIGRATE_MOVABLE;
}
extern int page_group_by_mobility_disabled;
# 96 "./include/linux/mmzone.h"
struct free_area {
struct list_head free_list[MIGRATE_TYPES];
unsigned long nr_free;
};
struct pglist_data;
# 110 "./include/linux/mmzone.h"
struct zone_padding {
char x[0];
} __attribute__((__aligned__(1 << (6))));
# 132 "./include/linux/mmzone.h"
enum zone_stat_item {
NR_FREE_PAGES,
NR_ZONE_LRU_BASE,
NR_ZONE_INACTIVE_ANON = NR_ZONE_LRU_BASE,
NR_ZONE_ACTIVE_ANON,
NR_ZONE_INACTIVE_FILE,
NR_ZONE_ACTIVE_FILE,
NR_ZONE_UNEVICTABLE,
NR_ZONE_WRITE_PENDING,
NR_MLOCK,
NR_PAGETABLE,
NR_KERNEL_STACK_KB,
NR_BOUNCE,
NR_ZSPAGES,
NR_FREE_CMA_PAGES,
NR_VM_ZONE_STAT_ITEMS };
enum node_stat_item {
NR_LRU_BASE,
NR_INACTIVE_ANON = NR_LRU_BASE,
NR_ACTIVE_ANON,
NR_INACTIVE_FILE,
NR_ACTIVE_FILE,
NR_UNEVICTABLE,
NR_SLAB_RECLAIMABLE,
NR_SLAB_UNRECLAIMABLE,
NR_ISOLATED_ANON,
NR_ISOLATED_FILE,
WORKINGSET_REFAULT,
WORKINGSET_ACTIVATE,
WORKINGSET_RESTORE,
WORKINGSET_NODERECLAIM,
NR_ANON_MAPPED,
NR_FILE_MAPPED,
NR_FILE_PAGES,
NR_FILE_DIRTY,
NR_WRITEBACK,
NR_WRITEBACK_TEMP,
NR_SHMEM,
NR_SHMEM_THPS,
NR_SHMEM_PMDMAPPED,
NR_ANON_THPS,
NR_UNSTABLE_NFS,
NR_VMSCAN_WRITE,
NR_VMSCAN_IMMEDIATE,
NR_DIRTIED,
NR_WRITTEN,
NR_KERNEL_MISC_RECLAIMABLE,
NR_VM_NODE_STAT_ITEMS
};
# 204 "./include/linux/mmzone.h"
enum lru_list {
LRU_INACTIVE_ANON = 0,
LRU_ACTIVE_ANON = 0 + 1,
LRU_INACTIVE_FILE = 0 + 2,
LRU_ACTIVE_FILE = 0 + 2 + 1,
LRU_UNEVICTABLE,
NR_LRU_LISTS
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_file_lru(enum lru_list lru)
{
return (lru == LRU_INACTIVE_FILE || lru == LRU_ACTIVE_FILE);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_active_lru(enum lru_list lru)
{
return (lru == LRU_ACTIVE_ANON || lru == LRU_ACTIVE_FILE);
}
struct zone_reclaim_stat {
# 236 "./include/linux/mmzone.h"
unsigned long recent_rotated[2];
unsigned long recent_scanned[2];
};
struct lruvec {
struct list_head lists[NR_LRU_LISTS];
struct zone_reclaim_stat reclaim_stat;
atomic_long_t inactive_age;
unsigned long refaults;
struct pglist_data *pgdat;
};
# 265 "./include/linux/mmzone.h"
typedef unsigned isolate_mode_t;
enum zone_watermarks {
WMARK_MIN,
WMARK_LOW,
WMARK_HIGH,
NR_WMARK
};
struct per_cpu_pages {
int count;
int high;
int batch;
struct list_head lists[MIGRATE_TYPES];
};
struct per_cpu_pageset {
struct per_cpu_pages pcp;
s8 stat_threshold;
s8 vm_stat_diff[NR_VM_ZONE_STAT_ITEMS];
};
struct per_cpu_nodestat {
s8 stat_threshold;
s8 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS];
};
enum zone_type {
# 345 "./include/linux/mmzone.h"
ZONE_NORMAL,
# 355 "./include/linux/mmzone.h"
ZONE_HIGHMEM,
# 366 "./include/linux/mmzone.h"
ZONE_MOVABLE,
__MAX_NR_ZONES
};
struct zone {
unsigned long watermark[NR_WMARK];
unsigned long nr_reserved_highatomic;
# 394 "./include/linux/mmzone.h"
long lowmem_reserve[3];
struct pglist_data *zone_pgdat;
struct per_cpu_pageset *pageset;
unsigned long *pageblock_flags;
unsigned long zone_start_pfn;
# 454 "./include/linux/mmzone.h"
unsigned long managed_pages;
unsigned long spanned_pages;
unsigned long present_pages;
const char *name;
unsigned long nr_isolate_pageblock;
int initialized;
struct zone_padding _pad1_;
struct free_area free_area[11];
unsigned long flags;
spinlock_t lock;
struct zone_padding _pad2_;
unsigned long percpu_drift_mark;
unsigned long compact_cached_free_pfn;
unsigned long compact_cached_migrate_pfn[2];
# 511 "./include/linux/mmzone.h"
unsigned int compact_considered;
unsigned int compact_defer_shift;
int compact_order_failed;
bool compact_blockskip_flush;
bool contiguous;
struct zone_padding _pad3_;
atomic_long_t vm_stat[NR_VM_ZONE_STAT_ITEMS];
atomic_long_t vm_numa_stat[0];
} __attribute__((__aligned__(1 << (6))));
enum pgdat_flags {
PGDAT_CONGESTED,
PGDAT_DIRTY,
PGDAT_WRITEBACK,
PGDAT_RECLAIM_LOCKED,
PGDAT_ALLOC_LOCKED,
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long zone_end_pfn(const struct zone *zone)
{
return zone->zone_start_pfn + zone->spanned_pages;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool zone_spans_pfn(const struct zone *zone, unsigned long pfn)
{
return zone->zone_start_pfn <= pfn && pfn < zone_end_pfn(zone);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool zone_is_initialized(struct zone *zone)
{
return zone->initialized;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool zone_is_empty(struct zone *zone)
{
return zone->spanned_pages == 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool zone_intersects(struct zone *zone,
unsigned long start_pfn, unsigned long nr_pages)
{
if (zone_is_empty(zone))
return false;
if (start_pfn >= zone_end_pfn(zone) ||
start_pfn + nr_pages <= zone->zone_start_pfn)
return false;
return true;
}
# 590 "./include/linux/mmzone.h"
enum {
ZONELIST_FALLBACK,
MAX_ZONELISTS
};
struct zoneref {
struct zone *zone;
int zone_idx;
};
# 625 "./include/linux/mmzone.h"
struct zonelist {
struct zoneref _zonerefs[((1 << 0) * 3) + 1];
};
extern struct page *mem_map;
# 642 "./include/linux/mmzone.h"
struct bootmem_data;
typedef struct pglist_data {
struct zone node_zones[3];
struct zonelist node_zonelists[MAX_ZONELISTS];
int nr_zones;
struct page *node_mem_map;
# 669 "./include/linux/mmzone.h"
unsigned long node_start_pfn;
unsigned long node_present_pages;
unsigned long node_spanned_pages;
int node_id;
wait_queue_head_t kswapd_wait;
wait_queue_head_t pfmemalloc_wait;
struct task_struct *kswapd;
int kswapd_order;
enum zone_type kswapd_classzone_idx;
int kswapd_failures;
int kcompactd_max_order;
enum zone_type kcompactd_classzone_idx;
wait_queue_head_t kcompactd_wait;
struct task_struct *kcompactd;
# 703 "./include/linux/mmzone.h"
unsigned long totalreserve_pages;
# 714 "./include/linux/mmzone.h"
struct zone_padding _pad1_;
spinlock_t lru_lock;
# 734 "./include/linux/mmzone.h"
struct lruvec lruvec;
unsigned int inactive_ratio;
unsigned long flags;
struct zone_padding _pad2_;
struct per_cpu_nodestat *per_cpu_nodestats;
atomic_long_t vm_stat[NR_VM_NODE_STAT_ITEMS];
} pg_data_t;
# 762 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *zone_lru_lock(struct zone *zone)
{
return &zone->zone_pgdat->lru_lock;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct lruvec *node_lruvec(struct pglist_data *pgdat)
{
return &pgdat->lruvec;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long pgdat_end_pfn(pg_data_t *pgdat)
{
return pgdat->node_start_pfn + pgdat->node_spanned_pages;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool pgdat_is_empty(pg_data_t *pgdat)
{
return !pgdat->node_start_pfn && !pgdat->node_spanned_pages;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int zone_id(const struct zone *zone)
{
struct pglist_data *pgdat = zone->zone_pgdat;
return zone - pgdat->node_zones;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_dev_zone(const struct zone *zone)
{
return false;
}
# 1 "./include/linux/memory_hotplug.h" 1
# 1 "./include/linux/mmzone.h" 1
# 6 "./include/linux/memory_hotplug.h" 2
# 1 "./include/linux/notifier.h" 1
# 15 "./include/linux/notifier.h"
# 1 "./include/linux/rwsem.h" 1
# 23 "./include/linux/rwsem.h"
struct rw_semaphore;
struct rw_semaphore {
atomic_long_t count;
struct list_head wait_list;
raw_spinlock_t wait_lock;
struct optimistic_spin_queue osq;
struct task_struct *owner;
};
extern struct rw_semaphore *rwsem_down_read_failed(struct rw_semaphore *sem);
extern struct rw_semaphore *rwsem_down_read_failed_killable(struct rw_semaphore *sem);
extern struct rw_semaphore *rwsem_down_write_failed(struct rw_semaphore *sem);
extern struct rw_semaphore *rwsem_down_write_failed_killable(struct rw_semaphore *sem);
extern struct rw_semaphore *rwsem_wake(struct rw_semaphore *);
extern struct rw_semaphore *rwsem_downgrade_wake(struct rw_semaphore *sem);
# 1 "./arch/arm/include/generated/asm/rwsem.h" 1
# 1 "./include/asm-generic/rwsem.h" 1
# 35 "./include/asm-generic/rwsem.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __down_read(struct rw_semaphore *sem)
{
if (__builtin_expect(!!(atomic_long_inc_return_acquire(&sem->count) <= 0), 0))
rwsem_down_read_failed(sem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __down_read_trylock(struct rw_semaphore *sem)
{
long tmp;
while ((tmp = atomic_long_read(&sem->count)) >= 0) {
if (tmp == (({ typeof(atomic_cmpxchg_relaxed((atomic_t *)(&sem->count), (tmp), (tmp + 0x00000001L))) __ret = atomic_cmpxchg_relaxed((atomic_t *)(&sem->count), (tmp), (tmp + 0x00000001L)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }))) {
return 1;
}
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __down_write(struct rw_semaphore *sem)
{
long tmp;
tmp = atomic_long_add_return_acquire(((-0x0000ffffL -1) + 0x00000001L),
&sem->count);
if (__builtin_expect(!!(tmp != ((-0x0000ffffL -1) + 0x00000001L)), 0))
rwsem_down_write_failed(sem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __down_write_killable(struct rw_semaphore *sem)
{
long tmp;
tmp = atomic_long_add_return_acquire(((-0x0000ffffL -1) + 0x00000001L),
&sem->count);
if (__builtin_expect(!!(tmp != ((-0x0000ffffL -1) + 0x00000001L)), 0))
if (IS_ERR(rwsem_down_write_failed_killable(sem)))
return -4;
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __down_write_trylock(struct rw_semaphore *sem)
{
long tmp;
tmp = (({ typeof(atomic_cmpxchg_relaxed((atomic_t *)(&sem->count), (0x00000000L), (((-0x0000ffffL -1) + 0x00000001L)))) __ret = atomic_cmpxchg_relaxed((atomic_t *)(&sem->count), (0x00000000L), (((-0x0000ffffL -1) + 0x00000001L))); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }));
return tmp == 0x00000000L;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __up_read(struct rw_semaphore *sem)
{
long tmp;
tmp = atomic_long_dec_return_release(&sem->count);
if (__builtin_expect(!!(tmp < -1 && (tmp & 0x0000ffffL) == 0), 0))
rwsem_wake(sem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __up_write(struct rw_semaphore *sem)
{
if (__builtin_expect(!!(atomic_long_sub_return_release(((-0x0000ffffL -1) + 0x00000001L), &sem->count) < 0), 0))
rwsem_wake(sem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __downgrade_write(struct rw_semaphore *sem)
{
long tmp;
# 124 "./include/asm-generic/rwsem.h"
tmp = atomic_long_add_return_release(-(-0x0000ffffL -1), &sem->count);
if (tmp < 0)
rwsem_downgrade_wake(sem);
}
# 2 "./arch/arm/include/generated/asm/rwsem.h" 2
# 62 "./include/linux/rwsem.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rwsem_is_locked(struct rw_semaphore *sem)
{
return atomic_long_read(&sem->count) != 0;
}
# 96 "./include/linux/rwsem.h"
extern void __init_rwsem(struct rw_semaphore *sem, const char *name,
struct lock_class_key *key);
# 112 "./include/linux/rwsem.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int rwsem_is_contended(struct rw_semaphore *sem)
{
return !list_empty(&sem->wait_list);
}
extern void down_read(struct rw_semaphore *sem);
extern int down_read_trylock(struct rw_semaphore *sem);
extern void down_write(struct rw_semaphore *sem);
extern int __attribute__((warn_unused_result)) down_write_killable(struct rw_semaphore *sem);
extern int down_write_trylock(struct rw_semaphore *sem);
extern void up_read(struct rw_semaphore *sem);
extern void up_write(struct rw_semaphore *sem);
extern void downgrade_write(struct rw_semaphore *sem);
# 16 "./include/linux/notifier.h" 2
# 1 "./include/linux/srcu.h" 1
# 35 "./include/linux/srcu.h"
# 1 "./include/linux/rcu_segcblist.h" 1
# 31 "./include/linux/rcu_segcblist.h"
struct rcu_cblist {
struct callback_head *head;
struct callback_head **tail;
long len;
long len_lazy;
};
# 77 "./include/linux/rcu_segcblist.h"
struct rcu_segcblist {
struct callback_head *head;
struct callback_head **tails[4];
unsigned long gp_seq[4];
long len;
long len_lazy;
};
# 36 "./include/linux/srcu.h" 2
struct srcu_struct;
# 54 "./include/linux/srcu.h"
int init_srcu_struct(struct srcu_struct *sp);
# 1 "./include/linux/srcutree.h" 1
# 27 "./include/linux/srcutree.h"
# 1 "./include/linux/rcu_node_tree.h" 1
# 28 "./include/linux/srcutree.h" 2
# 1 "./include/linux/completion.h" 1
# 29 "./include/linux/completion.h"
struct completion {
unsigned int done;
wait_queue_head_t wait;
};
# 63 "./include/linux/completion.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void complete_acquire(struct completion *x) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void complete_release(struct completion *x) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void complete_release_commit(struct completion *x) {}
# 117 "./include/linux/completion.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __init_completion(struct completion *x)
{
x->done = 0;
do { static struct lock_class_key __key; __init_waitqueue_head((&x->wait), "&x->wait", &__key); } while (0);
}
# 130 "./include/linux/completion.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void reinit_completion(struct completion *x)
{
x->done = 0;
}
extern void wait_for_completion(struct completion *);
extern void wait_for_completion_io(struct completion *);
extern int wait_for_completion_interruptible(struct completion *x);
extern int wait_for_completion_killable(struct completion *x);
extern unsigned long wait_for_completion_timeout(struct completion *x,
unsigned long timeout);
extern unsigned long wait_for_completion_io_timeout(struct completion *x,
unsigned long timeout);
extern long wait_for_completion_interruptible_timeout(
struct completion *x, unsigned long timeout);
extern long wait_for_completion_killable_timeout(
struct completion *x, unsigned long timeout);
extern bool try_wait_for_completion(struct completion *x);
extern bool completion_done(struct completion *x);
extern void complete(struct completion *);
extern void complete_all(struct completion *);
# 29 "./include/linux/srcutree.h" 2
struct srcu_node;
struct srcu_struct;
struct srcu_data {
unsigned long srcu_lock_count[2];
unsigned long srcu_unlock_count[2];
raw_spinlock_t lock __attribute__((__aligned__(1 << (6))));
struct rcu_segcblist srcu_cblist;
unsigned long srcu_gp_seq_needed;
unsigned long srcu_gp_seq_needed_exp;
bool srcu_cblist_invoking;
struct delayed_work work;
struct callback_head srcu_barrier_head;
struct srcu_node *mynode;
unsigned long grpmask;
int cpu;
struct srcu_struct *sp;
};
struct srcu_node {
raw_spinlock_t lock;
unsigned long srcu_have_cbs[4];
unsigned long srcu_data_have_cbs[4];
unsigned long srcu_gp_seq_needed_exp;
struct srcu_node *srcu_parent;
int grplo;
int grphi;
};
struct srcu_struct {
struct srcu_node node[1];
struct srcu_node *level[1 + 1];
struct mutex srcu_cb_mutex;
raw_spinlock_t lock;
struct mutex srcu_gp_mutex;
unsigned int srcu_idx;
unsigned long srcu_gp_seq;
unsigned long srcu_gp_seq_needed;
unsigned long srcu_gp_seq_needed_exp;
unsigned long srcu_last_gp_end;
struct srcu_data *sda;
unsigned long srcu_barrier_seq;
struct mutex srcu_barrier_mutex;
struct completion srcu_barrier_completion;
atomic_t srcu_barrier_cpu_cnt;
struct delayed_work work;
};
# 140 "./include/linux/srcutree.h"
void synchronize_srcu_expedited(struct srcu_struct *sp);
void srcu_barrier(struct srcu_struct *sp);
void srcu_torture_stats_print(struct srcu_struct *sp, char *tt, char *tf);
# 63 "./include/linux/srcu.h" 2
void call_srcu(struct srcu_struct *sp, struct callback_head *head,
void (*func)(struct callback_head *head));
void cleanup_srcu_struct(struct srcu_struct *sp);
int __srcu_read_lock(struct srcu_struct *sp) ;
void __srcu_read_unlock(struct srcu_struct *sp, int idx) ;
void synchronize_srcu(struct srcu_struct *sp);
# 104 "./include/linux/srcu.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int srcu_read_lock_held(struct srcu_struct *sp)
{
return 1;
}
# 155 "./include/linux/srcu.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int srcu_read_lock(struct srcu_struct *sp)
{
int retval;
retval = __srcu_read_lock(sp);
do { } while (0);
return retval;
}
# 171 "./include/linux/srcu.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void srcu_read_unlock(struct srcu_struct *sp, int idx)
{
do { } while (0);
__srcu_read_unlock(sp, idx);
}
# 187 "./include/linux/srcu.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void smp_mb__after_srcu_read_unlock(void)
{
}
# 17 "./include/linux/notifier.h" 2
# 51 "./include/linux/notifier.h"
struct notifier_block;
typedef int (*notifier_fn_t)(struct notifier_block *nb,
unsigned long action, void *data);
struct notifier_block {
notifier_fn_t notifier_call;
struct notifier_block *next;
int priority;
};
struct atomic_notifier_head {
spinlock_t lock;
struct notifier_block *head;
};
struct blocking_notifier_head {
struct rw_semaphore rwsem;
struct notifier_block *head;
};
struct raw_notifier_head {
struct notifier_block *head;
};
struct srcu_notifier_head {
struct mutex mutex;
struct srcu_struct srcu;
struct notifier_block *head;
};
# 95 "./include/linux/notifier.h"
extern void srcu_init_notifier_head(struct srcu_notifier_head *nh);
# 121 "./include/linux/notifier.h"
extern int atomic_notifier_chain_register(struct atomic_notifier_head *nh,
struct notifier_block *nb);
extern int blocking_notifier_chain_register(struct blocking_notifier_head *nh,
struct notifier_block *nb);
extern int raw_notifier_chain_register(struct raw_notifier_head *nh,
struct notifier_block *nb);
extern int srcu_notifier_chain_register(struct srcu_notifier_head *nh,
struct notifier_block *nb);
extern int blocking_notifier_chain_cond_register(
struct blocking_notifier_head *nh,
struct notifier_block *nb);
extern int atomic_notifier_chain_unregister(struct atomic_notifier_head *nh,
struct notifier_block *nb);
extern int blocking_notifier_chain_unregister(struct blocking_notifier_head *nh,
struct notifier_block *nb);
extern int raw_notifier_chain_unregister(struct raw_notifier_head *nh,
struct notifier_block *nb);
extern int srcu_notifier_chain_unregister(struct srcu_notifier_head *nh,
struct notifier_block *nb);
extern int atomic_notifier_call_chain(struct atomic_notifier_head *nh,
unsigned long val, void *v);
extern int __atomic_notifier_call_chain(struct atomic_notifier_head *nh,
unsigned long val, void *v, int nr_to_call, int *nr_calls);
extern int blocking_notifier_call_chain(struct blocking_notifier_head *nh,
unsigned long val, void *v);
extern int __blocking_notifier_call_chain(struct blocking_notifier_head *nh,
unsigned long val, void *v, int nr_to_call, int *nr_calls);
extern int raw_notifier_call_chain(struct raw_notifier_head *nh,
unsigned long val, void *v);
extern int __raw_notifier_call_chain(struct raw_notifier_head *nh,
unsigned long val, void *v, int nr_to_call, int *nr_calls);
extern int srcu_notifier_call_chain(struct srcu_notifier_head *nh,
unsigned long val, void *v);
extern int __srcu_notifier_call_chain(struct srcu_notifier_head *nh,
unsigned long val, void *v, int nr_to_call, int *nr_calls);
# 171 "./include/linux/notifier.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int notifier_from_errno(int err)
{
if (err)
return 0x8000 | (0x0001 - err);
return 0x0001;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int notifier_to_errno(int ret)
{
ret &= ~0x8000;
return ret > 0x0001 ? 0x0001 - ret : 0;
}
# 215 "./include/linux/notifier.h"
extern struct blocking_notifier_head reboot_notifier_list;
# 8 "./include/linux/memory_hotplug.h" 2
struct page;
struct zone;
struct pglist_data;
struct mem_section;
struct memory_block;
struct resource;
# 249 "./include/linux/memory_hotplug.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgdat_resize_lock(struct pglist_data *p, unsigned long *f) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgdat_resize_unlock(struct pglist_data *p, unsigned long *f) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgdat_resize_init(struct pglist_data *pgdat) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned zone_span_seqbegin(struct zone *zone)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int zone_span_seqretry(struct zone *zone, unsigned iv)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void zone_span_writelock(struct zone *zone) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void zone_span_writeunlock(struct zone *zone) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void zone_seqlock_init(struct zone *zone) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mhp_notimplemented(const char *func)
{
printk("\001" "4" "%s() called, with CONFIG_MEMORY_HOTPLUG disabled\n", func);
dump_stack();
return -38;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void register_page_bootmem_info_node(struct pglist_data *pgdat)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int try_online_node(int nid)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void get_online_mems(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void put_online_mems(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mem_hotplug_begin(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mem_hotplug_done(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool movable_node_is_enabled(void)
{
return false;
}
# 301 "./include/linux/memory_hotplug.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_mem_section_removable(unsigned long pfn,
unsigned long nr_pages)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void try_offline_node(int nid) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int offline_pages(unsigned long start_pfn, unsigned long nr_pages)
{
return -22;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void remove_memory(int nid, u64 start, u64 size) {}
extern int walk_memory_range(unsigned long start_pfn, unsigned long end_pfn,
void *arg, int (*func)(struct memory_block *, void *));
extern int __add_memory(int nid, u64 start, u64 size);
extern int add_memory(int nid, u64 start, u64 size);
extern int add_memory_resource(int nid, struct resource *resource, bool online);
extern int arch_add_memory(int nid, u64 start, u64 size, bool want_memblock);
extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn,
unsigned long nr_pages);
extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
extern bool is_memblock_offlined(struct memory_block *mem);
extern void remove_memory(int nid, u64 start, u64 size);
extern int sparse_add_one_section(struct pglist_data *pgdat, unsigned long start_pfn);
extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms,
unsigned long map_offset);
extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
unsigned long pnum);
extern bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages,
int online_type);
extern struct zone *zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
unsigned long nr_pages);
# 802 "./include/linux/mmzone.h" 2
void build_all_zonelists(pg_data_t *pgdat);
void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx);
bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
int classzone_idx, unsigned int alloc_flags,
long free_pages);
bool zone_watermark_ok(struct zone *z, unsigned int order,
unsigned long mark, int classzone_idx,
unsigned int alloc_flags);
bool zone_watermark_ok_safe(struct zone *z, unsigned int order,
unsigned long mark, int classzone_idx);
enum memmap_context {
MEMMAP_EARLY,
MEMMAP_HOTPLUG,
};
extern void init_currently_empty_zone(struct zone *zone, unsigned long start_pfn,
unsigned long size);
extern void lruvec_init(struct lruvec *lruvec);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pglist_data *lruvec_pgdat(struct lruvec *lruvec)
{
return lruvec->pgdat;
}
extern unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone_idx);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void memory_present(int nid, unsigned long start, unsigned long end) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int local_memory_node(int node_id) { return node_id; };
# 860 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool managed_zone(struct zone *zone)
{
return zone->managed_pages;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool populated_zone(struct zone *zone)
{
return zone->present_pages;
}
extern int movable_zone;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int zone_movable_is_highmem(void)
{
return (ZONE_MOVABLE - 1) == ZONE_HIGHMEM;
}
# 914 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_highmem_idx(enum zone_type idx)
{
return (idx == ZONE_HIGHMEM ||
(idx == ZONE_MOVABLE && zone_movable_is_highmem())
);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_movable(struct zone *zone)
{
return zone == zone->zone_pgdat->node_zones + ZONE_MOVABLE;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int zone_is_alloc_locked(const struct zone *zone)
{
return test_bit(PGDAT_ALLOC_LOCKED, &zone->zone_pgdat->flags);
}
# 946 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_highmem(struct zone *zone)
{
return is_highmem_idx(((zone) - (zone)->zone_pgdat->node_zones));
}
struct ctl_table;
int min_free_kbytes_sysctl_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
int watermark_scale_factor_sysctl_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int sysctl_lowmem_reserve_ratio[3 -1];
int lowmem_reserve_ratio_sysctl_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
int percpu_pagelist_fraction_sysctl_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
int sysctl_min_unmapped_ratio_sysctl_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
int sysctl_min_slab_ratio_sysctl_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int numa_zonelist_order_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern char numa_zonelist_order[];
extern struct pglist_data contig_page_data;
# 988 "./include/linux/mmzone.h"
extern struct pglist_data *first_online_pgdat(void);
extern struct pglist_data *next_online_pgdat(struct pglist_data *pgdat);
extern struct zone *next_zone(struct zone *zone);
# 1020 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct zone *zonelist_zone(struct zoneref *zoneref)
{
return zoneref->zone;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int zonelist_zone_idx(struct zoneref *zoneref)
{
return zoneref->zone_idx;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int zonelist_node_idx(struct zoneref *zoneref)
{
return 0;
}
struct zoneref *__next_zones_zonelist(struct zoneref *z,
enum zone_type highest_zoneidx,
nodemask_t *nodes);
# 1056 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) struct zoneref *next_zones_zonelist(struct zoneref *z,
enum zone_type highest_zoneidx,
nodemask_t *nodes)
{
if (__builtin_expect(!!(!nodes && zonelist_zone_idx(z) <= highest_zoneidx), 1))
return z;
return __next_zones_zonelist(z, highest_zoneidx, nodes);
}
# 1081 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct zoneref *first_zones_zonelist(struct zonelist *zonelist,
enum zone_type highest_zoneidx,
nodemask_t *nodes)
{
return next_zones_zonelist(zonelist->_zonerefs,
highest_zoneidx, nodes);
}
# 1131 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long early_pfn_to_nid(unsigned long pfn)
{
do { bool __cond = !(!(0)); extern void __compiletime_assert_35(void) ; if (__cond) __compiletime_assert_35(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return 0;
}
# 1345 "./include/linux/mmzone.h"
struct mminit_pfnnid_cache {
unsigned long last_start;
unsigned long last_end;
int last_nid;
};
void memory_present(int nid, unsigned long start, unsigned long end);
unsigned long __attribute__ ((__section__(".init.text"))) node_memmap_size_bytes(int, unsigned long, unsigned long);
# 1394 "./include/linux/mmzone.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool memmap_valid_within(unsigned long pfn,
struct page *page, struct zone *zone)
{
return true;
}
# 7 "./include/linux/gfp.h" 2
# 1 "./include/linux/topology.h" 1
# 35 "./include/linux/topology.h"
# 1 "./arch/arm/include/asm/topology.h" 1
struct cputopo_arm {
int thread_id;
int core_id;
int socket_id;
cpumask_t thread_sibling;
cpumask_t core_sibling;
};
extern struct cputopo_arm cpu_topology[4];
void init_cpu_topology(void);
void store_cpu_topology(unsigned int cpuid);
const struct cpumask *cpu_coregroup_mask(int cpu);
# 1 "./include/linux/arch_topology.h" 1
# 11 "./include/linux/arch_topology.h"
void topology_normalize_cpu_scale(void);
int topology_detect_flags(void);
int topology_smt_flags(void);
int topology_core_flags(void);
int topology_cpu_flags(void);
int topology_update_cpu_topology(void);
struct device_node;
bool topology_parse_cpu_capacity(struct device_node *cpu_node, int cpu);
extern __attribute__((section(".data..percpu" ""))) __typeof__(unsigned long) cpu_scale;
struct sched_domain;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
unsigned long topology_get_cpu_scale(struct sched_domain *sd, int cpu)
{
return (*({ do { const void *__vpp_verify = (typeof((&(cpu_scale)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*((&(cpu_scale)))) *)((&(cpu_scale))))); (typeof((typeof(*((&(cpu_scale)))) *)((&(cpu_scale))))) (__ptr + (((__per_cpu_offset[(cpu)])))); }); }));
}
void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity);
extern __attribute__((section(".data..percpu" ""))) __typeof__(unsigned long) freq_scale;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
unsigned long topology_get_freq_scale(struct sched_domain *sd, int cpu)
{
return (*({ do { const void *__vpp_verify = (typeof((&(freq_scale)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*((&(freq_scale)))) *)((&(freq_scale))))); (typeof((typeof(*((&(freq_scale)))) *)((&(freq_scale))))) (__ptr + (((__per_cpu_offset[(cpu)])))); }); }));
}
extern __attribute__((section(".data..percpu" ""))) __typeof__(unsigned long) max_freq_scale;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
unsigned long topology_get_max_freq_scale(struct sched_domain *sd, int cpu)
{
return (*({ do { const void *__vpp_verify = (typeof((&(max_freq_scale)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*((&(max_freq_scale)))) *)((&(max_freq_scale))))); (typeof((typeof(*((&(max_freq_scale)))) *)((&(max_freq_scale))))) (__ptr + (((__per_cpu_offset[(cpu)])))); }); }));
}
# 29 "./arch/arm/include/asm/topology.h" 2
# 49 "./arch/arm/include/asm/topology.h"
# 1 "./include/asm-generic/topology.h" 1
# 50 "./arch/arm/include/asm/topology.h" 2
# 36 "./include/linux/topology.h" 2
# 45 "./include/linux/topology.h"
int topology_update_cpu_topology(void);
# 102 "./include/linux/topology.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int numa_node_id(void)
{
return ((void)((current_thread_info()->cpu)),0);
}
# 162 "./include/linux/topology.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int numa_mem_id(void)
{
return numa_node_id();
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int node_to_mem_node(int node)
{
return node;
}
# 198 "./include/linux/topology.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const struct cpumask *cpu_smt_mask(int cpu)
{
return (&cpu_topology[cpu].thread_sibling);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const struct cpumask *cpu_cpu_mask(int cpu)
{
return ((void)(((void)(cpu),0)), ((const struct cpumask *)&__cpu_online_mask));
}
# 10 "./include/linux/gfp.h" 2
struct vm_area_struct;
# 309 "./include/linux/gfp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int gfpflags_to_migratetype(const gfp_t gfp_flags)
{
((void)(sizeof(( long)((gfp_flags & ((( gfp_t)0x10u)|(( gfp_t)0x08u))) == ((( gfp_t)0x10u)|(( gfp_t)0x08u))))));
do { bool __cond = !(!((1UL << 3) != 0x08u)); extern void __compiletime_assert_36(void) ; if (__cond) __compiletime_assert_36(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
do { bool __cond = !(!((0x08u >> 3) != MIGRATE_MOVABLE)); extern void __compiletime_assert_37(void) ; if (__cond) __compiletime_assert_37(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
if (__builtin_expect(!!(page_group_by_mobility_disabled), 0))
return MIGRATE_UNMOVABLE;
return (((gfp_flags & (( gfp_t)0x08u)) != 0) << 2) |
((gfp_flags & (( gfp_t)0x10u)) >> 3);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gfpflags_allow_blocking(const gfp_t gfp_flags)
{
return !!(gfp_flags & (( gfp_t)0x400000u));
}
# 353 "./include/linux/gfp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool gfpflags_normal_context(const gfp_t gfp_flags)
{
return (gfp_flags & ((( gfp_t)0x400000u) | (( gfp_t)0x2000u))) ==
(( gfp_t)0x400000u);
}
# 449 "./include/linux/gfp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) enum zone_type gfp_zone(gfp_t flags)
{
enum zone_type z;
int bit = ( int) (flags & ((( gfp_t)0x01u)|(( gfp_t)0x02u)|(( gfp_t)0x04u)|(( gfp_t)0x08u)));
z = (( (ZONE_NORMAL << 0 * 2) | (ZONE_NORMAL << 0x01u * 2) | (ZONE_HIGHMEM << 0x02u * 2) | (ZONE_NORMAL << 0x04u * 2) | (ZONE_NORMAL << 0x08u * 2) | (ZONE_NORMAL << (0x08u | 0x01u) * 2) | (ZONE_MOVABLE << (0x08u | 0x02u) * 2) | (ZONE_NORMAL << (0x08u | 0x04u) * 2)) >> (bit * 2)) &
((1 << 2) - 1);
((void)(sizeof(( long)((( 1 << (0x01u | 0x02u) | 1 << (0x01u | 0x04u) | 1 << (0x04u | 0x02u) | 1 << (0x01u | 0x04u | 0x02u) | 1 << (0x08u | 0x02u | 0x01u) | 1 << (0x08u | 0x04u | 0x01u) | 1 << (0x08u | 0x04u | 0x02u) | 1 << (0x08u | 0x04u | 0x01u | 0x02u) ) >> bit) & 1))));
return z;
}
# 472 "./include/linux/gfp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int gfp_zonelist(gfp_t flags)
{
return ZONELIST_FALLBACK;
}
# 490 "./include/linux/gfp.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct zonelist *node_zonelist(int nid, gfp_t flags)
{
return (&contig_page_data)->node_zonelists + gfp_zonelist(flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_free_page(struct page *page, int order) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_alloc_page(struct page *page, int order) { }
struct page *
__alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid,
nodemask_t *nodemask);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *
__alloc_pages(gfp_t gfp_mask, unsigned int order, int preferred_nid)
{
return __alloc_pages_nodemask(gfp_mask, order, preferred_nid, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *
__alloc_pages_node(int nid, gfp_t gfp_mask, unsigned int order)
{
((void)(sizeof(( long)(nid < 0 || nid >= (1 << 0)))));
((void)(sizeof(( long)(!node_state((nid), N_ONLINE)))));
return __alloc_pages(gfp_mask, order, nid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *alloc_pages_node(int nid, gfp_t gfp_mask,
unsigned int order)
{
if (nid == (-1))
nid = numa_mem_id();
return __alloc_pages_node(nid, gfp_mask, order);
}
# 566 "./include/linux/gfp.h"
extern unsigned long __get_free_pages(gfp_t gfp_mask, unsigned int order);
extern unsigned long get_zeroed_page(gfp_t gfp_mask);
void *alloc_pages_exact(size_t size, gfp_t gfp_mask);
void free_pages_exact(void *virt, size_t size);
void * __attribute__ ((__section__(".meminit.text"))) __attribute__((no_instrument_function)) alloc_pages_exact_nid(int nid, size_t size, gfp_t gfp_mask);
extern void __free_pages(struct page *page, unsigned int order);
extern void free_pages(unsigned long addr, unsigned int order);
extern void free_hot_cold_page(struct page *page, bool cold);
extern void free_hot_cold_page_list(struct list_head *list, bool cold);
struct page_frag_cache;
extern void __page_frag_cache_drain(struct page *page, unsigned int count);
extern void *page_frag_alloc(struct page_frag_cache *nc,
unsigned int fragsz, gfp_t gfp_mask);
extern void page_frag_free(void *addr);
void page_alloc_init(void);
void drain_zone_pages(struct zone *zone, struct per_cpu_pages *pcp);
void drain_all_pages(struct zone *zone);
void drain_local_pages(struct zone *zone);
void page_alloc_init_late(void);
# 607 "./include/linux/gfp.h"
extern gfp_t gfp_allowed_mask;
bool gfp_pfmemalloc_allowed(gfp_t gfp_mask);
extern void pm_restrict_gfp_mask(void);
extern void pm_restore_gfp_mask(void);
extern bool pm_suspended_storage(void);
# 626 "./include/linux/gfp.h"
extern int alloc_contig_range(unsigned long start, unsigned long end,
unsigned migratetype, gfp_t gfp_mask);
extern void free_contig_range(unsigned long pfn, unsigned nr_pages);
extern void init_cma_reserved_pageblock(struct page *page);
# 16 "./include/linux/slab.h" 2
# 114 "./include/linux/slab.h"
# 1 "./include/linux/kmemleak.h" 1
# 24 "./include/linux/kmemleak.h"
# 1 "./include/linux/slab.h" 1
# 25 "./include/linux/kmemleak.h" 2
# 1 "./include/linux/vmalloc.h" 1
# 12 "./include/linux/vmalloc.h"
struct vm_area_struct;
struct notifier_block;
# 36 "./include/linux/vmalloc.h"
struct vm_struct {
struct vm_struct *next;
void *addr;
unsigned long size;
unsigned long flags;
struct page **pages;
unsigned int nr_pages;
phys_addr_t phys_addr;
const void *caller;
};
struct vmap_area {
unsigned long va_start;
unsigned long va_end;
unsigned long flags;
struct rb_node rb_node;
struct list_head list;
struct llist_node purge_list;
struct vm_struct *vm;
struct callback_head callback_head;
};
extern void vm_unmap_ram(const void *mem, unsigned int count);
extern void *vm_map_ram(struct page **pages, unsigned int count,
int node, pgprot_t prot);
extern void vm_unmap_aliases(void);
extern void __attribute__ ((__section__(".init.text"))) vmalloc_init(void);
extern unsigned long vmalloc_nr_pages(void);
extern void *vmalloc(unsigned long size);
extern void *vzalloc(unsigned long size);
extern void *vmalloc_user(unsigned long size);
extern void *vmalloc_node(unsigned long size, int node);
extern void *vzalloc_node(unsigned long size, int node);
extern void *vmalloc_exec(unsigned long size);
extern void *vmalloc_32(unsigned long size);
extern void *vmalloc_32_user(unsigned long size);
extern void *__vmalloc(unsigned long size, gfp_t gfp_mask, pgprot_t prot);
extern void *__vmalloc_node_range(unsigned long size, unsigned long align,
unsigned long start, unsigned long end, gfp_t gfp_mask,
pgprot_t prot, unsigned long vm_flags, int node,
const void *caller);
# 97 "./include/linux/vmalloc.h"
extern void *__vmalloc_node_flags_caller(unsigned long size,
int node, gfp_t flags, void *caller);
extern void vfree(const void *addr);
extern void vfree_atomic(const void *addr);
extern void *vmap(struct page **pages, unsigned int count,
unsigned long flags, pgprot_t prot);
extern void vunmap(const void *addr);
extern int remap_vmalloc_range_partial(struct vm_area_struct *vma,
unsigned long uaddr, void *kaddr,
unsigned long pgoff, unsigned long size);
extern int remap_vmalloc_range(struct vm_area_struct *vma, void *addr,
unsigned long pgoff);
void vmalloc_sync_mappings(void);
void vmalloc_sync_unmappings(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) size_t get_vm_area_size(const struct vm_struct *area)
{
if (!(area->flags & 0x00000040))
return area->size - ((1UL) << 12);
else
return area->size;
}
extern struct vm_struct *get_vm_area(unsigned long size, unsigned long flags);
extern struct vm_struct *get_vm_area_caller(unsigned long size,
unsigned long flags, const void *caller);
extern struct vm_struct *__get_vm_area(unsigned long size, unsigned long flags,
unsigned long start, unsigned long end);
extern struct vm_struct *__get_vm_area_caller(unsigned long size,
unsigned long flags,
unsigned long start, unsigned long end,
const void *caller);
extern struct vm_struct *remove_vm_area(const void *addr);
extern struct vm_struct *find_vm_area(const void *addr);
extern int map_vm_area(struct vm_struct *area, pgprot_t prot,
struct page **pages);
extern int map_kernel_range_noflush(unsigned long start, unsigned long size,
pgprot_t prot, struct page **pages);
extern void unmap_kernel_range_noflush(unsigned long addr, unsigned long size);
extern void unmap_kernel_range(unsigned long addr, unsigned long size);
# 168 "./include/linux/vmalloc.h"
extern struct vm_struct *alloc_vm_area(size_t size, pte_t **ptes);
extern void free_vm_area(struct vm_struct *area);
extern long vread(char *buf, char *addr, unsigned long count);
extern long vwrite(char *buf, char *addr, unsigned long count);
extern struct list_head vmap_area_list;
extern __attribute__ ((__section__(".init.text"))) void vm_area_add_early(struct vm_struct *vm);
extern __attribute__ ((__section__(".init.text"))) void vm_area_register_early(struct vm_struct *vm, size_t align);
struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,
const size_t *sizes, int nr_vms,
size_t align);
void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms);
# 211 "./include/linux/vmalloc.h"
int register_vmap_purge_notifier(struct notifier_block *nb);
int unregister_vmap_purge_notifier(struct notifier_block *nb);
# 26 "./include/linux/kmemleak.h" 2
# 71 "./include/linux/kmemleak.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_init(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_alloc(const void *ptr, size_t size, int min_count,
gfp_t gfp)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_alloc_recursive(const void *ptr, size_t size,
int min_count, unsigned long flags,
gfp_t gfp)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_alloc_percpu(const void *ptr, size_t size,
gfp_t gfp)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_vmalloc(const struct vm_struct *area, size_t size,
gfp_t gfp)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_free(const void *ptr)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_free_part(const void *ptr, size_t size)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_free_recursive(const void *ptr, unsigned long flags)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_free_percpu(const void *ptr)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_update_trace(const void *ptr)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_not_leak(const void *ptr)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_ignore(const void *ptr)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_scan_area(const void *ptr, size_t size, gfp_t gfp)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_erase(void **ptr)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_no_scan(const void *ptr)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_alloc_phys(phys_addr_t phys, size_t size,
int min_count, gfp_t gfp)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_free_part_phys(phys_addr_t phys, size_t size)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_not_leak_phys(phys_addr_t phys)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmemleak_ignore_phys(phys_addr_t phys)
{
}
# 115 "./include/linux/slab.h" 2
# 1 "./include/linux/kasan.h" 1
struct kmem_cache;
struct page;
struct vm_struct;
struct task_struct;
# 88 "./include/linux/kasan.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_unpoison_shadow(const void *address, size_t size) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_unpoison_task_stack(struct task_struct *task) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_unpoison_stack_above_sp_to(const void *watermark) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_enable_current(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_disable_current(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_alloc_pages(struct page *page, unsigned int order) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_free_pages(struct page *page, unsigned int order) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_cache_create(struct kmem_cache *cache,
unsigned int *size,
unsigned long *flags) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_poison_slab(struct page *page) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_unpoison_object_data(struct kmem_cache *cache,
void *object) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_poison_object_data(struct kmem_cache *cache,
void *object) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kasan_init_slab_obj(struct kmem_cache *cache,
const void *object)
{
return (void *)object;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags)
{
return ptr;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_kfree_large(void *ptr, unsigned long ip) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_poison_kfree(void *ptr, unsigned long ip) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kasan_kmalloc(struct kmem_cache *s, const void *object,
size_t size, gfp_t flags)
{
return (void *)object;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kasan_krealloc(const void *object, size_t new_size,
gfp_t flags)
{
return (void *)object;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kasan_slab_alloc(struct kmem_cache *s, void *object,
gfp_t flags)
{
return object;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool kasan_slab_free(struct kmem_cache *s, void *object,
unsigned long ip)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kasan_module_alloc(void *addr, size_t size) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_free_shadow(const struct vm_struct *vm) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kasan_add_zero_shadow(void *start, unsigned long size)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_remove_zero_shadow(void *start,
unsigned long size)
{}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_unpoison_slab(const void *ptr) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
# 167 "./include/linux/kasan.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_cache_shrink(struct kmem_cache *cache) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_cache_shutdown(struct kmem_cache *cache) {}
# 185 "./include/linux/kasan.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kasan_init_tags(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kasan_reset_tag(const void *addr)
{
return (void *)addr;
}
# 116 "./include/linux/slab.h" 2
struct mem_cgroup;
void __attribute__ ((__section__(".init.text"))) kmem_cache_init(void);
bool slab_is_available(void);
struct kmem_cache *kmem_cache_create(const char *, size_t, size_t,
unsigned long,
void (*)(void *));
void kmem_cache_destroy(struct kmem_cache *);
int kmem_cache_shrink(struct kmem_cache *);
void memcg_create_kmem_cache(struct mem_cgroup *, struct kmem_cache *);
void memcg_deactivate_kmem_caches(struct mem_cgroup *);
void memcg_destroy_kmem_caches(struct mem_cgroup *);
# 149 "./include/linux/slab.h"
void * __attribute__((warn_unused_result)) __krealloc(const void *, size_t, gfp_t);
void * __attribute__((warn_unused_result)) krealloc(const void *, size_t, gfp_t);
void kfree(const void *);
void kzfree(const void *);
size_t ksize(const void *);
const char *__check_heap_object(const void *ptr, unsigned long n,
struct page *page);
# 274 "./include/linux/slab.h"
enum kmalloc_cache_type {
KMALLOC_NORMAL = 0,
KMALLOC_RECLAIM,
NR_KMALLOC_TYPES
};
extern struct kmem_cache *
kmalloc_caches[NR_KMALLOC_TYPES][(12 + 1) + 1];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) enum kmalloc_cache_type kmalloc_type(gfp_t flags)
{
# 303 "./include/linux/slab.h"
return flags & (( gfp_t)0x10u) ? KMALLOC_RECLAIM : KMALLOC_NORMAL;
}
# 315 "./include/linux/slab.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int kmalloc_index(size_t size)
{
if (!size)
return 0;
if (size <= (1 << 6))
return ( __builtin_constant_p((1 << 6)) ? ( __builtin_constant_p((1 << 6)) ? ( ((1 << 6)) < 2 ? 0 : ((1 << 6)) & (1ULL << 63) ? 63 : ((1 << 6)) & (1ULL << 62) ? 62 : ((1 << 6)) & (1ULL << 61) ? 61 : ((1 << 6)) & (1ULL << 60) ? 60 : ((1 << 6)) & (1ULL << 59) ? 59 : ((1 << 6)) & (1ULL << 58) ? 58 : ((1 << 6)) & (1ULL << 57) ? 57 : ((1 << 6)) & (1ULL << 56) ? 56 : ((1 << 6)) & (1ULL << 55) ? 55 : ((1 << 6)) & (1ULL << 54) ? 54 : ((1 << 6)) & (1ULL << 53) ? 53 : ((1 << 6)) & (1ULL << 52) ? 52 : ((1 << 6)) & (1ULL << 51) ? 51 : ((1 << 6)) & (1ULL << 50) ? 50 : ((1 << 6)) & (1ULL << 49) ? 49 : ((1 << 6)) & (1ULL << 48) ? 48 : ((1 << 6)) & (1ULL << 47) ? 47 : ((1 << 6)) & (1ULL << 46) ? 46 : ((1 << 6)) & (1ULL << 45) ? 45 : ((1 << 6)) & (1ULL << 44) ? 44 : ((1 << 6)) & (1ULL << 43) ? 43 : ((1 << 6)) & (1ULL << 42) ? 42 : ((1 << 6)) & (1ULL << 41) ? 41 : ((1 << 6)) & (1ULL << 40) ? 40 : ((1 << 6)) & (1ULL << 39) ? 39 : ((1 << 6)) & (1ULL << 38) ? 38 : ((1 << 6)) & (1ULL << 37) ? 37 : ((1 << 6)) & (1ULL << 36) ? 36 : ((1 << 6)) & (1ULL << 35) ? 35 : ((1 << 6)) & (1ULL << 34) ? 34 : ((1 << 6)) & (1ULL << 33) ? 33 : ((1 << 6)) & (1ULL << 32) ? 32 : ((1 << 6)) & (1ULL << 31) ? 31 : ((1 << 6)) & (1ULL << 30) ? 30 : ((1 << 6)) & (1ULL << 29) ? 29 : ((1 << 6)) & (1ULL << 28) ? 28 : ((1 << 6)) & (1ULL << 27) ? 27 : ((1 << 6)) & (1ULL << 26) ? 26 : ((1 << 6)) & (1ULL << 25) ? 25 : ((1 << 6)) & (1ULL << 24) ? 24 : ((1 << 6)) & (1ULL << 23) ? 23 : ((1 << 6)) & (1ULL << 22) ? 22 : ((1 << 6)) & (1ULL << 21) ? 21 : ((1 << 6)) & (1ULL << 20) ? 20 : ((1 << 6)) & (1ULL << 19) ? 19 : ((1 << 6)) & (1ULL << 18) ? 18 : ((1 << 6)) & (1ULL << 17) ? 17 : ((1 << 6)) & (1ULL << 16) ? 16 : ((1 << 6)) & (1ULL << 15) ? 15 : ((1 << 6)) & (1ULL << 14) ? 14 : ((1 << 6)) & (1ULL << 13) ? 13 : ((1 << 6)) & (1ULL << 12) ? 12 : ((1 << 6)) & (1ULL << 11) ? 11 : ((1 << 6)) & (1ULL << 10) ? 10 : ((1 << 6)) & (1ULL << 9) ? 9 : ((1 << 6)) & (1ULL << 8) ? 8 : ((1 << 6)) & (1ULL << 7) ? 7 : ((1 << 6)) & (1ULL << 6) ? 6 : ((1 << 6)) & (1ULL << 5) ? 5 : ((1 << 6)) & (1ULL << 4) ? 4 : ((1 << 6)) & (1ULL << 3) ? 3 : ((1 << 6)) & (1ULL << 2) ? 2 : 1) : -1) : (sizeof((1 << 6)) <= 4) ? __ilog2_u32((1 << 6)) : __ilog2_u64((1 << 6)) );
if ((1 << 6) <= 32 && size > 64 && size <= 96)
return 1;
if ((1 << 6) <= 64 && size > 128 && size <= 192)
return 2;
if (size <= 8) return 3;
if (size <= 16) return 4;
if (size <= 32) return 5;
if (size <= 64) return 6;
if (size <= 128) return 7;
if (size <= 256) return 8;
if (size <= 512) return 9;
if (size <= 1024) return 10;
if (size <= 2 * 1024) return 11;
if (size <= 4 * 1024) return 12;
if (size <= 8 * 1024) return 13;
if (size <= 16 * 1024) return 14;
if (size <= 32 * 1024) return 15;
if (size <= 64 * 1024) return 16;
if (size <= 128 * 1024) return 17;
if (size <= 256 * 1024) return 18;
if (size <= 512 * 1024) return 19;
if (size <= 1024 * 1024) return 20;
if (size <= 2 * 1024 * 1024) return 21;
if (size <= 4 * 1024 * 1024) return 22;
if (size <= 8 * 1024 * 1024) return 23;
if (size <= 16 * 1024 * 1024) return 24;
if (size <= 32 * 1024 * 1024) return 25;
if (size <= 64 * 1024 * 1024) return 26;
do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/slab.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "351" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0);
return -1;
}
void *__kmalloc(size_t size, gfp_t flags) __attribute__((__malloc__));
void *kmem_cache_alloc(struct kmem_cache *, gfp_t flags) __attribute__((__malloc__));
void kmem_cache_free(struct kmem_cache *, void *);
# 369 "./include/linux/slab.h"
void kmem_cache_free_bulk(struct kmem_cache *, size_t, void **);
int kmem_cache_alloc_bulk(struct kmem_cache *, gfp_t, size_t, void **);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void kfree_bulk(size_t size, void **p)
{
kmem_cache_free_bulk(((void *)0), size, p);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void *__kmalloc_node(size_t size, gfp_t flags, int node)
{
return __kmalloc(size, flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void *kmem_cache_alloc_node(struct kmem_cache *s, gfp_t flags, int node)
{
return kmem_cache_alloc(s, flags);
}
extern void *kmem_cache_alloc_trace(struct kmem_cache *, gfp_t, size_t) __attribute__((__malloc__));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void *
kmem_cache_alloc_node_trace(struct kmem_cache *s,
gfp_t gfpflags,
int node, size_t size)
{
return kmem_cache_alloc_trace(s, gfpflags, size);
}
# 435 "./include/linux/slab.h"
extern void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) __attribute__((__malloc__));
extern void *kmalloc_order_trace(size_t size, gfp_t flags, unsigned int order) __attribute__((__malloc__));
# 447 "./include/linux/slab.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void *kmalloc_large(size_t size, gfp_t flags)
{
unsigned int order = get_order(size);
return kmalloc_order_trace(size, flags, order);
}
# 507 "./include/linux/slab.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void *kmalloc(size_t size, gfp_t flags)
{
if (__builtin_constant_p(size)) {
unsigned int index;
if (size > (1UL << (12 + 1)))
return kmalloc_large(size, flags);
index = kmalloc_index(size);
if (!index)
return ((void *)16);
return kmem_cache_alloc_trace(
kmalloc_caches[kmalloc_type(flags)][index],
flags, size);
}
return __kmalloc(size, flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int kmalloc_size(int n)
{
if (n > 2)
return 1 << n;
if (n == 1 && (1 << 6) <= 32)
return 96;
if (n == 2 && (1 << 6) <= 64)
return 192;
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void *kmalloc_node(size_t size, gfp_t flags, int node)
{
if (__builtin_constant_p(size) &&
size <= (1UL << (12 + 1))) {
int i = kmalloc_index(size);
if (!i)
return ((void *)16);
return kmem_cache_alloc_node_trace(
kmalloc_caches[kmalloc_type(flags)][i],
flags, node, size);
}
return __kmalloc_node(size, flags, node);
}
struct memcg_cache_array {
struct callback_head rcu;
struct kmem_cache *entries[0];
};
# 605 "./include/linux/slab.h"
struct memcg_cache_params {
struct kmem_cache *root_cache;
union {
struct {
struct memcg_cache_array *memcg_caches;
struct list_head __root_caches_node;
struct list_head children;
};
struct {
struct mem_cgroup *memcg;
struct list_head children_node;
struct list_head kmem_caches_node;
void (*deact_fn)(struct kmem_cache *);
union {
struct callback_head deact_rcu_head;
struct work_struct deact_work;
};
};
};
};
int memcg_update_all_caches(int num_memcgs);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kmalloc_array(size_t n, size_t size, gfp_t flags)
{
if (size != 0 && n > (~(size_t)0) / size)
return ((void *)0);
if (__builtin_constant_p(n) && __builtin_constant_p(size))
return kmalloc(n * size, flags);
return __kmalloc(n * size, flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kcalloc(size_t n, size_t size, gfp_t flags)
{
return kmalloc_array(n, size, flags | (( gfp_t)0x8000u));
}
# 663 "./include/linux/slab.h"
extern void *__kmalloc_track_caller(size_t, gfp_t, unsigned long);
# 683 "./include/linux/slab.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kmem_cache_zalloc(struct kmem_cache *k, gfp_t flags)
{
return kmem_cache_alloc(k, flags | (( gfp_t)0x8000u));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kzalloc(size_t size, gfp_t flags)
{
return kmalloc(size, flags | (( gfp_t)0x8000u));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kzalloc_node(size_t size, gfp_t flags, int node)
{
return kmalloc_node(size, flags | (( gfp_t)0x8000u), node);
}
unsigned int kmem_cache_size(struct kmem_cache *s);
void __attribute__ ((__section__(".init.text"))) kmem_cache_init_late(void);
# 17 "./include/linux/genhd.h" 2
# 1 "./include/linux/percpu-refcount.h" 1
# 55 "./include/linux/percpu-refcount.h"
struct percpu_ref;
typedef void (percpu_ref_func_t)(struct percpu_ref *);
enum {
__PERCPU_REF_ATOMIC = 1LU << 0,
__PERCPU_REF_DEAD = 1LU << 1,
__PERCPU_REF_ATOMIC_DEAD = __PERCPU_REF_ATOMIC | __PERCPU_REF_DEAD,
__PERCPU_REF_FLAG_BITS = 2,
};
enum {
PERCPU_REF_INIT_ATOMIC = 1 << 0,
PERCPU_REF_INIT_DEAD = 1 << 1,
};
struct percpu_ref {
atomic_long_t count;
unsigned long percpu_count_ptr;
percpu_ref_func_t *release;
percpu_ref_func_t *confirm_switch;
bool force_atomic:1;
struct callback_head rcu;
};
int __attribute__((warn_unused_result)) percpu_ref_init(struct percpu_ref *ref,
percpu_ref_func_t *release, unsigned int flags,
gfp_t gfp);
void percpu_ref_exit(struct percpu_ref *ref);
void percpu_ref_switch_to_atomic(struct percpu_ref *ref,
percpu_ref_func_t *confirm_switch);
void percpu_ref_switch_to_atomic_sync(struct percpu_ref *ref);
void percpu_ref_switch_to_percpu(struct percpu_ref *ref);
void percpu_ref_kill_and_confirm(struct percpu_ref *ref,
percpu_ref_func_t *confirm_kill);
void percpu_ref_reinit(struct percpu_ref *ref);
# 119 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_ref_kill(struct percpu_ref *ref)
{
percpu_ref_kill_and_confirm(ref, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool __ref_is_percpu(struct percpu_ref *ref,
unsigned long **percpu_countp)
{
unsigned long percpu_ptr;
# 143 "./include/linux/percpu-refcount.h"
percpu_ptr = ({ union { typeof(ref->percpu_count_ptr) __val; char __c[1]; } __u; if (1) __read_once_size(&(ref->percpu_count_ptr), __u.__c, sizeof(ref->percpu_count_ptr)); else __read_once_size_nocheck(&(ref->percpu_count_ptr), __u.__c, sizeof(ref->percpu_count_ptr)); do { } while (0); __u.__val; });
do { } while (0);
if (__builtin_expect(!!(percpu_ptr & __PERCPU_REF_ATOMIC_DEAD), 0))
return false;
*percpu_countp = (unsigned long *)percpu_ptr;
return true;
}
# 170 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_ref_get_many(struct percpu_ref *ref, unsigned long nr)
{
unsigned long *percpu_count;
rcu_read_lock_sched();
if (__ref_is_percpu(ref, &percpu_count))
do { do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(*percpu_count)) { case 1: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += nr; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 2: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += nr; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 4: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += nr; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 8: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += nr; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
else
atomic_long_add(nr, &ref->count);
rcu_read_unlock_sched();
}
# 192 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_ref_get(struct percpu_ref *ref)
{
percpu_ref_get_many(ref, 1);
}
# 206 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool percpu_ref_tryget(struct percpu_ref *ref)
{
unsigned long *percpu_count;
bool ret;
rcu_read_lock_sched();
if (__ref_is_percpu(ref, &percpu_count)) {
do { do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(*percpu_count)) { case 1: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 2: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 4: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 8: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
ret = true;
} else {
ret = atomic_add_unless(((atomic_t *)(&ref->count)), 1, 0);
}
rcu_read_unlock_sched();
return ret;
}
# 240 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool percpu_ref_tryget_live(struct percpu_ref *ref)
{
unsigned long *percpu_count;
bool ret = false;
rcu_read_lock_sched();
if (__ref_is_percpu(ref, &percpu_count)) {
do { do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(*percpu_count)) { case 1: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 2: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 4: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 8: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
ret = true;
} else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) {
ret = atomic_add_unless(((atomic_t *)(&ref->count)), 1, 0);
}
rcu_read_unlock_sched();
return ret;
}
# 269 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_ref_put_many(struct percpu_ref *ref, unsigned long nr)
{
unsigned long *percpu_count;
rcu_read_lock_sched();
if (__ref_is_percpu(ref, &percpu_count))
do { do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(*percpu_count)) { case 1: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*percpu_count))(nr); } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 2: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*percpu_count))(nr); } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 4: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*percpu_count))(nr); } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 8: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(*percpu_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))); (typeof((typeof(*(&(*percpu_count))) *)(&(*percpu_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*percpu_count))(nr); } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
else if (__builtin_expect(!!(atomic_long_sub_and_test(nr, &ref->count)), 0))
ref->release(ref);
rcu_read_unlock_sched();
}
# 292 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_ref_put(struct percpu_ref *ref)
{
percpu_ref_put_many(ref, 1);
}
# 306 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool percpu_ref_is_dying(struct percpu_ref *ref)
{
return ref->percpu_count_ptr & __PERCPU_REF_DEAD;
}
# 319 "./include/linux/percpu-refcount.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool percpu_ref_is_zero(struct percpu_ref *ref)
{
unsigned long *percpu_count;
if (__ref_is_percpu(ref, &percpu_count))
return false;
return !atomic_long_read(&ref->count);
}
# 18 "./include/linux/genhd.h" 2
# 1 "./include/linux/uuid.h" 1
# 19 "./include/linux/uuid.h"
# 1 "./include/uapi/linux/uuid.h" 1
# 24 "./include/uapi/linux/uuid.h"
typedef struct {
__u8 b[16];
} guid_t;
# 36 "./include/uapi/linux/uuid.h"
typedef guid_t uuid_le;
# 20 "./include/linux/uuid.h" 2
typedef struct {
__u8 b[16];
} uuid_t;
# 40 "./include/linux/uuid.h"
extern const guid_t guid_null;
extern const uuid_t uuid_null;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool guid_equal(const guid_t *u1, const guid_t *u2)
{
return memcmp(u1, u2, sizeof(guid_t)) == 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void guid_copy(guid_t *dst, const guid_t *src)
{
memcpy(dst, src, sizeof(guid_t));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool guid_is_null(const guid_t *guid)
{
return guid_equal(guid, &guid_null);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uuid_equal(const uuid_t *u1, const uuid_t *u2)
{
return memcmp(u1, u2, sizeof(uuid_t)) == 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void uuid_copy(uuid_t *dst, const uuid_t *src)
{
memcpy(dst, src, sizeof(uuid_t));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool uuid_is_null(const uuid_t *uuid)
{
return uuid_equal(uuid, &uuid_null);
}
void generate_random_uuid(unsigned char uuid[16]);
extern void guid_gen(guid_t *u);
extern void uuid_gen(uuid_t *u);
bool __attribute__((warn_unused_result)) uuid_is_valid(const char *uuid);
extern const u8 guid_index[16];
extern const u8 uuid_index[16];
int guid_parse(const char *uuid, guid_t *u);
int uuid_parse(const char *uuid, uuid_t *u);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int uuid_le_cmp(const guid_t u1, const guid_t u2)
{
return memcmp(&u1, &u2, sizeof(guid_t));
}
# 19 "./include/linux/genhd.h" 2
extern struct device_type part_type;
extern struct kobject *block_depr;
extern struct class block_class;
enum {
DOS_EXTENDED_PARTITION = 5,
LINUX_EXTENDED_PARTITION = 0x85,
WIN98_EXTENDED_PARTITION = 0x0f,
SUN_WHOLE_DISK = DOS_EXTENDED_PARTITION,
LINUX_SWAP_PARTITION = 0x82,
LINUX_DATA_PARTITION = 0x83,
LINUX_LVM_PARTITION = 0x8e,
LINUX_RAID_PARTITION = 0xfd,
SOLARIS_X86_PARTITION = LINUX_SWAP_PARTITION,
NEW_SOLARIS_X86_PARTITION = 0xbf,
DM6_AUX1PARTITION = 0x51,
DM6_AUX3PARTITION = 0x53,
DM6_PARTITION = 0x54,
EZD_PARTITION = 0x55,
FREEBSD_PARTITION = 0xa5,
OPENBSD_PARTITION = 0xa6,
NETBSD_PARTITION = 0xa9,
BSDI_PARTITION = 0xb7,
MINIX_PARTITION = 0x81,
UNIXWARE_PARTITION = 0x63,
};
# 1 "./include/linux/device.h" 1
# 16 "./include/linux/device.h"
# 1 "./include/linux/ioport.h" 1
# 19 "./include/linux/ioport.h"
struct resource {
resource_size_t start;
resource_size_t end;
const char *name;
unsigned long flags;
unsigned long desc;
struct resource *parent, *sibling, *child;
};
# 127 "./include/linux/ioport.h"
enum {
IORES_DESC_NONE = 0,
IORES_DESC_CRASH_KERNEL = 1,
IORES_DESC_ACPI_TABLES = 2,
IORES_DESC_ACPI_NV_STORAGE = 3,
IORES_DESC_PERSISTENT_MEMORY = 4,
IORES_DESC_PERSISTENT_MEMORY_LEGACY = 5,
IORES_DESC_DEVICE_PRIVATE_MEMORY = 6,
IORES_DESC_DEVICE_PUBLIC_MEMORY = 7,
};
# 169 "./include/linux/ioport.h"
extern struct resource ioport_resource;
extern struct resource iomem_resource;
extern struct resource *request_resource_conflict(struct resource *root, struct resource *new);
extern int request_resource(struct resource *root, struct resource *new);
extern int release_resource(struct resource *new);
void release_child_resources(struct resource *new);
extern void reserve_region_with_split(struct resource *root,
resource_size_t start, resource_size_t end,
const char *name);
extern struct resource *insert_resource_conflict(struct resource *parent, struct resource *new);
extern int insert_resource(struct resource *parent, struct resource *new);
extern void insert_resource_expand_to_fit(struct resource *root, struct resource *new);
extern int remove_resource(struct resource *old);
extern void arch_remove_reservations(struct resource *avail);
extern int allocate_resource(struct resource *root, struct resource *new,
resource_size_t size, resource_size_t min,
resource_size_t max, resource_size_t align,
resource_size_t (*alignf)(void *,
const struct resource *,
resource_size_t,
resource_size_t),
void *alignf_data);
struct resource *lookup_resource(struct resource *root, resource_size_t start);
int adjust_resource(struct resource *res, resource_size_t start,
resource_size_t size);
resource_size_t resource_alignment(struct resource *res);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) resource_size_t resource_size(const struct resource *res)
{
return res->end - res->start + 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long resource_type(const struct resource *res)
{
return res->flags & 0x00001f00;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long resource_ext_type(const struct resource *res)
{
return res->flags & 0x01000000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool resource_contains(struct resource *r1, struct resource *r2)
{
if (resource_type(r1) != resource_type(r2))
return false;
if (r1->flags & 0x20000000 || r2->flags & 0x20000000)
return false;
return r1->start <= r2->start && r1->end >= r2->end;
}
# 228 "./include/linux/ioport.h"
extern struct resource * __request_region(struct resource *,
resource_size_t start,
resource_size_t n,
const char *name, int flags);
extern void __release_region(struct resource *, resource_size_t,
resource_size_t);
struct device;
extern int devm_request_resource(struct device *dev, struct resource *root,
struct resource *new);
extern void devm_release_resource(struct device *dev, struct resource *new);
extern struct resource * __devm_request_region(struct device *dev,
struct resource *parent, resource_size_t start,
resource_size_t n, const char *name);
extern void __devm_release_region(struct device *dev, struct resource *parent,
resource_size_t start, resource_size_t n);
extern int iomem_map_sanity_check(resource_size_t addr, unsigned long size);
extern int iomem_is_exclusive(u64 addr);
extern int
walk_system_ram_range(unsigned long start_pfn, unsigned long nr_pages,
void *arg, int (*func)(unsigned long, unsigned long, void *));
extern int
walk_system_ram_res(u64 start, u64 end, void *arg,
int (*func)(u64, u64, void *));
extern int
walk_iomem_res_desc(unsigned long desc, unsigned long flags, u64 start, u64 end,
void *arg, int (*func)(u64, u64, void *));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool resource_overlaps(struct resource *r1, struct resource *r2)
{
return (r1->start <= r2->end && r1->end >= r2->start);
}
# 17 "./include/linux/device.h" 2
# 1 "./include/linux/kobject.h" 1
# 21 "./include/linux/kobject.h"
# 1 "./include/linux/sysfs.h" 1
# 16 "./include/linux/sysfs.h"
# 1 "./include/linux/kernfs.h" 1
# 14 "./include/linux/kernfs.h"
# 1 "./include/linux/idr.h" 1
# 15 "./include/linux/idr.h"
# 1 "./include/linux/radix-tree.h" 1
# 61 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool radix_tree_is_internal_node(void *ptr)
{
return ((unsigned long)ptr & 3UL) ==
1UL;
}
# 93 "./include/linux/radix-tree.h"
struct radix_tree_node {
unsigned char shift;
unsigned char offset;
unsigned char count;
unsigned char exceptional;
struct radix_tree_node *parent;
struct radix_tree_root *root;
union {
struct list_head private_list;
struct callback_head callback_head;
};
void *slots[(1UL << (1 ? 4 : 6))];
unsigned long tags[3][(((1UL << (1 ? 4 : 6)) + 32 - 1) / 32)];
};
struct radix_tree_root {
gfp_t gfp_mask;
struct radix_tree_node *rnode;
};
# 131 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool radix_tree_empty(const struct radix_tree_root *root)
{
return root->rnode == ((void *)0);
}
# 152 "./include/linux/radix-tree.h"
struct radix_tree_iter {
unsigned long index;
unsigned long next_index;
unsigned long tags;
struct radix_tree_node *node;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int iter_shift(const struct radix_tree_iter *iter)
{
return 0;
}
# 236 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *radix_tree_deref_slot(void **slot)
{
return ({ typeof(*(*slot)) *________p1 = (typeof(*(*slot)) *)({ union { typeof((*slot)) __val; char __c[1]; } __u; if (1) __read_once_size(&((*slot)), __u.__c, sizeof((*slot))); else __read_once_size_nocheck(&((*slot)), __u.__c, sizeof((*slot))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(*slot)) *)(________p1)); });
}
# 250 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *radix_tree_deref_slot_protected(void **slot,
spinlock_t *treelock)
{
return ({ do { } while (0); ; ((typeof(*(*slot)) *)((*slot))); });
}
# 263 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int radix_tree_deref_retry(void *arg)
{
return __builtin_expect(!!(radix_tree_is_internal_node(arg)), 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int radix_tree_exceptional_entry(void *arg)
{
return (unsigned long)arg & 2;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int radix_tree_exception(void *arg)
{
return __builtin_expect(!!((unsigned long)arg & 3UL), 0);
}
int __radix_tree_create(struct radix_tree_root *, unsigned long index,
unsigned order, struct radix_tree_node **nodep,
void ***slotp);
int __radix_tree_insert(struct radix_tree_root *, unsigned long index,
unsigned order, void *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int radix_tree_insert(struct radix_tree_root *root,
unsigned long index, void *entry)
{
return __radix_tree_insert(root, index, 0, entry);
}
void *__radix_tree_lookup(const struct radix_tree_root *, unsigned long index,
struct radix_tree_node **nodep, void ***slotp);
void *radix_tree_lookup(const struct radix_tree_root *, unsigned long);
void **radix_tree_lookup_slot(const struct radix_tree_root *,
unsigned long index);
typedef void (*radix_tree_update_node_t)(struct radix_tree_node *, void *);
void __radix_tree_replace(struct radix_tree_root *, struct radix_tree_node *,
void **slot, void *entry,
radix_tree_update_node_t update_node, void *private);
void radix_tree_iter_replace(struct radix_tree_root *,
const struct radix_tree_iter *, void **slot, void *entry);
void radix_tree_replace_slot(struct radix_tree_root *,
void **slot, void *entry);
void __radix_tree_delete_node(struct radix_tree_root *,
struct radix_tree_node *,
radix_tree_update_node_t update_node,
void *private);
void radix_tree_iter_delete(struct radix_tree_root *,
struct radix_tree_iter *iter, void **slot);
void *radix_tree_delete_item(struct radix_tree_root *, unsigned long, void *);
void *radix_tree_delete(struct radix_tree_root *, unsigned long);
void radix_tree_clear_tags(struct radix_tree_root *, struct radix_tree_node *,
void **slot);
unsigned int radix_tree_gang_lookup(const struct radix_tree_root *,
void **results, unsigned long first_index,
unsigned int max_items);
unsigned int radix_tree_gang_lookup_slot(const struct radix_tree_root *,
void ***results, unsigned long *indices,
unsigned long first_index, unsigned int max_items);
int radix_tree_preload(gfp_t gfp_mask);
int radix_tree_maybe_preload(gfp_t gfp_mask);
int radix_tree_maybe_preload_order(gfp_t gfp_mask, int order);
void radix_tree_init(void);
void *radix_tree_tag_set(struct radix_tree_root *,
unsigned long index, unsigned int tag);
void *radix_tree_tag_clear(struct radix_tree_root *,
unsigned long index, unsigned int tag);
int radix_tree_tag_get(const struct radix_tree_root *,
unsigned long index, unsigned int tag);
void radix_tree_iter_tag_set(struct radix_tree_root *,
const struct radix_tree_iter *iter, unsigned int tag);
void radix_tree_iter_tag_clear(struct radix_tree_root *,
const struct radix_tree_iter *iter, unsigned int tag);
unsigned int radix_tree_gang_lookup_tag(const struct radix_tree_root *,
void **results, unsigned long first_index,
unsigned int max_items, unsigned int tag);
unsigned int radix_tree_gang_lookup_tag_slot(const struct radix_tree_root *,
void ***results, unsigned long first_index,
unsigned int max_items, unsigned int tag);
int radix_tree_tagged(const struct radix_tree_root *, unsigned int tag);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void radix_tree_preload_end(void)
{
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
int radix_tree_split_preload(unsigned old_order, unsigned new_order, gfp_t);
int radix_tree_split(struct radix_tree_root *, unsigned long index,
unsigned new_order);
int radix_tree_join(struct radix_tree_root *, unsigned long index,
unsigned new_order, void *);
void **idr_get_free_cmn(struct radix_tree_root *root,
struct radix_tree_iter *iter, gfp_t gfp,
unsigned long max);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void **idr_get_free(struct radix_tree_root *root,
struct radix_tree_iter *iter,
gfp_t gfp,
int end)
{
return idr_get_free_cmn(root, iter, gfp, end > 0 ? end - 1 : ((int)(~0U>>1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void **idr_get_free_ext(struct radix_tree_root *root,
struct radix_tree_iter *iter,
gfp_t gfp,
unsigned long end)
{
return idr_get_free_cmn(root, iter, gfp, end - 1);
}
enum {
RADIX_TREE_ITER_TAG_MASK = 0x0f,
RADIX_TREE_ITER_TAGGED = 0x10,
RADIX_TREE_ITER_CONTIG = 0x20,
};
# 393 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void **
radix_tree_iter_init(struct radix_tree_iter *iter, unsigned long start)
{
# 404 "./include/linux/radix-tree.h"
iter->index = 0;
iter->next_index = start;
return ((void *)0);
}
# 422 "./include/linux/radix-tree.h"
void **radix_tree_next_chunk(const struct radix_tree_root *,
struct radix_tree_iter *iter, unsigned flags);
# 435 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void **
radix_tree_iter_lookup(const struct radix_tree_root *root,
struct radix_tree_iter *iter, unsigned long index)
{
radix_tree_iter_init(iter, index);
return radix_tree_next_chunk(root, iter, RADIX_TREE_ITER_CONTIG);
}
# 453 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void **
radix_tree_iter_find(const struct radix_tree_root *root,
struct radix_tree_iter *iter, unsigned long index)
{
radix_tree_iter_init(iter, index);
return radix_tree_next_chunk(root, iter, 0);
}
# 470 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result))
void **radix_tree_iter_retry(struct radix_tree_iter *iter)
{
iter->next_index = iter->index;
iter->tags = 0;
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
__radix_tree_iter_add(struct radix_tree_iter *iter, unsigned long slots)
{
return iter->index + (slots << iter_shift(iter));
}
# 494 "./include/linux/radix-tree.h"
void **__attribute__((warn_unused_result)) radix_tree_iter_resume(void **slot,
struct radix_tree_iter *iter);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) long
radix_tree_chunk_size(struct radix_tree_iter *iter)
{
return (iter->next_index - iter->index) >> iter_shift(iter);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void **__radix_tree_next_slot(void **slot,
struct radix_tree_iter *iter, unsigned flags)
{
return slot;
}
# 540 "./include/linux/radix-tree.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void **radix_tree_next_slot(void **slot,
struct radix_tree_iter *iter, unsigned flags)
{
if (flags & RADIX_TREE_ITER_TAGGED) {
iter->tags >>= 1;
if (__builtin_expect(!!(!iter->tags), 0))
return ((void *)0);
if (__builtin_expect(!!(iter->tags & 1ul), 1)) {
iter->index = __radix_tree_iter_add(iter, 1);
slot++;
goto found;
}
if (!(flags & RADIX_TREE_ITER_CONTIG)) {
unsigned offset = __ffs(iter->tags);
iter->tags >>= offset++;
iter->index = __radix_tree_iter_add(iter, offset);
slot += offset;
goto found;
}
} else {
long count = radix_tree_chunk_size(iter);
while (--count > 0) {
slot++;
iter->index = __radix_tree_iter_add(iter, 1);
if (__builtin_expect(!!(*slot), 1))
goto found;
if (flags & RADIX_TREE_ITER_CONTIG) {
iter->next_index = 0;
break;
}
}
}
return ((void *)0);
found:
if (__builtin_expect(!!(radix_tree_is_internal_node(({ typeof(*slot) ________p1 = ({ union { typeof(*slot) __val; char __c[1]; } __u; if (1) __read_once_size(&(*slot), __u.__c, sizeof(*slot)); else __read_once_size_nocheck(&(*slot), __u.__c, sizeof(*slot)); do { } while (0); __u.__val; }); ((typeof(**slot) *)(________p1)); }))), 0))
return __radix_tree_next_slot(slot, iter, flags);
return slot;
}
# 16 "./include/linux/idr.h" 2
struct idr {
struct radix_tree_root idr_rt;
unsigned int idr_next;
};
# 47 "./include/linux/idr.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int idr_get_cursor(const struct idr *idr)
{
return ({ union { typeof(idr->idr_next) __val; char __c[1]; } __u; if (1) __read_once_size(&(idr->idr_next), __u.__c, sizeof(idr->idr_next)); else __read_once_size_nocheck(&(idr->idr_next), __u.__c, sizeof(idr->idr_next)); do { } while (0); __u.__val; });
}
# 60 "./include/linux/idr.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void idr_set_cursor(struct idr *idr, unsigned int val)
{
({ union { typeof(idr->idr_next) __val; char __c[1]; } __u = { .__val = ( typeof(idr->idr_next)) (val) }; __write_once_size(&(idr->idr_next), __u.__c, sizeof(idr->idr_next)); __u.__val; });
}
# 82 "./include/linux/idr.h"
void idr_preload(gfp_t gfp_mask);
int idr_alloc_cmn(struct idr *idr, void *ptr, unsigned long *index,
unsigned long start, unsigned long end, gfp_t gfp,
bool ext);
# 107 "./include/linux/idr.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int idr_alloc(struct idr *idr, void *ptr,
int start, int end, gfp_t gfp)
{
unsigned long id;
int ret;
if (({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(start < 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/idr.h", 113); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); }))
return -22;
ret = idr_alloc_cmn(idr, ptr, &id, start, end, gfp, false);
if (ret)
return ret;
return id;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int idr_alloc_ext(struct idr *idr, void *ptr,
unsigned long *index,
unsigned long start,
unsigned long end,
gfp_t gfp)
{
return idr_alloc_cmn(idr, ptr, index, start, end, gfp, true);
}
int idr_alloc_cyclic(struct idr *, void *entry, int start, int end, gfp_t);
int idr_for_each(const struct idr *,
int (*fn)(int id, void *p, void *data), void *data);
void *idr_get_next(struct idr *, int *nextid);
void *idr_get_next_ext(struct idr *idr, unsigned long *nextid);
void *idr_replace(struct idr *, void *, int id);
void *idr_replace_ext(struct idr *idr, void *ptr, unsigned long id);
void idr_destroy(struct idr *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *idr_remove_ext(struct idr *idr, unsigned long id)
{
return radix_tree_delete_item(&idr->idr_rt, id, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *idr_remove(struct idr *idr, int id)
{
return idr_remove_ext(idr, id);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void idr_init(struct idr *idr)
{
do { (&idr->idr_rt)->gfp_mask = ((( gfp_t)(3 << (25 + 0 + 0)))); (&idr->idr_rt)->rnode = ((void *)0); } while (0);
idr->idr_next = 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool idr_is_empty(const struct idr *idr)
{
return radix_tree_empty(&idr->idr_rt) &&
radix_tree_tagged(&idr->idr_rt, 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void idr_preload_end(void)
{
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
# 187 "./include/linux/idr.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *idr_find_ext(const struct idr *idr, unsigned long id)
{
return radix_tree_lookup(&idr->idr_rt, id);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *idr_find(const struct idr *idr, int id)
{
return idr_find_ext(idr, id);
}
# 234 "./include/linux/idr.h"
struct ida_bitmap {
unsigned long bitmap[(128 / sizeof(long))];
};
extern __attribute__((section(".data..percpu" ""))) __typeof__(struct ida_bitmap *) ida_bitmap;
struct ida {
struct radix_tree_root ida_rt;
};
int ida_pre_get(struct ida *ida, gfp_t gfp_mask);
int ida_get_new_above(struct ida *ida, int starting_id, int *p_id);
void ida_remove(struct ida *ida, int id);
void ida_destroy(struct ida *ida);
int ida_simple_get(struct ida *ida, unsigned int start, unsigned int end,
gfp_t gfp_mask);
void ida_simple_remove(struct ida *ida, unsigned int id);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ida_init(struct ida *ida)
{
do { (&ida->ida_rt)->gfp_mask = ((( gfp_t)(3 << (25 + 0 + 0))) | ((( gfp_t)0x1000000u))); (&ida->ida_rt)->rnode = ((void *)0); } while (0);
}
# 270 "./include/linux/idr.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int ida_get_new(struct ida *ida, int *p_id)
{
return ida_get_new_above(ida, 0, p_id);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool ida_is_empty(const struct ida *ida)
{
return radix_tree_empty(&ida->ida_rt);
}
# 15 "./include/linux/kernfs.h" 2
struct file;
struct dentry;
struct iattr;
struct seq_file;
struct vm_area_struct;
struct super_block;
struct file_system_type;
struct poll_table_struct;
struct kernfs_open_node;
struct kernfs_iattrs;
enum kernfs_node_type {
KERNFS_DIR = 0x0001,
KERNFS_FILE = 0x0002,
KERNFS_LINK = 0x0004,
};
enum kernfs_node_flag {
KERNFS_ACTIVATED = 0x0010,
KERNFS_NS = 0x0020,
KERNFS_HAS_SEQ_SHOW = 0x0040,
KERNFS_HAS_MMAP = 0x0080,
KERNFS_LOCKDEP = 0x0100,
KERNFS_SUICIDAL = 0x0400,
KERNFS_SUICIDED = 0x0800,
KERNFS_EMPTY_DIR = 0x1000,
KERNFS_HAS_RELEASE = 0x2000,
};
enum kernfs_root_flag {
KERNFS_ROOT_CREATE_DEACTIVATED = 0x0001,
# 72 "./include/linux/kernfs.h"
KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK = 0x0002,
KERNFS_ROOT_SUPPORT_EXPORTOP = 0x0004,
};
struct kernfs_elem_dir {
unsigned long subdirs;
struct rb_root children;
struct kernfs_root *root;
};
struct kernfs_elem_symlink {
struct kernfs_node *target_kn;
};
struct kernfs_elem_attr {
const struct kernfs_ops *ops;
struct kernfs_open_node *open;
loff_t size;
struct kernfs_node *notify_next;
};
union kernfs_node_id {
struct {
u32 ino;
u32 generation;
};
u64 id;
};
# 129 "./include/linux/kernfs.h"
struct kernfs_node {
atomic_t count;
atomic_t active;
# 141 "./include/linux/kernfs.h"
struct kernfs_node *parent;
const char *name;
struct rb_node rb;
const void *ns;
unsigned int hash;
union {
struct kernfs_elem_dir dir;
struct kernfs_elem_symlink symlink;
struct kernfs_elem_attr attr;
};
void *priv;
union kernfs_node_id id;
unsigned short flags;
umode_t mode;
struct kernfs_iattrs *iattr;
};
# 169 "./include/linux/kernfs.h"
struct kernfs_syscall_ops {
int (*remount_fs)(struct kernfs_root *root, int *flags, char *data);
int (*show_options)(struct seq_file *sf, struct kernfs_root *root);
int (*mkdir)(struct kernfs_node *parent, const char *name,
umode_t mode);
int (*rmdir)(struct kernfs_node *kn);
int (*rename)(struct kernfs_node *kn, struct kernfs_node *new_parent,
const char *new_name);
int (*show_path)(struct seq_file *sf, struct kernfs_node *kn,
struct kernfs_root *root);
};
struct kernfs_root {
struct kernfs_node *kn;
unsigned int flags;
struct idr ino_idr;
u32 last_ino;
u32 next_generation;
struct kernfs_syscall_ops *syscall_ops;
struct list_head supers;
wait_queue_head_t deactivate_waitq;
};
struct kernfs_open_file {
struct kernfs_node *kn;
struct file *file;
struct seq_file *seq_file;
void *priv;
struct mutex mutex;
struct mutex prealloc_mutex;
int event;
struct list_head list;
char *prealloc_buf;
size_t atomic_write_len;
bool mmapped:1;
bool released:1;
const struct vm_operations_struct *vm_ops;
};
struct kernfs_ops {
int (*open)(struct kernfs_open_file *of);
void (*release)(struct kernfs_open_file *of);
# 238 "./include/linux/kernfs.h"
int (*seq_show)(struct seq_file *sf, void *v);
void *(*seq_start)(struct seq_file *sf, loff_t *ppos);
void *(*seq_next)(struct seq_file *sf, void *v, loff_t *ppos);
void (*seq_stop)(struct seq_file *sf, void *v);
ssize_t (*read)(struct kernfs_open_file *of, char *buf, size_t bytes,
loff_t off);
# 254 "./include/linux/kernfs.h"
size_t atomic_write_len;
bool prealloc;
ssize_t (*write)(struct kernfs_open_file *of, char *buf, size_t bytes,
loff_t off);
unsigned int (*poll)(struct kernfs_open_file *of,
struct poll_table_struct *pt);
int (*mmap)(struct kernfs_open_file *of, struct vm_area_struct *vma);
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) enum kernfs_node_type kernfs_type(struct kernfs_node *kn)
{
return kn->flags & 0x000f;
}
# 290 "./include/linux/kernfs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kernfs_enable_ns(struct kernfs_node *kn)
{
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(kernfs_type(kn) != KERNFS_DIR); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/kernfs.h", 292); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(!(({ union { typeof((&kn->dir.children)->rb_node) __val; char __c[1]; } __u; if (1) __read_once_size(&((&kn->dir.children)->rb_node), __u.__c, sizeof((&kn->dir.children)->rb_node)); else __read_once_size_nocheck(&((&kn->dir.children)->rb_node), __u.__c, sizeof((&kn->dir.children)->rb_node)); do { } while (0); __u.__val; }) == ((void *)0))); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/kernfs.h", 293); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
kn->flags |= KERNFS_NS;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool kernfs_ns_enabled(struct kernfs_node *kn)
{
return kn->flags & KERNFS_NS;
}
int kernfs_name(struct kernfs_node *kn, char *buf, size_t buflen);
int kernfs_path_from_node(struct kernfs_node *root_kn, struct kernfs_node *kn,
char *buf, size_t buflen);
void pr_cont_kernfs_name(struct kernfs_node *kn);
void pr_cont_kernfs_path(struct kernfs_node *kn);
struct kernfs_node *kernfs_get_parent(struct kernfs_node *kn);
struct kernfs_node *kernfs_find_and_get_ns(struct kernfs_node *parent,
const char *name, const void *ns);
struct kernfs_node *kernfs_walk_and_get_ns(struct kernfs_node *parent,
const char *path, const void *ns);
void kernfs_get(struct kernfs_node *kn);
void kernfs_put(struct kernfs_node *kn);
struct kernfs_node *kernfs_node_from_dentry(struct dentry *dentry);
struct kernfs_root *kernfs_root_from_sb(struct super_block *sb);
struct inode *kernfs_get_inode(struct super_block *sb, struct kernfs_node *kn);
struct dentry *kernfs_node_dentry(struct kernfs_node *kn,
struct super_block *sb);
struct kernfs_root *kernfs_create_root(struct kernfs_syscall_ops *scops,
unsigned int flags, void *priv);
void kernfs_destroy_root(struct kernfs_root *root);
struct kernfs_node *kernfs_create_dir_ns(struct kernfs_node *parent,
const char *name, umode_t mode,
void *priv, const void *ns);
struct kernfs_node *kernfs_create_empty_dir(struct kernfs_node *parent,
const char *name);
struct kernfs_node *__kernfs_create_file(struct kernfs_node *parent,
const char *name,
umode_t mode, loff_t size,
const struct kernfs_ops *ops,
void *priv, const void *ns,
struct lock_class_key *key);
struct kernfs_node *kernfs_create_link(struct kernfs_node *parent,
const char *name,
struct kernfs_node *target);
void kernfs_activate(struct kernfs_node *kn);
void kernfs_remove(struct kernfs_node *kn);
void kernfs_break_active_protection(struct kernfs_node *kn);
void kernfs_unbreak_active_protection(struct kernfs_node *kn);
bool kernfs_remove_self(struct kernfs_node *kn);
int kernfs_remove_by_name_ns(struct kernfs_node *parent, const char *name,
const void *ns);
int kernfs_rename_ns(struct kernfs_node *kn, struct kernfs_node *new_parent,
const char *new_name, const void *new_ns);
int kernfs_setattr(struct kernfs_node *kn, const struct iattr *iattr);
unsigned int kernfs_generic_poll(struct kernfs_open_file *of,
struct poll_table_struct *pt);
void kernfs_notify(struct kernfs_node *kn);
const void *kernfs_super_ns(struct super_block *sb);
struct dentry *kernfs_mount_ns(struct file_system_type *fs_type, int flags,
struct kernfs_root *root, unsigned long magic,
bool *new_sb_created, const void *ns);
void kernfs_kill_sb(struct super_block *sb);
struct super_block *kernfs_pin_sb(struct kernfs_root *root, const void *ns);
void kernfs_init(void);
struct kernfs_node *kernfs_get_node_by_id(struct kernfs_root *root,
const union kernfs_node_id *id);
# 487 "./include/linux/kernfs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kernfs_path(struct kernfs_node *kn, char *buf, size_t buflen)
{
return kernfs_path_from_node(kn, ((void *)0), buf, buflen);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kernfs_node *
kernfs_find_and_get(struct kernfs_node *kn, const char *name)
{
return kernfs_find_and_get_ns(kn, name, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kernfs_node *
kernfs_walk_and_get(struct kernfs_node *kn, const char *path)
{
return kernfs_walk_and_get_ns(kn, path, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kernfs_node *
kernfs_create_dir(struct kernfs_node *parent, const char *name, umode_t mode,
void *priv)
{
return kernfs_create_dir_ns(parent, name, mode, priv, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kernfs_node *
kernfs_create_file_ns(struct kernfs_node *parent, const char *name,
umode_t mode, loff_t size, const struct kernfs_ops *ops,
void *priv, const void *ns)
{
struct lock_class_key *key = ((void *)0);
return __kernfs_create_file(parent, name, mode, size, ops, priv, ns,
key);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kernfs_node *
kernfs_create_file(struct kernfs_node *parent, const char *name, umode_t mode,
loff_t size, const struct kernfs_ops *ops, void *priv)
{
return kernfs_create_file_ns(parent, name, mode, size, ops, priv, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kernfs_remove_by_name(struct kernfs_node *parent,
const char *name)
{
return kernfs_remove_by_name_ns(parent, name, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kernfs_rename(struct kernfs_node *kn,
struct kernfs_node *new_parent,
const char *new_name)
{
return kernfs_rename_ns(kn, new_parent, new_name, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dentry *
kernfs_mount(struct file_system_type *fs_type, int flags,
struct kernfs_root *root, unsigned long magic,
bool *new_sb_created)
{
return kernfs_mount_ns(fs_type, flags, root,
magic, new_sb_created, ((void *)0));
}
# 17 "./include/linux/sysfs.h" 2
# 1 "./include/linux/kobject_ns.h" 1
# 20 "./include/linux/kobject_ns.h"
struct sock;
struct kobject;
enum kobj_ns_type {
KOBJ_NS_TYPE_NONE = 0,
KOBJ_NS_TYPE_NET,
KOBJ_NS_TYPES
};
# 40 "./include/linux/kobject_ns.h"
struct kobj_ns_type_operations {
enum kobj_ns_type type;
bool (*current_may_mount)(void);
void *(*grab_current_ns)(void);
const void *(*netlink_ns)(struct sock *sk);
const void *(*initial_ns)(void);
void (*drop_ns)(void *);
};
int kobj_ns_type_register(const struct kobj_ns_type_operations *ops);
int kobj_ns_type_registered(enum kobj_ns_type type);
const struct kobj_ns_type_operations *kobj_child_ns_ops(struct kobject *parent);
const struct kobj_ns_type_operations *kobj_ns_ops(struct kobject *kobj);
bool kobj_ns_current_may_mount(enum kobj_ns_type type);
void *kobj_ns_grab_current(enum kobj_ns_type type);
const void *kobj_ns_netlink(enum kobj_ns_type type, struct sock *sk);
const void *kobj_ns_initial(enum kobj_ns_type type);
void kobj_ns_drop(enum kobj_ns_type type, void *ns);
# 22 "./include/linux/sysfs.h" 2
# 1 "./include/linux/stat.h" 1
# 1 "./arch/arm/include/uapi/asm/stat.h" 1
struct __old_kernel_stat {
unsigned short st_dev;
unsigned short st_ino;
unsigned short st_mode;
unsigned short st_nlink;
unsigned short st_uid;
unsigned short st_gid;
unsigned short st_rdev;
unsigned long st_size;
unsigned long st_atime;
unsigned long st_mtime;
unsigned long st_ctime;
};
struct stat {
unsigned long st_dev;
unsigned long st_ino;
unsigned short st_mode;
unsigned short st_nlink;
unsigned short st_uid;
unsigned short st_gid;
unsigned long st_rdev;
unsigned long st_size;
unsigned long st_blksize;
unsigned long st_blocks;
unsigned long st_atime;
unsigned long st_atime_nsec;
unsigned long st_mtime;
unsigned long st_mtime_nsec;
unsigned long st_ctime;
unsigned long st_ctime_nsec;
unsigned long __unused4;
unsigned long __unused5;
};
struct stat64 {
unsigned long long st_dev;
unsigned char __pad0[4];
unsigned long __st_ino;
unsigned int st_mode;
unsigned int st_nlink;
unsigned long st_uid;
unsigned long st_gid;
unsigned long long st_rdev;
unsigned char __pad3[4];
long long st_size;
unsigned long st_blksize;
unsigned long long st_blocks;
unsigned long st_atime;
unsigned long st_atime_nsec;
unsigned long st_mtime;
unsigned long st_mtime_nsec;
unsigned long st_ctime;
unsigned long st_ctime_nsec;
unsigned long long st_ino;
};
# 7 "./include/linux/stat.h" 2
# 1 "./include/uapi/linux/stat.h" 1
# 56 "./include/uapi/linux/stat.h"
struct statx_timestamp {
__s64 tv_sec;
__u32 tv_nsec;
__s32 __reserved;
};
# 99 "./include/uapi/linux/stat.h"
struct statx {
__u32 stx_mask;
__u32 stx_blksize;
__u64 stx_attributes;
__u32 stx_nlink;
__u32 stx_uid;
__u32 stx_gid;
__u16 stx_mode;
__u16 __spare0[1];
__u64 stx_ino;
__u64 stx_size;
__u64 stx_blocks;
__u64 stx_attributes_mask;
struct statx_timestamp stx_atime;
struct statx_timestamp stx_btime;
struct statx_timestamp stx_ctime;
struct statx_timestamp stx_mtime;
__u32 stx_rdev_major;
__u32 stx_rdev_minor;
__u32 stx_dev_major;
__u32 stx_dev_minor;
__u64 __spare2[14];
};
# 8 "./include/linux/stat.h" 2
# 24 "./include/linux/stat.h"
struct kstat {
u32 result_mask;
umode_t mode;
unsigned int nlink;
uint32_t blksize;
u64 attributes;
u64 attributes_mask;
# 39 "./include/linux/stat.h"
u64 ino;
dev_t dev;
dev_t rdev;
kuid_t uid;
kgid_t gid;
loff_t size;
struct timespec atime;
struct timespec mtime;
struct timespec ctime;
struct timespec btime;
u64 blocks;
};
# 23 "./include/linux/sysfs.h" 2
struct kobject;
struct module;
struct bin_attribute;
enum kobj_ns_type;
struct attribute {
const char *name;
umode_t mode;
};
# 84 "./include/linux/sysfs.h"
struct attribute_group {
const char *name;
umode_t (*is_visible)(struct kobject *,
struct attribute *, int);
umode_t (*is_bin_visible)(struct kobject *,
struct bin_attribute *, int);
struct attribute **attrs;
struct bin_attribute **bin_attrs;
};
# 159 "./include/linux/sysfs.h"
struct file;
struct vm_area_struct;
struct bin_attribute {
struct attribute attr;
size_t size;
void *private;
ssize_t (*read)(struct file *, struct kobject *, struct bin_attribute *,
char *, loff_t, size_t);
ssize_t (*write)(struct file *, struct kobject *, struct bin_attribute *,
char *, loff_t, size_t);
int (*mmap)(struct file *, struct kobject *, struct bin_attribute *attr,
struct vm_area_struct *vma);
};
# 216 "./include/linux/sysfs.h"
struct sysfs_ops {
ssize_t (*show)(struct kobject *, struct attribute *, char *);
ssize_t (*store)(struct kobject *, struct attribute *, const char *, size_t);
};
int __attribute__((warn_unused_result)) sysfs_create_dir_ns(struct kobject *kobj, const void *ns);
void sysfs_remove_dir(struct kobject *kobj);
int __attribute__((warn_unused_result)) sysfs_rename_dir_ns(struct kobject *kobj, const char *new_name,
const void *new_ns);
int __attribute__((warn_unused_result)) sysfs_move_dir_ns(struct kobject *kobj,
struct kobject *new_parent_kobj,
const void *new_ns);
int __attribute__((warn_unused_result)) sysfs_create_mount_point(struct kobject *parent_kobj,
const char *name);
void sysfs_remove_mount_point(struct kobject *parent_kobj,
const char *name);
int __attribute__((warn_unused_result)) sysfs_create_file_ns(struct kobject *kobj,
const struct attribute *attr,
const void *ns);
int __attribute__((warn_unused_result)) sysfs_create_files(struct kobject *kobj,
const struct attribute **attr);
int __attribute__((warn_unused_result)) sysfs_chmod_file(struct kobject *kobj,
const struct attribute *attr, umode_t mode);
struct kernfs_node *sysfs_break_active_protection(struct kobject *kobj,
const struct attribute *attr);
void sysfs_unbreak_active_protection(struct kernfs_node *kn);
void sysfs_remove_file_ns(struct kobject *kobj, const struct attribute *attr,
const void *ns);
bool sysfs_remove_file_self(struct kobject *kobj, const struct attribute *attr);
void sysfs_remove_files(struct kobject *kobj, const struct attribute **attr);
int __attribute__((warn_unused_result)) sysfs_create_bin_file(struct kobject *kobj,
const struct bin_attribute *attr);
void sysfs_remove_bin_file(struct kobject *kobj,
const struct bin_attribute *attr);
int __attribute__((warn_unused_result)) sysfs_create_link(struct kobject *kobj, struct kobject *target,
const char *name);
int __attribute__((warn_unused_result)) sysfs_create_link_nowarn(struct kobject *kobj,
struct kobject *target,
const char *name);
void sysfs_remove_link(struct kobject *kobj, const char *name);
int sysfs_rename_link_ns(struct kobject *kobj, struct kobject *target,
const char *old_name, const char *new_name,
const void *new_ns);
void sysfs_delete_link(struct kobject *dir, struct kobject *targ,
const char *name);
int __attribute__((warn_unused_result)) sysfs_create_group(struct kobject *kobj,
const struct attribute_group *grp);
int __attribute__((warn_unused_result)) sysfs_create_groups(struct kobject *kobj,
const struct attribute_group **groups);
int sysfs_update_group(struct kobject *kobj,
const struct attribute_group *grp);
void sysfs_remove_group(struct kobject *kobj,
const struct attribute_group *grp);
void sysfs_remove_groups(struct kobject *kobj,
const struct attribute_group **groups);
int sysfs_add_file_to_group(struct kobject *kobj,
const struct attribute *attr, const char *group);
void sysfs_remove_file_from_group(struct kobject *kobj,
const struct attribute *attr, const char *group);
int sysfs_merge_group(struct kobject *kobj,
const struct attribute_group *grp);
void sysfs_unmerge_group(struct kobject *kobj,
const struct attribute_group *grp);
int sysfs_add_link_to_group(struct kobject *kobj, const char *group_name,
struct kobject *target, const char *link_name);
void sysfs_remove_link_from_group(struct kobject *kobj, const char *group_name,
const char *link_name);
int __compat_only_sysfs_link_entry_to_kobj(struct kobject *kobj,
struct kobject *target_kobj,
const char *target_name);
void sysfs_notify(struct kobject *kobj, const char *dir, const char *attr);
int __attribute__((warn_unused_result)) sysfs_init(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sysfs_enable_ns(struct kernfs_node *kn)
{
return kernfs_enable_ns(kn);
}
# 512 "./include/linux/sysfs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) sysfs_create_file(struct kobject *kobj,
const struct attribute *attr)
{
return sysfs_create_file_ns(kobj, attr, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sysfs_remove_file(struct kobject *kobj,
const struct attribute *attr)
{
sysfs_remove_file_ns(kobj, attr, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int sysfs_rename_link(struct kobject *kobj, struct kobject *target,
const char *old_name, const char *new_name)
{
return sysfs_rename_link_ns(kobj, target, old_name, new_name, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sysfs_notify_dirent(struct kernfs_node *kn)
{
kernfs_notify(kn);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kernfs_node *sysfs_get_dirent(struct kernfs_node *parent,
const char *name)
{
return kernfs_find_and_get(parent, name);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kernfs_node *sysfs_get(struct kernfs_node *kn)
{
kernfs_get(kn);
return kn;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sysfs_put(struct kernfs_node *kn)
{
kernfs_put(kn);
}
# 22 "./include/linux/kobject.h" 2
# 1 "./include/linux/kref.h" 1
# 21 "./include/linux/kref.h"
struct kref {
refcount_t refcount;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kref_init(struct kref *kref)
{
refcount_set(&kref->refcount, 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int kref_read(const struct kref *kref)
{
return refcount_read(&kref->refcount);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kref_get(struct kref *kref)
{
refcount_inc(&kref->refcount);
}
# 67 "./include/linux/kref.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kref_put(struct kref *kref, void (*release)(struct kref *kref))
{
if (refcount_dec_and_test(&kref->refcount)) {
release(kref);
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kref_put_mutex(struct kref *kref,
void (*release)(struct kref *kref),
struct mutex *lock)
{
if (refcount_dec_and_mutex_lock(&kref->refcount, lock)) {
release(kref);
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kref_put_lock(struct kref *kref,
void (*release)(struct kref *kref),
spinlock_t *lock)
{
if (refcount_dec_and_lock(&kref->refcount, lock)) {
release(kref);
return 1;
}
return 0;
}
# 114 "./include/linux/kref.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) kref_get_unless_zero(struct kref *kref)
{
return refcount_inc_not_zero(&kref->refcount);
}
# 25 "./include/linux/kobject.h" 2
# 37 "./include/linux/kobject.h"
extern char uevent_helper[];
extern u64 uevent_seqnum;
# 53 "./include/linux/kobject.h"
enum kobject_action {
KOBJ_ADD,
KOBJ_REMOVE,
KOBJ_CHANGE,
KOBJ_MOVE,
KOBJ_ONLINE,
KOBJ_OFFLINE,
KOBJ_BIND,
KOBJ_UNBIND,
KOBJ_MAX
};
struct kobject {
const char *name;
struct list_head entry;
struct kobject *parent;
struct kset *kset;
struct kobj_type *ktype;
struct kernfs_node *sd;
struct kref kref;
unsigned int state_initialized:1;
unsigned int state_in_sysfs:1;
unsigned int state_add_uevent_sent:1;
unsigned int state_remove_uevent_sent:1;
unsigned int uevent_suppress:1;
};
extern __attribute__((format(printf, 2, 3)))
int kobject_set_name(struct kobject *kobj, const char *name, ...);
extern __attribute__((format(printf, 2, 0)))
int kobject_set_name_vargs(struct kobject *kobj, const char *fmt,
va_list vargs);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const char *kobject_name(const struct kobject *kobj)
{
return kobj->name;
}
extern void kobject_init(struct kobject *kobj, struct kobj_type *ktype);
extern __attribute__((format(printf, 3, 4))) __attribute__((warn_unused_result))
int kobject_add(struct kobject *kobj, struct kobject *parent,
const char *fmt, ...);
extern __attribute__((format(printf, 4, 5))) __attribute__((warn_unused_result))
int kobject_init_and_add(struct kobject *kobj,
struct kobj_type *ktype, struct kobject *parent,
const char *fmt, ...);
extern void kobject_del(struct kobject *kobj);
extern struct kobject * __attribute__((warn_unused_result)) kobject_create(void);
extern struct kobject * __attribute__((warn_unused_result)) kobject_create_and_add(const char *name,
struct kobject *parent);
extern int __attribute__((warn_unused_result)) kobject_rename(struct kobject *, const char *new_name);
extern int __attribute__((warn_unused_result)) kobject_move(struct kobject *, struct kobject *);
extern struct kobject *kobject_get(struct kobject *kobj);
extern struct kobject * __attribute__((warn_unused_result)) kobject_get_unless_zero(
struct kobject *kobj);
extern void kobject_put(struct kobject *kobj);
extern const void *kobject_namespace(struct kobject *kobj);
extern char *kobject_get_path(struct kobject *kobj, gfp_t flag);
# 130 "./include/linux/kobject.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool kobject_has_children(struct kobject *kobj)
{
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(kref_read(&kobj->kref) == 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/kobject.h", 132); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
return kobj->sd && kobj->sd->dir.subdirs;
}
struct kobj_type {
void (*release)(struct kobject *kobj);
const struct sysfs_ops *sysfs_ops;
struct attribute **default_attrs;
const struct kobj_ns_type_operations *(*child_ns_type)(struct kobject *kobj);
const void *(*namespace)(struct kobject *kobj);
};
struct kobj_uevent_env {
char *argv[3];
char *envp[32];
int envp_idx;
char buf[2048];
int buflen;
};
struct kset_uevent_ops {
int (* const filter)(struct kset *kset, struct kobject *kobj);
const char *(* const name)(struct kset *kset, struct kobject *kobj);
int (* const uevent)(struct kset *kset, struct kobject *kobj,
struct kobj_uevent_env *env);
};
struct kobj_attribute {
struct attribute attr;
ssize_t (*show)(struct kobject *kobj, struct kobj_attribute *attr,
char *buf);
ssize_t (*store)(struct kobject *kobj, struct kobj_attribute *attr,
const char *buf, size_t count);
};
extern const struct sysfs_ops kobj_sysfs_ops;
struct sock;
# 189 "./include/linux/kobject.h"
struct kset {
struct list_head list;
spinlock_t list_lock;
struct kobject kobj;
const struct kset_uevent_ops *uevent_ops;
} ;
extern void kset_init(struct kset *kset);
extern int __attribute__((warn_unused_result)) kset_register(struct kset *kset);
extern void kset_unregister(struct kset *kset);
extern struct kset * __attribute__((warn_unused_result)) kset_create_and_add(const char *name,
const struct kset_uevent_ops *u,
struct kobject *parent_kobj);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kset *to_kset(struct kobject *kobj)
{
return kobj ? ({ void *__mptr = (void *)(kobj); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(kobj)), typeof(((struct kset *)0)->kobj)) && !__builtin_types_compatible_p(typeof(*(kobj)), typeof(void)))); extern void __compiletime_assert_38(void) ; if (__cond) __compiletime_assert_38(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct kset *)(__mptr - __builtin_offsetof(struct kset, kobj))); }) : ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kset *kset_get(struct kset *k)
{
return k ? to_kset(kobject_get(&k->kobj)) : ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kset_put(struct kset *k)
{
kobject_put(&k->kobj);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kobj_type *get_ktype(struct kobject *kobj)
{
return kobj->ktype;
}
extern struct kobject *kset_find_obj(struct kset *, const char *);
extern struct kobject *kernel_kobj;
extern struct kobject *mm_kobj;
extern struct kobject *hypervisor_kobj;
extern struct kobject *power_kobj;
extern struct kobject *firmware_kobj;
int kobject_uevent(struct kobject *kobj, enum kobject_action action);
int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
char *envp[]);
int kobject_synth_uevent(struct kobject *kobj, const char *buf, size_t count);
__attribute__((format(printf, 2, 3)))
int add_uevent_var(struct kobj_uevent_env *env, const char *format, ...);
# 18 "./include/linux/device.h" 2
# 1 "./include/linux/klist.h" 1
# 19 "./include/linux/klist.h"
struct klist_node;
struct klist {
spinlock_t k_lock;
struct list_head k_list;
void (*get)(struct klist_node *);
void (*put)(struct klist_node *);
} __attribute__ ((aligned (sizeof(void *))));
# 36 "./include/linux/klist.h"
extern void klist_init(struct klist *k, void (*get)(struct klist_node *),
void (*put)(struct klist_node *));
struct klist_node {
void *n_klist;
struct list_head n_node;
struct kref n_ref;
};
extern void klist_add_tail(struct klist_node *n, struct klist *k);
extern void klist_add_head(struct klist_node *n, struct klist *k);
extern void klist_add_behind(struct klist_node *n, struct klist_node *pos);
extern void klist_add_before(struct klist_node *n, struct klist_node *pos);
extern void klist_del(struct klist_node *n);
extern void klist_remove(struct klist_node *n);
extern int klist_node_attached(struct klist_node *n);
struct klist_iter {
struct klist *i_klist;
struct klist_node *i_cur;
};
extern void klist_iter_init(struct klist *k, struct klist_iter *i);
extern void klist_iter_init_node(struct klist *k, struct klist_iter *i,
struct klist_node *n);
extern void klist_iter_exit(struct klist_iter *i);
extern struct klist_node *klist_prev(struct klist_iter *i);
extern struct klist_node *klist_next(struct klist_iter *i);
# 19 "./include/linux/device.h" 2
# 1 "./include/linux/pinctrl/devinfo.h" 1
# 48 "./include/linux/pinctrl/devinfo.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pinctrl_bind_pins(struct device *dev)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pinctrl_init_done(struct device *dev)
{
return 0;
}
# 25 "./include/linux/device.h" 2
# 1 "./include/linux/pm.h" 1
# 30 "./include/linux/pm.h"
# 1 "./include/linux/seq_file.h" 1
# 11 "./include/linux/seq_file.h"
# 1 "./include/linux/fs.h" 1
# 1 "./include/linux/wait_bit.h" 1
# 10 "./include/linux/wait_bit.h"
struct wait_bit_key {
void *flags;
int bit_nr;
unsigned long timeout;
};
struct wait_bit_queue_entry {
struct wait_bit_key key;
struct wait_queue_entry wq_entry;
};
typedef int wait_bit_action_f(struct wait_bit_key *key, int mode);
void __wake_up_bit(struct wait_queue_head *wq_head, void *word, int bit);
int __wait_on_bit(struct wait_queue_head *wq_head, struct wait_bit_queue_entry *wbq_entry, wait_bit_action_f *action, unsigned int mode);
int __wait_on_bit_lock(struct wait_queue_head *wq_head, struct wait_bit_queue_entry *wbq_entry, wait_bit_action_f *action, unsigned int mode);
void wake_up_bit(void *word, int bit);
void wake_up_atomic_t(atomic_t *p);
int out_of_line_wait_on_bit(void *word, int, wait_bit_action_f *action, unsigned int mode);
int out_of_line_wait_on_bit_timeout(void *word, int, wait_bit_action_f *action, unsigned int mode, unsigned long timeout);
int out_of_line_wait_on_bit_lock(void *word, int, wait_bit_action_f *action, unsigned int mode);
int out_of_line_wait_on_atomic_t(atomic_t *p, int (*)(atomic_t *), unsigned int mode);
struct wait_queue_head *bit_waitqueue(void *word, int bit);
extern void __attribute__ ((__section__(".init.text"))) wait_bit_init(void);
int wake_bit_function(struct wait_queue_entry *wq_entry, unsigned mode, int sync, void *key);
# 54 "./include/linux/wait_bit.h"
extern int bit_wait(struct wait_bit_key *key, int bit);
extern int bit_wait_io(struct wait_bit_key *key, int bit);
extern int bit_wait_timeout(struct wait_bit_key *key, int bit);
extern int bit_wait_io_timeout(struct wait_bit_key *key, int bit);
# 75 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
wait_on_bit(unsigned long *word, int bit, unsigned mode)
{
do { do { } while (0); } while (0);
if (!test_bit(bit, word))
return 0;
return out_of_line_wait_on_bit(word, bit,
bit_wait,
mode);
}
# 100 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
wait_on_bit_io(unsigned long *word, int bit, unsigned mode)
{
do { do { } while (0); } while (0);
if (!test_bit(bit, word))
return 0;
return out_of_line_wait_on_bit(word, bit,
bit_wait_io,
mode);
}
# 126 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
wait_on_bit_timeout(unsigned long *word, int bit, unsigned mode,
unsigned long timeout)
{
do { do { } while (0); } while (0);
if (!test_bit(bit, word))
return 0;
return out_of_line_wait_on_bit_timeout(word, bit,
bit_wait_timeout,
mode, timeout);
}
# 154 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
wait_on_bit_action(unsigned long *word, int bit, wait_bit_action_f *action,
unsigned mode)
{
do { do { } while (0); } while (0);
if (!test_bit(bit, word))
return 0;
return out_of_line_wait_on_bit(word, bit, action, mode);
}
# 183 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
wait_on_bit_lock(unsigned long *word, int bit, unsigned mode)
{
do { do { } while (0); } while (0);
if (!_test_and_set_bit(bit,word))
return 0;
return out_of_line_wait_on_bit_lock(word, bit, bit_wait, mode);
}
# 207 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
wait_on_bit_lock_io(unsigned long *word, int bit, unsigned mode)
{
do { do { } while (0); } while (0);
if (!_test_and_set_bit(bit,word))
return 0;
return out_of_line_wait_on_bit_lock(word, bit, bit_wait_io, mode);
}
# 233 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
wait_on_bit_lock_action(unsigned long *word, int bit, wait_bit_action_f *action,
unsigned mode)
{
do { do { } while (0); } while (0);
if (!_test_and_set_bit(bit,word))
return 0;
return out_of_line_wait_on_bit_lock(word, bit, action, mode);
}
# 253 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
int wait_on_atomic_t(atomic_t *val, int (*action)(atomic_t *), unsigned mode)
{
do { do { } while (0); } while (0);
if (({ union { typeof((val)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((val)->counter), __u.__c, sizeof((val)->counter)); else __read_once_size_nocheck(&((val)->counter), __u.__c, sizeof((val)->counter)); do { } while (0); __u.__val; }) == 0)
return 0;
return out_of_line_wait_on_atomic_t(val, action, mode);
}
# 271 "./include/linux/wait_bit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_and_wake_up_bit(int bit, void *word)
{
do { __asm__ __volatile__ ("dmb " "ish" : : : "memory"); _clear_bit(bit,word); } while (0);
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
wake_up_bit(word, bit);
}
# 7 "./include/linux/fs.h" 2
# 1 "./include/linux/dcache.h" 1
# 1 "./include/linux/rculist_bl.h" 1
# 1 "./include/linux/list_bl.h" 1
# 1 "./include/linux/bit_spinlock.h" 1
# 16 "./include/linux/bit_spinlock.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bit_spin_lock(int bitnum, unsigned long *addr)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
while (__builtin_expect(!!(_test_and_set_bit(bitnum,addr)), 0)) {
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
do {
__asm__ __volatile__("": : :"memory");
} while (test_bit(bitnum, addr));
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
}
(void)0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bit_spin_trylock(int bitnum, unsigned long *addr)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
if (__builtin_expect(!!(_test_and_set_bit(bitnum,addr)), 0)) {
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
return 0;
}
(void)0;
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bit_spin_unlock(int bitnum, unsigned long *addr)
{
do { __asm__ __volatile__ ("dmb " "ish" : : : "memory"); _clear_bit(bitnum,addr); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
(void)0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __bit_spin_unlock(int bitnum, unsigned long *addr)
{
do { __asm__ __volatile__ ("dmb " "ish" : : : "memory"); _clear_bit(bitnum,addr); } while (0);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
(void)0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bit_spin_is_locked(int bitnum, unsigned long *addr)
{
return test_bit(bitnum, addr);
}
# 7 "./include/linux/list_bl.h" 2
# 34 "./include/linux/list_bl.h"
struct hlist_bl_head {
struct hlist_bl_node *first;
};
struct hlist_bl_node {
struct hlist_bl_node *next, **pprev;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void INIT_HLIST_BL_NODE(struct hlist_bl_node *h)
{
h->next = ((void *)0);
h->pprev = ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool hlist_bl_unhashed(const struct hlist_bl_node *h)
{
return !h->pprev;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct hlist_bl_node *hlist_bl_first(struct hlist_bl_head *h)
{
return (struct hlist_bl_node *)
((unsigned long)h->first & ~1UL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_set_first(struct hlist_bl_head *h,
struct hlist_bl_node *n)
{
do { if (__builtin_expect(!!((unsigned long)n & 1UL), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/list_bl.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "66" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
do { if (__builtin_expect(!!(((unsigned long)h->first & 1UL) != 1UL), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/list_bl.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "68" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
h->first = (struct hlist_bl_node *)((unsigned long)n | 1UL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool hlist_bl_empty(const struct hlist_bl_head *h)
{
return !((unsigned long)({ union { typeof(h->first) __val; char __c[1]; } __u; if (1) __read_once_size(&(h->first), __u.__c, sizeof(h->first)); else __read_once_size_nocheck(&(h->first), __u.__c, sizeof(h->first)); do { } while (0); __u.__val; }) & ~1UL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_add_head(struct hlist_bl_node *n,
struct hlist_bl_head *h)
{
struct hlist_bl_node *first = hlist_bl_first(h);
n->next = first;
if (first)
first->pprev = &n->next;
n->pprev = &h->first;
hlist_bl_set_first(h, n);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __hlist_bl_del(struct hlist_bl_node *n)
{
struct hlist_bl_node *next = n->next;
struct hlist_bl_node **pprev = n->pprev;
do { if (__builtin_expect(!!((unsigned long)n & 1UL), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/list_bl.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "94" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
({ union { typeof(*pprev) __val; char __c[1]; } __u = { .__val = ( typeof(*pprev)) ((struct hlist_bl_node *) ((unsigned long)next | ((unsigned long)*pprev & 1UL))) }; __write_once_size(&(*pprev), __u.__c, sizeof(*pprev)); __u.__val; });
if (next)
next->pprev = pprev;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_del(struct hlist_bl_node *n)
{
__hlist_bl_del(n);
n->next = ((void *) 0x100 + 0);
n->pprev = ((void *) 0x200 + 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_del_init(struct hlist_bl_node *n)
{
if (!hlist_bl_unhashed(n)) {
__hlist_bl_del(n);
INIT_HLIST_BL_NODE(n);
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_lock(struct hlist_bl_head *b)
{
bit_spin_lock(0, (unsigned long *)b);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_unlock(struct hlist_bl_head *b)
{
__bit_spin_unlock(0, (unsigned long *)b);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool hlist_bl_is_locked(struct hlist_bl_head *b)
{
return bit_spin_is_locked(0, (unsigned long *)b);
}
# 9 "./include/linux/rculist_bl.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_set_first_rcu(struct hlist_bl_head *h,
struct hlist_bl_node *n)
{
do { if (__builtin_expect(!!((unsigned long)n & 1UL), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/rculist_bl.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "14" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
do { if (__builtin_expect(!!(((unsigned long)h->first & 1UL) != 1UL), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/rculist_bl.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "16" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
({ uintptr_t _r_a_p__v = (uintptr_t)((struct hlist_bl_node *)((unsigned long)n | 1UL)); if (__builtin_constant_p((struct hlist_bl_node *)((unsigned long)n | 1UL)) && (_r_a_p__v) == (uintptr_t)((void *)0)) ({ union { typeof((h->first)) __val; char __c[1]; } __u = { .__val = ( typeof((h->first))) ((typeof(h->first))(_r_a_p__v)) }; __write_once_size(&((h->first)), __u.__c, sizeof((h->first))); __u.__val; }); else do { do { bool __cond = !((sizeof(*&h->first) == sizeof(char) || sizeof(*&h->first) == sizeof(short) || sizeof(*&h->first) == sizeof(int) || sizeof(*&h->first) == sizeof(long))); extern void __compiletime_assert_39(void) ; if (__cond) __compiletime_assert_39(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ({ union { typeof(*&h->first) __val; char __c[1]; } __u = { .__val = ( typeof(*&h->first)) ((typeof(*((typeof(h->first))_r_a_p__v)) *)((typeof(h->first))_r_a_p__v)) }; __write_once_size(&(*&h->first), __u.__c, sizeof(*&h->first)); __u.__val; }); } while (0); _r_a_p__v; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct hlist_bl_node *hlist_bl_first_rcu(struct hlist_bl_head *h)
{
return (struct hlist_bl_node *)
((unsigned long)({ typeof(*(h->first)) *________p1 = (typeof(*(h->first)) *)({ union { typeof((h->first)) __val; char __c[1]; } __u; if (1) __read_once_size(&((h->first)), __u.__c, sizeof((h->first))); else __read_once_size_nocheck(&((h->first)), __u.__c, sizeof((h->first))); do { } while (0); __u.__val; }); do { } while (0); ; ((typeof(*(h->first)) *)(________p1)); }) & ~1UL);
}
# 47 "./include/linux/rculist_bl.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_del_init_rcu(struct hlist_bl_node *n)
{
if (!hlist_bl_unhashed(n)) {
__hlist_bl_del(n);
n->pprev = ((void *)0);
}
}
# 74 "./include/linux/rculist_bl.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_del_rcu(struct hlist_bl_node *n)
{
__hlist_bl_del(n);
n->pprev = ((void *) 0x200 + 0);
}
# 99 "./include/linux/rculist_bl.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hlist_bl_add_head_rcu(struct hlist_bl_node *n,
struct hlist_bl_head *h)
{
struct hlist_bl_node *first;
first = hlist_bl_first(h);
n->next = first;
if (first)
first->pprev = &n->next;
n->pprev = &h->first;
hlist_bl_set_first_rcu(h, n);
}
# 9 "./include/linux/dcache.h" 2
# 1 "./include/linux/lockref.h" 1
# 25 "./include/linux/lockref.h"
struct lockref {
union {
__u64 __attribute__((aligned(8))) lock_count;
struct {
spinlock_t lock;
int count;
};
};
};
extern void lockref_get(struct lockref *);
extern int lockref_put_return(struct lockref *);
extern int lockref_get_not_zero(struct lockref *);
extern int lockref_get_or_lock(struct lockref *);
extern int lockref_put_or_lock(struct lockref *);
extern void lockref_mark_dead(struct lockref *);
extern int lockref_get_not_dead(struct lockref *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __lockref_is_dead(const struct lockref *l)
{
return ((int)l->count < 0);
}
# 14 "./include/linux/dcache.h" 2
# 1 "./include/linux/stringhash.h" 1
# 1 "./include/linux/hash.h" 1
# 60 "./include/linux/hash.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 __hash_32_generic(u32 val)
{
return val * 0x61C88647;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 hash_32_generic(u32 val, unsigned int bits)
{
return __hash_32_generic(val) >> (32 - bits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) u32 hash_64_generic(u64 val, unsigned int bits)
{
return hash_32_generic((u32)val ^ __hash_32_generic(val >> 32), bits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 hash_ptr(const void *ptr, unsigned int bits)
{
return hash_32_generic((unsigned long)ptr, bits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 hash32_ptr(const void *ptr)
{
unsigned long val = (unsigned long)ptr;
return (u32)val;
}
# 8 "./include/linux/stringhash.h" 2
# 42 "./include/linux/stringhash.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
partial_name_hash(unsigned long c, unsigned long prevhash)
{
return (prevhash + (c << 4) + (c >> 4)) * 11;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int end_name_hash(unsigned long hash)
{
return hash_32_generic(hash, 32);
}
# 66 "./include/linux/stringhash.h"
extern unsigned int __attribute__((pure)) full_name_hash(const void *salt, const char *, unsigned int);
# 77 "./include/linux/stringhash.h"
extern u64 __attribute__((pure)) hashlen_string(const void *salt, const char *name);
# 15 "./include/linux/dcache.h" 2
struct path;
struct vfsmount;
# 47 "./include/linux/dcache.h"
struct qstr {
union {
struct {
u32 hash; u32 len;
};
u64 hash_len;
};
const unsigned char *name;
};
extern const char empty_string[];
extern const struct qstr empty_name;
extern const char slash_string[];
extern const struct qstr slash_name;
struct dentry_stat_t {
long nr_dentry;
long nr_unused;
long age_limit;
long want_pages;
long dummy[2];
};
extern struct dentry_stat_t dentry_stat;
# 90 "./include/linux/dcache.h"
struct dentry {
unsigned int d_flags;
seqcount_t d_seq;
struct hlist_bl_node d_hash;
struct dentry *d_parent;
struct qstr d_name;
struct inode *d_inode;
unsigned char d_iname[36];
struct lockref d_lockref;
const struct dentry_operations *d_op;
struct super_block *d_sb;
unsigned long d_time;
void *d_fsdata;
union {
struct list_head d_lru;
wait_queue_head_t *d_wait;
};
struct list_head d_child;
struct list_head d_subdirs;
union {
struct hlist_node d_alias;
struct hlist_bl_node d_in_lookup_hash;
struct callback_head d_rcu;
} d_u;
} ;
enum dentry_d_lock_class
{
DENTRY_D_LOCK_NORMAL,
DENTRY_D_LOCK_NESTED
};
struct dentry_operations {
int (*d_revalidate)(struct dentry *, unsigned int);
int (*d_weak_revalidate)(struct dentry *, unsigned int);
int (*d_hash)(const struct dentry *, struct qstr *);
int (*d_compare)(const struct dentry *,
unsigned int, const char *, const struct qstr *);
int (*d_delete)(const struct dentry *);
int (*d_init)(struct dentry *);
void (*d_release)(struct dentry *);
void (*d_prune)(struct dentry *);
void (*d_iput)(struct dentry *, struct inode *);
char *(*d_dname)(struct dentry *, char *, int);
struct vfsmount *(*d_automount)(struct path *);
int (*d_manage)(const struct path *, bool);
struct dentry *(*d_real)(struct dentry *, const struct inode *,
unsigned int, unsigned int);
void (*d_canonical_path)(const struct path *, struct path *);
} __attribute__((__aligned__((1 << 6))));
# 224 "./include/linux/dcache.h"
extern seqlock_t rename_lock;
extern void d_instantiate(struct dentry *, struct inode *);
extern void d_instantiate_new(struct dentry *, struct inode *);
extern struct dentry * d_instantiate_unique(struct dentry *, struct inode *);
extern int d_instantiate_no_diralias(struct dentry *, struct inode *);
extern void __d_drop(struct dentry *dentry);
extern void d_drop(struct dentry *dentry);
extern void d_delete(struct dentry *);
extern void d_set_d_op(struct dentry *dentry, const struct dentry_operations *op);
extern struct dentry * d_alloc(struct dentry *, const struct qstr *);
extern struct dentry * d_alloc_pseudo(struct super_block *, const struct qstr *);
extern struct dentry * d_alloc_parallel(struct dentry *, const struct qstr *,
wait_queue_head_t *);
extern struct dentry * d_splice_alias(struct inode *, struct dentry *);
extern struct dentry * d_add_ci(struct dentry *, struct inode *, struct qstr *);
extern struct dentry * d_exact_alias(struct dentry *, struct inode *);
extern struct dentry *d_find_any_alias(struct inode *inode);
extern struct dentry * d_obtain_alias(struct inode *);
extern struct dentry * d_obtain_root(struct inode *);
extern void shrink_dcache_sb(struct super_block *);
extern void shrink_dcache_parent(struct dentry *);
extern void shrink_dcache_for_umount(struct super_block *);
extern void d_invalidate(struct dentry *);
extern struct dentry * d_make_root(struct inode *);
extern void d_genocide(struct dentry *);
extern void d_tmpfile(struct dentry *, struct inode *);
extern struct dentry *d_find_alias(struct inode *);
extern void d_prune_aliases(struct inode *);
extern int path_has_submounts(const struct path *);
extern void d_rehash(struct dentry *);
extern void d_add(struct dentry *, struct inode *);
extern void dentry_update_name_case(struct dentry *, const struct qstr *);
extern void d_move(struct dentry *, struct dentry *);
extern void d_exchange(struct dentry *, struct dentry *);
extern struct dentry *d_ancestor(struct dentry *, struct dentry *);
extern struct dentry *d_lookup(const struct dentry *, const struct qstr *);
extern struct dentry *d_hash_and_lookup(struct dentry *, struct qstr *);
extern struct dentry *__d_lookup(const struct dentry *, const struct qstr *);
extern struct dentry *__d_lookup_rcu(const struct dentry *parent,
const struct qstr *name, unsigned *seq);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned d_count(const struct dentry *dentry)
{
return dentry->d_lockref.count;
}
extern __attribute__((format(printf, 4, 5)))
char *dynamic_dname(struct dentry *, char *, int, const char *, ...);
extern char *simple_dname(struct dentry *, char *, int);
extern char *__d_path(const struct path *, const struct path *, char *, int);
extern char *d_absolute_path(const struct path *, char *, int);
extern char *d_path(const struct path *, char *, int);
extern char *dentry_path_raw(struct dentry *, char *, int);
extern char *dentry_path(struct dentry *, char *, int);
# 317 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dentry *dget_dlock(struct dentry *dentry)
{
if (dentry)
dentry->d_lockref.count++;
return dentry;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dentry *dget(struct dentry *dentry)
{
if (dentry)
lockref_get(&dentry->d_lockref);
return dentry;
}
extern struct dentry *dget_parent(struct dentry *dentry);
# 340 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int d_unhashed(const struct dentry *dentry)
{
return hlist_bl_unhashed(&dentry->d_hash);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int d_unlinked(const struct dentry *dentry)
{
return d_unhashed(dentry) && !((dentry) == (dentry)->d_parent);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cant_mount(const struct dentry *dentry)
{
return (dentry->d_flags & 0x00000100);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dont_mount(struct dentry *dentry)
{
spin_lock(&dentry->d_lockref.lock);
dentry->d_flags |= 0x00000100;
spin_unlock(&dentry->d_lockref.lock);
}
extern void __d_lookup_done(struct dentry *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int d_in_lookup(struct dentry *dentry)
{
return dentry->d_flags & 0x10000000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void d_lookup_done(struct dentry *dentry)
{
if (__builtin_expect(!!(d_in_lookup(dentry)), 0)) {
spin_lock(&dentry->d_lockref.lock);
__d_lookup_done(dentry);
spin_unlock(&dentry->d_lockref.lock);
}
}
extern void dput(struct dentry *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_managed(const struct dentry *dentry)
{
return dentry->d_flags & (0x00010000|0x00020000|0x00040000);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_mountpoint(const struct dentry *dentry)
{
return dentry->d_flags & 0x00010000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned __d_entry_type(const struct dentry *dentry)
{
return dentry->d_flags & 0x00700000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_miss(const struct dentry *dentry)
{
return __d_entry_type(dentry) == 0x00000000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_whiteout(const struct dentry *dentry)
{
return __d_entry_type(dentry) == 0x00100000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_can_lookup(const struct dentry *dentry)
{
return __d_entry_type(dentry) == 0x00200000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_autodir(const struct dentry *dentry)
{
return __d_entry_type(dentry) == 0x00300000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_dir(const struct dentry *dentry)
{
return d_can_lookup(dentry) || d_is_autodir(dentry);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_symlink(const struct dentry *dentry)
{
return __d_entry_type(dentry) == 0x00600000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_reg(const struct dentry *dentry)
{
return __d_entry_type(dentry) == 0x00400000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_special(const struct dentry *dentry)
{
return __d_entry_type(dentry) == 0x00500000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_file(const struct dentry *dentry)
{
return d_is_reg(dentry) || d_is_special(dentry);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_negative(const struct dentry *dentry)
{
return d_is_miss(dentry);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_positive(const struct dentry *dentry)
{
return !d_is_negative(dentry);
}
# 469 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_really_is_negative(const struct dentry *dentry)
{
return dentry->d_inode == ((void *)0);
}
# 487 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_really_is_positive(const struct dentry *dentry)
{
return dentry->d_inode != ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int simple_positive(struct dentry *dentry)
{
return d_really_is_positive(dentry) && !d_unhashed(dentry);
}
extern void d_set_fallthru(struct dentry *dentry);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool d_is_fallthru(const struct dentry *dentry)
{
return dentry->d_flags & 0x01000000;
}
extern int sysctl_vfs_cache_pressure;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long vfs_pressure_ratio(unsigned long val)
{
return ( { typeof(val) quot = (val) / (100); typeof(val) rem = (val) % (100); (quot * (sysctl_vfs_cache_pressure)) + ((rem * (sysctl_vfs_cache_pressure)) / (100)); } );
}
# 519 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct inode *d_inode(const struct dentry *dentry)
{
return dentry->d_inode;
}
# 531 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct inode *d_inode_rcu(const struct dentry *dentry)
{
return (*({ __attribute__((unused)) typeof(dentry->d_inode) __var = ( typeof(dentry->d_inode)) 0; (volatile typeof(dentry->d_inode) *)&(dentry->d_inode); }));
}
# 546 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct inode *d_backing_inode(const struct dentry *upper)
{
struct inode *inode = upper->d_inode;
return inode;
}
# 563 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dentry *d_backing_dentry(struct dentry *upper)
{
return upper;
}
# 583 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dentry *d_real(struct dentry *dentry,
const struct inode *inode,
unsigned int open_flags, unsigned int flags)
{
if (__builtin_expect(!!(dentry->d_flags & 0x04000000), 0))
return dentry->d_op->d_real(dentry, inode, open_flags, flags);
else
return dentry;
}
# 600 "./include/linux/dcache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct inode *d_real_inode(const struct dentry *dentry)
{
return d_backing_inode(d_real((struct dentry *) dentry, ((void *)0), 0, 0));
}
struct name_snapshot {
const unsigned char *name;
unsigned char inline_name[36];
};
void take_dentry_name_snapshot(struct name_snapshot *, struct dentry *);
void release_dentry_name_snapshot(struct name_snapshot *);
# 9 "./include/linux/fs.h" 2
# 1 "./include/linux/path.h" 1
struct dentry;
struct vfsmount;
struct path {
struct vfsmount *mnt;
struct dentry *dentry;
} ;
extern void path_get(const struct path *);
extern void path_put(const struct path *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int path_equal(const struct path *path1, const struct path *path2)
{
return path1->mnt == path2->mnt && path1->dentry == path2->dentry;
}
# 10 "./include/linux/fs.h" 2
# 1 "./include/linux/list_lru.h" 1
# 13 "./include/linux/list_lru.h"
# 1 "./include/linux/shrinker.h" 1
# 12 "./include/linux/shrinker.h"
struct shrink_control {
gfp_t gfp_mask;
unsigned long nr_to_scan;
unsigned long nr_scanned;
int nid;
struct mem_cgroup *memcg;
};
# 57 "./include/linux/shrinker.h"
struct shrinker {
unsigned long (*count_objects)(struct shrinker *,
struct shrink_control *sc);
unsigned long (*scan_objects)(struct shrinker *,
struct shrink_control *sc);
int seeks;
long batch;
unsigned long flags;
struct list_head list;
atomic_long_t *nr_deferred;
};
extern int register_shrinker(struct shrinker *);
extern void unregister_shrinker(struct shrinker *);
# 14 "./include/linux/list_lru.h" 2
struct mem_cgroup;
enum lru_status {
LRU_REMOVED,
LRU_REMOVED_RETRY,
LRU_ROTATE,
LRU_SKIP,
LRU_RETRY,
};
struct list_lru_one {
struct list_head list;
long nr_items;
};
struct list_lru_memcg {
struct list_lru_one *lru[0];
};
struct list_lru_node {
spinlock_t lock;
struct list_lru_one lru;
struct list_lru_memcg *memcg_lrus;
long nr_items;
} __attribute__((__aligned__((1 << 6))));
struct list_lru {
struct list_lru_node *node;
struct list_head list;
bool memcg_aware;
};
void list_lru_destroy(struct list_lru *lru);
int __list_lru_init(struct list_lru *lru, bool memcg_aware,
struct lock_class_key *key);
int memcg_update_all_list_lrus(int num_memcgs);
void memcg_drain_all_list_lrus(int src_idx, int dst_idx);
# 86 "./include/linux/list_lru.h"
bool list_lru_add(struct list_lru *lru, struct list_head *item);
# 99 "./include/linux/list_lru.h"
bool list_lru_del(struct list_lru *lru, struct list_head *item);
# 111 "./include/linux/list_lru.h"
unsigned long list_lru_count_one(struct list_lru *lru,
int nid, struct mem_cgroup *memcg);
unsigned long list_lru_count_node(struct list_lru *lru, int nid);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long list_lru_shrink_count(struct list_lru *lru,
struct shrink_control *sc)
{
return list_lru_count_one(lru, sc->nid, sc->memcg);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long list_lru_count(struct list_lru *lru)
{
long count = 0;
int nid;
for ( (nid) = 0; (nid) == 0; (nid) = 1)
count += list_lru_count_node(lru, nid);
return count;
}
void list_lru_isolate(struct list_lru_one *list, struct list_head *item);
void list_lru_isolate_move(struct list_lru_one *list, struct list_head *item,
struct list_head *head);
typedef enum lru_status (*list_lru_walk_cb)(struct list_head *item,
struct list_lru_one *list, spinlock_t *lock, void *cb_arg);
# 161 "./include/linux/list_lru.h"
unsigned long list_lru_walk_one(struct list_lru *lru,
int nid, struct mem_cgroup *memcg,
list_lru_walk_cb isolate, void *cb_arg,
unsigned long *nr_to_walk);
unsigned long list_lru_walk_node(struct list_lru *lru, int nid,
list_lru_walk_cb isolate, void *cb_arg,
unsigned long *nr_to_walk);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
list_lru_shrink_walk(struct list_lru *lru, struct shrink_control *sc,
list_lru_walk_cb isolate, void *cb_arg)
{
return list_lru_walk_one(lru, sc->nid, sc->memcg, isolate, cb_arg,
&sc->nr_to_scan);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
list_lru_walk(struct list_lru *lru, list_lru_walk_cb isolate,
void *cb_arg, unsigned long nr_to_walk)
{
long isolated = 0;
int nid;
for ( (nid) = 0; (nid) == 0; (nid) = 1) {
isolated += list_lru_walk_node(lru, nid, isolate,
cb_arg, &nr_to_walk);
if (nr_to_walk <= 0)
break;
}
return isolated;
}
# 14 "./include/linux/fs.h" 2
# 1 "./include/linux/mm_types.h" 1
# 1 "./include/linux/auxvec.h" 1
# 1 "./include/uapi/linux/auxvec.h" 1
# 1 "./arch/arm/include/asm/auxvec.h" 1
# 1 "./arch/arm/include/uapi/asm/auxvec.h" 1
# 2 "./arch/arm/include/asm/auxvec.h" 2
# 6 "./include/uapi/linux/auxvec.h" 2
# 6 "./include/linux/auxvec.h" 2
# 8 "./include/linux/mm_types.h" 2
# 1 "./include/linux/uprobes.h" 1
# 32 "./include/linux/uprobes.h"
struct vm_area_struct;
struct mm_struct;
struct inode;
struct notifier_block;
struct page;
enum uprobe_filter_ctx {
UPROBE_FILTER_REGISTER,
UPROBE_FILTER_UNREGISTER,
UPROBE_FILTER_MMAP,
};
struct uprobe_consumer {
int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
int (*ret_handler)(struct uprobe_consumer *self,
unsigned long func,
struct pt_regs *regs);
bool (*filter)(struct uprobe_consumer *self,
enum uprobe_filter_ctx ctx,
struct mm_struct *mm);
struct uprobe_consumer *next;
};
# 1 "./arch/arm/include/asm/uprobes.h" 1
# 12 "./arch/arm/include/asm/uprobes.h"
# 1 "./arch/arm/include/asm/probes.h" 1
# 24 "./arch/arm/include/asm/probes.h"
typedef u32 probes_opcode_t;
struct arch_probes_insn;
typedef void (probes_insn_handler_t)(probes_opcode_t,
struct arch_probes_insn *,
struct pt_regs *);
typedef unsigned long (probes_check_cc)(unsigned long);
typedef void (probes_insn_singlestep_t)(probes_opcode_t,
struct arch_probes_insn *,
struct pt_regs *);
typedef void (probes_insn_fn_t)(void);
struct arch_probes_insn {
probes_opcode_t *insn;
probes_insn_handler_t *insn_handler;
probes_check_cc *insn_check_cc;
probes_insn_singlestep_t *insn_singlestep;
probes_insn_fn_t *insn_fn;
int stack_space;
unsigned long register_usage_flags;
bool kprobe_direct_exec;
};
# 13 "./arch/arm/include/asm/uprobes.h" 2
typedef u32 uprobe_opcode_t;
# 25 "./arch/arm/include/asm/uprobes.h"
struct arch_uprobe_task {
u32 backup;
unsigned long saved_trap_no;
};
struct arch_uprobe {
u8 insn[4];
unsigned long ixol[2];
uprobe_opcode_t bpinsn;
bool simulate;
u32 pcreg;
void (*prehandler)(struct arch_uprobe *auprobe,
struct arch_uprobe_task *autask,
struct pt_regs *regs);
void (*posthandler)(struct arch_uprobe *auprobe,
struct arch_uprobe_task *autask,
struct pt_regs *regs);
struct arch_probes_insn asi;
};
# 63 "./include/linux/uprobes.h" 2
enum uprobe_task_state {
UTASK_RUNNING,
UTASK_SSTEP,
UTASK_SSTEP_ACK,
UTASK_SSTEP_TRAPPED,
};
struct uprobe_task {
enum uprobe_task_state state;
union {
struct {
struct arch_uprobe_task autask;
unsigned long vaddr;
};
struct {
struct callback_head dup_xol_work;
unsigned long dup_xol_addr;
};
};
struct uprobe *active_uprobe;
unsigned long xol_vaddr;
struct return_instance *return_instances;
unsigned int depth;
};
struct return_instance {
struct uprobe *uprobe;
unsigned long func;
unsigned long stack;
unsigned long orig_ret_vaddr;
bool chained;
struct return_instance *next;
};
enum rp_check {
RP_CHECK_CALL,
RP_CHECK_CHAIN_CALL,
RP_CHECK_RET,
};
struct xol_area;
struct uprobes_state {
struct xol_area *xol_area;
};
extern int set_swbp(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
extern int set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
extern bool is_swbp_insn(uprobe_opcode_t *insn);
extern bool is_trap_insn(uprobe_opcode_t *insn);
extern unsigned long uprobe_get_swbp_addr(struct pt_regs *regs);
extern unsigned long uprobe_get_trap_addr(struct pt_regs *regs);
extern int uprobe_write_opcode(struct mm_struct *mm, unsigned long vaddr, uprobe_opcode_t);
extern int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
extern int uprobe_apply(struct inode *inode, loff_t offset, struct uprobe_consumer *uc, bool);
extern void uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
extern int uprobe_mmap(struct vm_area_struct *vma);
extern void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end);
extern void uprobe_start_dup_mmap(void);
extern void uprobe_end_dup_mmap(void);
extern void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm);
extern void uprobe_free_utask(struct task_struct *t);
extern void uprobe_copy_process(struct task_struct *t, unsigned long flags);
extern int uprobe_post_sstep_notifier(struct pt_regs *regs);
extern int uprobe_pre_sstep_notifier(struct pt_regs *regs);
extern void uprobe_notify_resume(struct pt_regs *regs);
extern bool uprobe_deny_signal(void);
extern bool arch_uprobe_skip_sstep(struct arch_uprobe *aup, struct pt_regs *regs);
extern void uprobe_clear_state(struct mm_struct *mm);
extern int arch_uprobe_analyze_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long addr);
extern int arch_uprobe_pre_xol(struct arch_uprobe *aup, struct pt_regs *regs);
extern int arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs);
extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk);
extern int arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data);
extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, struct pt_regs *regs);
extern unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs);
extern bool arch_uretprobe_is_alive(struct return_instance *ret, enum rp_check ctx, struct pt_regs *regs);
extern bool arch_uprobe_ignore(struct arch_uprobe *aup, struct pt_regs *regs);
extern void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
void *src, unsigned long len);
# 15 "./include/linux/mm_types.h" 2
# 1 "./arch/arm/include/asm/mmu.h" 1
typedef struct {
atomic64_t id;
unsigned int vmalloc_seq;
unsigned long sigpage;
unsigned long vdso;
} mm_context_t;
# 19 "./include/linux/mm_types.h" 2
struct address_space;
struct mem_cgroup;
struct hmm;
# 42 "./include/linux/mm_types.h"
struct page {
unsigned long flags;
union {
struct address_space *mapping;
void *s_mem;
atomic_t compound_mapcount;
};
union {
unsigned long index;
void *freelist;
};
union {
# 77 "./include/linux/mm_types.h"
unsigned counters;
struct {
union {
# 91 "./include/linux/mm_types.h"
atomic_t _mapcount;
unsigned int active;
struct {
unsigned inuse:16;
unsigned objects:15;
unsigned frozen:1;
};
int units;
};
atomic_t _refcount;
};
};
# 116 "./include/linux/mm_types.h"
union {
struct list_head lru;
struct dev_pagemap *pgmap;
struct {
struct page *next;
short int pages;
short int pobjects;
};
struct callback_head callback_head;
struct {
unsigned long compound_head;
# 156 "./include/linux/mm_types.h"
unsigned short int compound_dtor;
unsigned short int compound_order;
};
# 170 "./include/linux/mm_types.h"
};
union {
unsigned long private;
# 185 "./include/linux/mm_types.h"
spinlock_t ptl;
struct kmem_cache *slab_cache;
};
struct mem_cgroup *mem_cgroup;
# 213 "./include/linux/mm_types.h"
}
;
struct page_frag_cache {
void * va;
__u16 offset;
__u16 size;
unsigned int pagecnt_bias;
bool pfmemalloc;
};
typedef unsigned long vm_flags_t;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) atomic_t *compound_mapcount_ptr(struct page *page)
{
return &page[1].compound_mapcount;
}
struct vm_region {
struct rb_node vm_rb;
vm_flags_t vm_flags;
unsigned long vm_start;
unsigned long vm_end;
unsigned long vm_top;
unsigned long vm_pgoff;
struct file *vm_file;
int vm_usage;
bool vm_icache_flushed : 1;
};
# 274 "./include/linux/mm_types.h"
struct vm_userfaultfd_ctx {};
# 283 "./include/linux/mm_types.h"
struct vm_area_struct {
unsigned long vm_start;
unsigned long vm_end;
struct vm_area_struct *vm_next, *vm_prev;
struct rb_node vm_rb;
unsigned long rb_subtree_gap;
struct mm_struct *vm_mm;
pgprot_t vm_page_prot;
unsigned long vm_flags;
# 317 "./include/linux/mm_types.h"
union {
struct {
struct rb_node rb;
unsigned long rb_subtree_last;
} shared;
const char *anon_name;
};
struct list_head anon_vma_chain;
struct anon_vma *anon_vma;
const struct vm_operations_struct *vm_ops;
unsigned long vm_pgoff;
struct file * vm_file;
void * vm_private_data;
atomic_long_t swap_readahead_info;
struct vm_userfaultfd_ctx vm_userfaultfd_ctx;
} ;
struct core_thread {
struct task_struct *task;
struct core_thread *next;
};
struct core_state {
atomic_t nr_threads;
struct core_thread dumper;
struct completion startup;
};
struct kioctx_table;
struct mm_struct {
struct vm_area_struct *mmap;
struct rb_root mm_rb;
u64 vmacache_seqnum;
unsigned long (*get_unmapped_area) (struct file *filp,
unsigned long addr, unsigned long len,
unsigned long pgoff, unsigned long flags);
unsigned long mmap_base;
unsigned long mmap_legacy_base;
unsigned long task_size;
unsigned long highest_vm_end;
pgd_t * pgd;
# 395 "./include/linux/mm_types.h"
atomic_t mm_users;
# 404 "./include/linux/mm_types.h"
atomic_t mm_count;
atomic_long_t nr_ptes;
int map_count;
spinlock_t page_table_lock;
struct rw_semaphore mmap_sem;
struct list_head mmlist;
unsigned long hiwater_rss;
unsigned long hiwater_vm;
unsigned long total_vm;
unsigned long locked_vm;
unsigned long pinned_vm;
unsigned long data_vm;
unsigned long exec_vm;
unsigned long stack_vm;
unsigned long def_flags;
unsigned long start_code, end_code, start_data, end_data;
unsigned long start_brk, brk, start_stack;
unsigned long arg_start, arg_end, env_start, env_end;
unsigned long saved_auxv[(2*(0 + 20 + 1))];
struct mm_rss_stat rss_stat;
struct linux_binfmt *binfmt;
cpumask_var_t cpu_vm_mask_var;
mm_context_t context;
unsigned long flags;
struct core_state *core_state;
atomic_t membarrier_state;
spinlock_t ioctx_lock;
struct kioctx_table *ioctx_table;
# 471 "./include/linux/mm_types.h"
struct task_struct *owner;
struct user_namespace *user_ns;
struct file *exe_file;
# 505 "./include/linux/mm_types.h"
atomic_t tlb_flush_pending;
struct uprobes_state uprobes_state;
struct work_struct async_put_work;
} ;
extern struct mm_struct init_mm;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mm_init_cpumask(struct mm_struct *mm)
{
cpumask_clear(mm->cpu_vm_mask_var);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) cpumask_t *mm_cpumask(struct mm_struct *mm)
{
return mm->cpu_vm_mask_var;
}
struct mmu_gather;
extern void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
unsigned long start, unsigned long end);
extern void tlb_finish_mmu(struct mmu_gather *tlb,
unsigned long start, unsigned long end);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_tlb_flush_pending(struct mm_struct *mm)
{
({ union { typeof(((&mm->tlb_flush_pending)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&mm->tlb_flush_pending)->counter))) ((0)) }; __write_once_size(&(((&mm->tlb_flush_pending)->counter)), __u.__c, sizeof(((&mm->tlb_flush_pending)->counter))); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inc_tlb_flush_pending(struct mm_struct *mm)
{
atomic_add(1, &mm->tlb_flush_pending);
# 588 "./include/linux/mm_types.h"
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dec_tlb_flush_pending(struct mm_struct *mm)
{
# 600 "./include/linux/mm_types.h"
atomic_sub(1, &mm->tlb_flush_pending);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool mm_tlb_flush_pending(struct mm_struct *mm)
{
# 613 "./include/linux/mm_types.h"
return ({ union { typeof((&mm->tlb_flush_pending)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&mm->tlb_flush_pending)->counter), __u.__c, sizeof((&mm->tlb_flush_pending)->counter)); else __read_once_size_nocheck(&((&mm->tlb_flush_pending)->counter), __u.__c, sizeof((&mm->tlb_flush_pending)->counter)); do { } while (0); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool mm_tlb_flush_nested(struct mm_struct *mm)
{
return ({ union { typeof((&mm->tlb_flush_pending)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&mm->tlb_flush_pending)->counter), __u.__c, sizeof((&mm->tlb_flush_pending)->counter)); else __read_once_size_nocheck(&((&mm->tlb_flush_pending)->counter), __u.__c, sizeof((&mm->tlb_flush_pending)->counter)); do { } while (0); __u.__val; }) > 1;
}
struct vm_fault;
struct vm_special_mapping {
const char *name;
struct page **pages;
int (*fault)(const struct vm_special_mapping *sm,
struct vm_area_struct *vma,
struct vm_fault *vmf);
int (*mremap)(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma);
};
enum tlb_flush_reason {
TLB_FLUSH_ON_TASK_SWITCH,
TLB_REMOTE_SHOOTDOWN,
TLB_LOCAL_SHOOTDOWN,
TLB_LOCAL_MM_SHOOTDOWN,
TLB_REMOTE_SEND_IPI,
NR_TLB_FLUSH_REASONS,
};
typedef struct {
unsigned long val;
} swp_entry_t;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const char *vma_get_anon_name(struct vm_area_struct *vma)
{
if (vma->vm_file)
return ((void *)0);
return vma->anon_name;
}
# 23 "./include/linux/fs.h" 2
# 1 "./include/linux/capability.h" 1
# 16 "./include/linux/capability.h"
# 1 "./include/uapi/linux/capability.h" 1
# 39 "./include/uapi/linux/capability.h"
typedef struct __user_cap_header_struct {
__u32 version;
int pid;
} *cap_user_header_t;
typedef struct __user_cap_data_struct {
__u32 effective;
__u32 permitted;
__u32 inheritable;
} *cap_user_data_t;
# 72 "./include/uapi/linux/capability.h"
struct vfs_cap_data {
__le32 magic_etc;
struct {
__le32 permitted;
__le32 inheritable;
} data[2];
};
struct vfs_ns_cap_data {
__le32 magic_etc;
struct {
__le32 permitted;
__le32 inheritable;
} data[2];
__le32 rootid;
};
# 17 "./include/linux/capability.h" 2
extern int file_caps_enabled;
typedef struct kernel_cap_struct {
__u32 cap[2];
} kernel_cap_t;
struct cpu_vfs_cap_data {
__u32 magic_etc;
kernel_cap_t permitted;
kernel_cap_t inheritable;
};
struct file;
struct inode;
struct dentry;
struct task_struct;
struct user_namespace;
extern const kernel_cap_t __cap_empty_set;
extern const kernel_cap_t __cap_init_eff_set;
# 117 "./include/linux/capability.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_combine(const kernel_cap_t a,
const kernel_cap_t b)
{
kernel_cap_t dest;
do { unsigned __capi; for (__capi = 0; __capi < 2; ++__capi) { dest.cap[__capi] = a.cap[__capi] | b.cap[__capi]; } } while (0);
return dest;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_intersect(const kernel_cap_t a,
const kernel_cap_t b)
{
kernel_cap_t dest;
do { unsigned __capi; for (__capi = 0; __capi < 2; ++__capi) { dest.cap[__capi] = a.cap[__capi] & b.cap[__capi]; } } while (0);
return dest;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_drop(const kernel_cap_t a,
const kernel_cap_t drop)
{
kernel_cap_t dest;
do { unsigned __capi; for (__capi = 0; __capi < 2; ++__capi) { dest.cap[__capi] = a.cap[__capi] &~ drop.cap[__capi]; } } while (0);
return dest;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_invert(const kernel_cap_t c)
{
kernel_cap_t dest;
do { unsigned __capi; for (__capi = 0; __capi < 2; ++__capi) { dest.cap[__capi] = ~ c.cap[__capi]; } } while (0);
return dest;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cap_isclear(const kernel_cap_t a)
{
unsigned __capi;
for (__capi = 0; __capi < 2; ++__capi) {
if (a.cap[__capi] != 0)
return false;
}
return true;
}
# 165 "./include/linux/capability.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cap_issubset(const kernel_cap_t a, const kernel_cap_t set)
{
kernel_cap_t dest;
dest = cap_drop(a, set);
return cap_isclear(dest);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_drop_fs_set(const kernel_cap_t a)
{
const kernel_cap_t __cap_fs_set = ((kernel_cap_t){{ ((1 << ((0) & 31)) | (1 << ((27) & 31)) | (1 << ((1) & 31)) | (1 << ((2) & 31)) | (1 << ((3) & 31)) | (1 << ((4) & 31))) | (1 << ((9) & 31)), ((1 << ((32) & 31))) } });
return cap_drop(a, __cap_fs_set);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_raise_fs_set(const kernel_cap_t a,
const kernel_cap_t permitted)
{
const kernel_cap_t __cap_fs_set = ((kernel_cap_t){{ ((1 << ((0) & 31)) | (1 << ((27) & 31)) | (1 << ((1) & 31)) | (1 << ((2) & 31)) | (1 << ((3) & 31)) | (1 << ((4) & 31))) | (1 << ((9) & 31)), ((1 << ((32) & 31))) } });
return cap_combine(a,
cap_intersect(permitted, __cap_fs_set));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_drop_nfsd_set(const kernel_cap_t a)
{
const kernel_cap_t __cap_fs_set = ((kernel_cap_t){{ ((1 << ((0) & 31)) | (1 << ((27) & 31)) | (1 << ((1) & 31)) | (1 << ((2) & 31)) | (1 << ((3) & 31)) | (1 << ((4) & 31))) | (1 << ((24) & 31)), ((1 << ((32) & 31))) } });
return cap_drop(a, __cap_fs_set);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kernel_cap_t cap_raise_nfsd_set(const kernel_cap_t a,
const kernel_cap_t permitted)
{
const kernel_cap_t __cap_nfsd_set = ((kernel_cap_t){{ ((1 << ((0) & 31)) | (1 << ((27) & 31)) | (1 << ((1) & 31)) | (1 << ((2) & 31)) | (1 << ((3) & 31)) | (1 << ((4) & 31))) | (1 << ((24) & 31)), ((1 << ((32) & 31))) } });
return cap_combine(a,
cap_intersect(permitted, __cap_nfsd_set));
}
extern bool has_capability(struct task_struct *t, int cap);
extern bool has_ns_capability(struct task_struct *t,
struct user_namespace *ns, int cap);
extern bool has_capability_noaudit(struct task_struct *t, int cap);
extern bool has_ns_capability_noaudit(struct task_struct *t,
struct user_namespace *ns, int cap);
extern bool capable(int cap);
extern bool ns_capable(struct user_namespace *ns, int cap);
extern bool ns_capable_noaudit(struct user_namespace *ns, int cap);
# 244 "./include/linux/capability.h"
extern bool privileged_wrt_inode_uidgid(struct user_namespace *ns, const struct inode *inode);
extern bool capable_wrt_inode_uidgid(const struct inode *inode, int cap);
extern bool file_ns_capable(const struct file *file, struct user_namespace *ns, int cap);
extern bool ptracer_capable(struct task_struct *tsk, struct user_namespace *ns);
extern int get_vfs_caps_from_disk(const struct dentry *dentry, struct cpu_vfs_cap_data *cpu_caps);
extern int cap_convert_nscap(struct dentry *dentry, void **ivalue, size_t size);
# 24 "./include/linux/fs.h" 2
# 1 "./include/linux/semaphore.h" 1
# 16 "./include/linux/semaphore.h"
struct semaphore {
raw_spinlock_t lock;
unsigned int count;
struct list_head wait_list;
};
# 32 "./include/linux/semaphore.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sema_init(struct semaphore *sem, int val)
{
static struct lock_class_key __key;
*sem = (struct semaphore) { .lock = (raw_spinlock_t) { .raw_lock = { { 0 } }, }, .count = val, .wait_list = { &((*sem).wait_list), &((*sem).wait_list) }, };
do { (void)("semaphore->lock"); (void)(&__key); } while (0);
}
extern void down(struct semaphore *sem);
extern int __attribute__((warn_unused_result)) down_interruptible(struct semaphore *sem);
extern int __attribute__((warn_unused_result)) down_killable(struct semaphore *sem);
extern int __attribute__((warn_unused_result)) down_trylock(struct semaphore *sem);
extern int __attribute__((warn_unused_result)) down_timeout(struct semaphore *sem, long jiffies);
extern void up(struct semaphore *sem);
# 25 "./include/linux/fs.h" 2
# 1 "./include/linux/fcntl.h" 1
# 1 "./include/uapi/linux/fcntl.h" 1
# 1 "./arch/arm/include/uapi/asm/fcntl.h" 1
# 10 "./arch/arm/include/uapi/asm/fcntl.h"
# 1 "./include/uapi/asm-generic/fcntl.h" 1
# 156 "./include/uapi/asm-generic/fcntl.h"
struct f_owner_ex {
int type;
__kernel_pid_t pid;
};
# 196 "./include/uapi/asm-generic/fcntl.h"
struct flock {
short l_type;
short l_whence;
__kernel_off_t l_start;
__kernel_off_t l_len;
__kernel_pid_t l_pid;
};
struct flock64 {
short l_type;
short l_whence;
__kernel_loff_t l_start;
__kernel_loff_t l_len;
__kernel_pid_t l_pid;
};
# 11 "./arch/arm/include/uapi/asm/fcntl.h" 2
# 6 "./include/uapi/linux/fcntl.h" 2
# 6 "./include/linux/fcntl.h" 2
# 26 "./include/linux/fs.h" 2
# 1 "./include/uapi/linux/fiemap.h" 1
# 17 "./include/uapi/linux/fiemap.h"
struct fiemap_extent {
__u64 fe_logical;
__u64 fe_physical;
__u64 fe_length;
__u64 fe_reserved64[2];
__u32 fe_flags;
__u32 fe_reserved[3];
};
struct fiemap {
__u64 fm_start;
__u64 fm_length;
__u32 fm_flags;
__u32 fm_mapped_extents;
__u32 fm_extent_count;
__u32 fm_reserved;
struct fiemap_extent fm_extents[0];
};
# 27 "./include/linux/fs.h" 2
# 1 "./include/linux/migrate_mode.h" 1
# 15 "./include/linux/migrate_mode.h"
enum migrate_mode {
MIGRATE_ASYNC,
MIGRATE_SYNC_LIGHT,
MIGRATE_SYNC,
MIGRATE_SYNC_NO_COPY,
};
# 31 "./include/linux/fs.h" 2
# 1 "./include/linux/percpu-rwsem.h" 1
# 1 "./include/linux/rcuwait.h" 1
# 20 "./include/linux/rcuwait.h"
struct rcuwait {
struct task_struct *task;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rcuwait_init(struct rcuwait *w)
{
w->task = ((void *)0);
}
extern void rcuwait_wake_up(struct rcuwait *w);
# 9 "./include/linux/percpu-rwsem.h" 2
# 1 "./include/linux/rcu_sync.h" 1
# 29 "./include/linux/rcu_sync.h"
enum rcu_sync_type { RCU_SYNC, RCU_SCHED_SYNC, RCU_BH_SYNC };
struct rcu_sync {
int gp_state;
int gp_count;
wait_queue_head_t gp_wait;
int cb_state;
struct callback_head cb_head;
enum rcu_sync_type gp_type;
};
extern void rcu_sync_lockdep_assert(struct rcu_sync *);
# 53 "./include/linux/rcu_sync.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rcu_sync_is_idle(struct rcu_sync *rsp)
{
return !rsp->gp_state;
}
extern void rcu_sync_init(struct rcu_sync *, enum rcu_sync_type);
extern void rcu_sync_enter_start(struct rcu_sync *);
extern void rcu_sync_enter(struct rcu_sync *);
extern void rcu_sync_exit(struct rcu_sync *);
extern void rcu_sync_dtor(struct rcu_sync *);
# 10 "./include/linux/percpu-rwsem.h" 2
struct percpu_rw_semaphore {
struct rcu_sync rss;
unsigned int *read_count;
struct rw_semaphore rw_sem;
struct rcuwait writer;
int readers_block;
};
# 29 "./include/linux/percpu-rwsem.h"
extern int __percpu_down_read(struct percpu_rw_semaphore *, int);
extern void __percpu_up_read(struct percpu_rw_semaphore *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_down_read_preempt_disable(struct percpu_rw_semaphore *sem)
{
do { do { } while (0); } while (0);
do { } while (0);
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
# 47 "./include/linux/percpu-rwsem.h"
({ __this_cpu_preempt_check("add"); do { do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(*sem->read_count)) { case 1: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 2: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 4: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 8: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; default: __bad_size_call_parameter();break; } } while (0); });
if (__builtin_expect(!!(!rcu_sync_is_idle(&sem->rss)), 0))
__percpu_down_read(sem, false);
__asm__ __volatile__("": : :"memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_down_read(struct percpu_rw_semaphore *sem)
{
percpu_down_read_preempt_disable(sem);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int percpu_down_read_trylock(struct percpu_rw_semaphore *sem)
{
int ret = 1;
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
({ __this_cpu_preempt_check("add"); do { do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(*sem->read_count)) { case 1: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 2: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 4: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 8: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; default: __bad_size_call_parameter();break; } } while (0); });
if (__builtin_expect(!!(!rcu_sync_is_idle(&sem->rss)), 0))
ret = __percpu_down_read(sem, true);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
if (ret)
do { } while (0);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_up_read_preempt_enable(struct percpu_rw_semaphore *sem)
{
__asm__ __volatile__("": : :"memory");
if (__builtin_expect(!!(rcu_sync_is_idle(&sem->rss)), 1))
({ __this_cpu_preempt_check("add"); do { do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(*sem->read_count)) { case 1: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*sem->read_count))(1); } while (0);break; case 2: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*sem->read_count))(1); } while (0);break; case 4: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*sem->read_count))(1); } while (0);break; case 8: do { *({ do { const void *__vpp_verify = (typeof((&(*sem->read_count)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))); (typeof((typeof(*(&(*sem->read_count))) *)(&(*sem->read_count)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(*sem->read_count))(1); } while (0);break; default: __bad_size_call_parameter();break; } } while (0); });
else
__percpu_up_read(sem);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
do { } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_up_read(struct percpu_rw_semaphore *sem)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
percpu_up_read_preempt_enable(sem);
}
extern void percpu_down_write(struct percpu_rw_semaphore *);
extern void percpu_up_write(struct percpu_rw_semaphore *);
extern int __percpu_init_rwsem(struct percpu_rw_semaphore *,
const char *, struct lock_class_key *);
extern void percpu_free_rwsem(struct percpu_rw_semaphore *);
# 130 "./include/linux/percpu-rwsem.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_rwsem_release(struct percpu_rw_semaphore *sem,
bool read, unsigned long ip)
{
do { } while (0);
if (!read)
sem->rw_sem.owner = ((struct task_struct *)-1L);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_rwsem_acquire(struct percpu_rw_semaphore *sem,
bool read, unsigned long ip)
{
do { } while (0);
if (!read)
sem->rw_sem.owner = (current_thread_info()->task);
}
# 34 "./include/linux/fs.h" 2
# 1 "./include/linux/delayed_call.h" 1
# 10 "./include/linux/delayed_call.h"
struct delayed_call {
void (*fn)(void *);
void *arg;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_delayed_call(struct delayed_call *call,
void (*fn)(void *), void *arg)
{
call->fn = fn;
call->arg = arg;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void do_delayed_call(struct delayed_call *call)
{
if (call->fn)
call->fn(call->arg);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_delayed_call(struct delayed_call *call)
{
call->fn = ((void *)0);
}
# 36 "./include/linux/fs.h" 2
# 1 "./include/linux/errseq.h" 1
typedef u32 errseq_t;
errseq_t errseq_set(errseq_t *eseq, int err);
errseq_t errseq_sample(errseq_t *eseq);
int errseq_check(errseq_t *eseq, errseq_t since);
int errseq_check_and_advance(errseq_t *eseq, errseq_t *since);
# 38 "./include/linux/fs.h" 2
# 1 "./include/uapi/linux/fs.h" 1
# 13 "./include/uapi/linux/fs.h"
# 1 "./include/uapi/linux/limits.h" 1
# 14 "./include/uapi/linux/fs.h" 2
# 1 "./include/uapi/linux/ioctl.h" 1
# 1 "./arch/arm/include/generated/uapi/asm/ioctl.h" 1
# 1 "./include/asm-generic/ioctl.h" 1
# 1 "./include/uapi/asm-generic/ioctl.h" 1
# 6 "./include/asm-generic/ioctl.h" 2
extern unsigned int __invalid_size_argument_for_IOC;
# 2 "./arch/arm/include/generated/uapi/asm/ioctl.h" 2
# 6 "./include/uapi/linux/ioctl.h" 2
# 15 "./include/uapi/linux/fs.h" 2
# 49 "./include/uapi/linux/fs.h"
struct file_clone_range {
__s64 src_fd;
__u64 src_offset;
__u64 src_length;
__u64 dest_offset;
};
struct fstrim_range {
__u64 start;
__u64 len;
__u64 minlen;
};
struct file_dedupe_range_info {
__s64 dest_fd;
__u64 dest_offset;
__u64 bytes_deduped;
__s32 status;
__u32 reserved;
};
struct file_dedupe_range {
__u64 src_offset;
__u64 src_length;
__u16 dest_count;
__u16 reserved1;
__u32 reserved2;
struct file_dedupe_range_info info[0];
};
struct files_stat_struct {
unsigned long nr_files;
unsigned long nr_free_files;
unsigned long max_files;
};
struct inodes_stat_t {
long nr_inodes;
long nr_unused;
long dummy[5];
};
# 161 "./include/uapi/linux/fs.h"
struct fsxattr {
__u32 fsx_xflags;
__u32 fsx_extsize;
__u32 fsx_nextents;
__u32 fsx_projid;
__u32 fsx_cowextsize;
unsigned char fsx_pad[8];
};
# 324 "./include/uapi/linux/fs.h"
typedef int __kernel_rwf_t;
# 41 "./include/linux/fs.h" 2
struct backing_dev_info;
struct bdi_writeback;
struct bio;
struct export_operations;
struct hd_geometry;
struct iovec;
struct kiocb;
struct kobject;
struct pipe_inode_info;
struct poll_table_struct;
struct kstatfs;
struct vm_area_struct;
struct vfsmount;
struct cred;
struct swap_info_struct;
struct seq_file;
struct workqueue_struct;
struct iov_iter;
struct fscrypt_info;
struct fscrypt_operations;
struct fsverity_info;
struct fsverity_operations;
extern void __attribute__ ((__section__(".init.text"))) inode_init(void);
extern void __attribute__ ((__section__(".init.text"))) inode_init_early(void);
extern void __attribute__ ((__section__(".init.text"))) files_init(void);
extern void __attribute__ ((__section__(".init.text"))) files_maxfiles_init(void);
extern struct files_stat_struct files_stat;
extern unsigned long get_max_files(void);
extern unsigned int sysctl_nr_open;
extern struct inodes_stat_t inodes_stat;
extern int leases_enable, lease_break_time;
extern int sysctl_protected_symlinks;
extern int sysctl_protected_hardlinks;
extern int sysctl_protected_fifos;
extern int sysctl_protected_regular;
typedef __kernel_rwf_t rwf_t;
struct buffer_head;
typedef int (get_block_t)(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create);
typedef int (dio_iodone_t)(struct kiocb *iocb, loff_t offset,
ssize_t bytes, void *private);
# 209 "./include/linux/fs.h"
struct iattr {
unsigned int ia_valid;
umode_t ia_mode;
kuid_t ia_uid;
kgid_t ia_gid;
loff_t ia_size;
struct timespec ia_atime;
struct timespec ia_mtime;
struct timespec ia_ctime;
struct file *ia_file;
};
# 1 "./include/linux/quota.h" 1
# 40 "./include/linux/quota.h"
# 1 "./include/linux/percpu_counter.h" 1
# 20 "./include/linux/percpu_counter.h"
struct percpu_counter {
raw_spinlock_t lock;
s64 count;
struct list_head list;
s32 *counters;
};
extern int percpu_counter_batch;
int __percpu_counter_init(struct percpu_counter *fbc, s64 amount, gfp_t gfp,
struct lock_class_key *key);
# 41 "./include/linux/percpu_counter.h"
void percpu_counter_destroy(struct percpu_counter *fbc);
void percpu_counter_set(struct percpu_counter *fbc, s64 amount);
void percpu_counter_add_batch(struct percpu_counter *fbc, s64 amount,
s32 batch);
s64 __percpu_counter_sum(struct percpu_counter *fbc);
int __percpu_counter_compare(struct percpu_counter *fbc, s64 rhs, s32 batch);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int percpu_counter_compare(struct percpu_counter *fbc, s64 rhs)
{
return __percpu_counter_compare(fbc, rhs, percpu_counter_batch);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_counter_add(struct percpu_counter *fbc, s64 amount)
{
percpu_counter_add_batch(fbc, amount, percpu_counter_batch);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 percpu_counter_sum_positive(struct percpu_counter *fbc)
{
s64 ret = __percpu_counter_sum(fbc);
return ret < 0 ? 0 : ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 percpu_counter_sum(struct percpu_counter *fbc)
{
return __percpu_counter_sum(fbc);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 percpu_counter_read(struct percpu_counter *fbc)
{
return fbc->count;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) s64 percpu_counter_read_positive(struct percpu_counter *fbc)
{
s64 ret = ({ union { typeof(fbc->count) __val; char __c[1]; } __u; if (1) __read_once_size(&(fbc->count), __u.__c, sizeof(fbc->count)); else __read_once_size_nocheck(&(fbc->count), __u.__c, sizeof(fbc->count)); do { } while (0); __u.__val; });
if (ret >= 0)
return ret;
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int percpu_counter_initialized(struct percpu_counter *fbc)
{
return (fbc->counters != ((void *)0));
}
# 177 "./include/linux/percpu_counter.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_counter_inc(struct percpu_counter *fbc)
{
percpu_counter_add(fbc, 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_counter_dec(struct percpu_counter *fbc)
{
percpu_counter_add(fbc, -1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void percpu_counter_sub(struct percpu_counter *fbc, s64 amount)
{
percpu_counter_add(fbc, -amount);
}
# 41 "./include/linux/quota.h" 2
# 1 "./include/uapi/linux/dqblk_xfs.h" 1
# 53 "./include/uapi/linux/dqblk_xfs.h"
typedef struct fs_disk_quota {
__s8 d_version;
__s8 d_flags;
__u16 d_fieldmask;
__u32 d_id;
__u64 d_blk_hardlimit;
__u64 d_blk_softlimit;
__u64 d_ino_hardlimit;
__u64 d_ino_softlimit;
__u64 d_bcount;
__u64 d_icount;
__s32 d_itimer;
__s32 d_btimer;
__u16 d_iwarns;
__u16 d_bwarns;
__s32 d_padding2;
__u64 d_rtb_hardlimit;
__u64 d_rtb_softlimit;
__u64 d_rtbcount;
__s32 d_rtbtimer;
__u16 d_rtbwarns;
__s16 d_padding3;
char d_padding4[8];
} fs_disk_quota_t;
# 149 "./include/uapi/linux/dqblk_xfs.h"
typedef struct fs_qfilestat {
__u64 qfs_ino;
__u64 qfs_nblks;
__u32 qfs_nextents;
} fs_qfilestat_t;
typedef struct fs_quota_stat {
__s8 qs_version;
__u16 qs_flags;
__s8 qs_pad;
fs_qfilestat_t qs_uquota;
fs_qfilestat_t qs_gquota;
__u32 qs_incoredqs;
__s32 qs_btimelimit;
__s32 qs_itimelimit;
__s32 qs_rtbtimelimit;
__u16 qs_bwarnlimit;
__u16 qs_iwarnlimit;
} fs_quota_stat_t;
# 192 "./include/uapi/linux/dqblk_xfs.h"
struct fs_qfilestatv {
__u64 qfs_ino;
__u64 qfs_nblks;
__u32 qfs_nextents;
__u32 qfs_pad;
};
struct fs_quota_statv {
__s8 qs_version;
__u8 qs_pad1;
__u16 qs_flags;
__u32 qs_incoredqs;
struct fs_qfilestatv qs_uquota;
struct fs_qfilestatv qs_gquota;
struct fs_qfilestatv qs_pquota;
__s32 qs_btimelimit;
__s32 qs_itimelimit;
__s32 qs_rtbtimelimit;
__u16 qs_bwarnlimit;
__u16 qs_iwarnlimit;
__u64 qs_pad2[8];
};
# 43 "./include/linux/quota.h" 2
# 1 "./include/linux/dqblk_v1.h" 1
# 44 "./include/linux/quota.h" 2
# 1 "./include/linux/dqblk_v2.h" 1
# 1 "./include/linux/dqblk_qtree.h" 1
# 18 "./include/linux/dqblk_qtree.h"
struct dquot;
struct kqid;
struct qtree_fmt_operations {
void (*mem2disk_dqblk)(void *disk, struct dquot *dquot);
void (*disk2mem_dqblk)(struct dquot *dquot, void *disk);
int (*is_id)(void *disk, struct dquot *dquot);
};
struct qtree_mem_dqinfo {
struct super_block *dqi_sb;
int dqi_type;
unsigned int dqi_blocks;
unsigned int dqi_free_blk;
unsigned int dqi_free_entry;
unsigned int dqi_blocksize_bits;
unsigned int dqi_entry_size;
unsigned int dqi_usable_bs;
unsigned int dqi_qtree_depth;
const struct qtree_fmt_operations *dqi_ops;
};
int qtree_write_dquot(struct qtree_mem_dqinfo *info, struct dquot *dquot);
int qtree_read_dquot(struct qtree_mem_dqinfo *info, struct dquot *dquot);
int qtree_delete_dquot(struct qtree_mem_dqinfo *info, struct dquot *dquot);
int qtree_release_dquot(struct qtree_mem_dqinfo *info, struct dquot *dquot);
int qtree_entry_unused(struct qtree_mem_dqinfo *info, char *disk);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int qtree_depth(struct qtree_mem_dqinfo *info)
{
unsigned int epb = info->dqi_usable_bs >> 2;
unsigned long long entries = epb;
int i;
for (i = 1; entries < (1ULL << 32); i++)
entries *= epb;
return i;
}
int qtree_get_next_id(struct qtree_mem_dqinfo *info, struct kqid *qid);
# 10 "./include/linux/dqblk_v2.h" 2
# 45 "./include/linux/quota.h" 2
# 1 "./include/linux/projid.h" 1
# 17 "./include/linux/projid.h"
struct user_namespace;
extern struct user_namespace init_user_ns;
typedef __kernel_uid32_t projid_t;
typedef struct {
projid_t val;
} kprojid_t;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) projid_t __kprojid_val(kprojid_t projid)
{
return projid.val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool projid_eq(kprojid_t left, kprojid_t right)
{
return __kprojid_val(left) == __kprojid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool projid_lt(kprojid_t left, kprojid_t right)
{
return __kprojid_val(left) < __kprojid_val(right);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool projid_valid(kprojid_t projid)
{
return !projid_eq(projid, (kprojid_t){ -1 });
}
# 65 "./include/linux/projid.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) kprojid_t make_kprojid(struct user_namespace *from, projid_t projid)
{
return (kprojid_t){ projid };
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) projid_t from_kprojid(struct user_namespace *to, kprojid_t kprojid)
{
return __kprojid_val(kprojid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) projid_t from_kprojid_munged(struct user_namespace *to, kprojid_t kprojid)
{
projid_t projid = from_kprojid(to, kprojid);
if (projid == (projid_t)-1)
projid = 65534;
return projid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool kprojid_has_mapping(struct user_namespace *ns, kprojid_t projid)
{
return true;
}
# 49 "./include/linux/quota.h" 2
# 1 "./include/uapi/linux/quota.h" 1
# 90 "./include/uapi/linux/quota.h"
enum {
QIF_BLIMITS_B = 0,
QIF_SPACE_B,
QIF_ILIMITS_B,
QIF_INODES_B,
QIF_BTIME_B,
QIF_ITIME_B,
};
# 110 "./include/uapi/linux/quota.h"
struct if_dqblk {
__u64 dqb_bhardlimit;
__u64 dqb_bsoftlimit;
__u64 dqb_curspace;
__u64 dqb_ihardlimit;
__u64 dqb_isoftlimit;
__u64 dqb_curinodes;
__u64 dqb_btime;
__u64 dqb_itime;
__u32 dqb_valid;
};
struct if_nextdqblk {
__u64 dqb_bhardlimit;
__u64 dqb_bsoftlimit;
__u64 dqb_curspace;
__u64 dqb_ihardlimit;
__u64 dqb_isoftlimit;
__u64 dqb_curinodes;
__u64 dqb_btime;
__u64 dqb_itime;
__u32 dqb_valid;
__u32 dqb_id;
};
# 144 "./include/uapi/linux/quota.h"
enum {
DQF_ROOT_SQUASH_B = 0,
DQF_SYS_FILE_B = 16,
DQF_PRIVATE
};
struct if_dqinfo {
__u64 dqi_bgrace;
__u64 dqi_igrace;
__u32 dqi_flags;
__u32 dqi_valid;
};
# 178 "./include/uapi/linux/quota.h"
enum {
QUOTA_NL_C_UNSPEC,
QUOTA_NL_C_WARNING,
__QUOTA_NL_C_MAX,
};
enum {
QUOTA_NL_A_UNSPEC,
QUOTA_NL_A_QTYPE,
QUOTA_NL_A_EXCESS_ID,
QUOTA_NL_A_WARNING,
QUOTA_NL_A_DEV_MAJOR,
QUOTA_NL_A_DEV_MINOR,
QUOTA_NL_A_CAUSED_ID,
QUOTA_NL_A_PAD,
__QUOTA_NL_A_MAX,
};
# 50 "./include/linux/quota.h" 2
enum quota_type {
USRQUOTA = 0,
GRPQUOTA = 1,
PRJQUOTA = 2,
};
typedef __kernel_uid32_t qid_t;
typedef long long qsize_t;
struct kqid {
union {
kuid_t uid;
kgid_t gid;
kprojid_t projid;
};
enum quota_type type;
};
extern bool qid_eq(struct kqid left, struct kqid right);
extern bool qid_lt(struct kqid left, struct kqid right);
extern qid_t from_kqid(struct user_namespace *to, struct kqid qid);
extern qid_t from_kqid_munged(struct user_namespace *to, struct kqid qid);
extern bool qid_valid(struct kqid qid);
# 97 "./include/linux/quota.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kqid make_kqid(struct user_namespace *from,
enum quota_type type, qid_t qid)
{
struct kqid kqid;
kqid.type = type;
switch (type) {
case USRQUOTA:
kqid.uid = make_kuid(from, qid);
break;
case GRPQUOTA:
kqid.gid = make_kgid(from, qid);
break;
case PRJQUOTA:
kqid.projid = make_kprojid(from, qid);
break;
default:
do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/quota.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "114" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0);
}
return kqid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kqid make_kqid_invalid(enum quota_type type)
{
struct kqid kqid;
kqid.type = type;
switch (type) {
case USRQUOTA:
kqid.uid = (kuid_t){ -1 };
break;
case GRPQUOTA:
kqid.gid = (kgid_t){ -1 };
break;
case PRJQUOTA:
kqid.projid = (kprojid_t){ -1 };
break;
default:
do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/quota.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "141" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0);
}
return kqid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kqid make_kqid_uid(kuid_t uid)
{
struct kqid kqid;
kqid.type = USRQUOTA;
kqid.uid = uid;
return kqid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kqid make_kqid_gid(kgid_t gid)
{
struct kqid kqid;
kqid.type = GRPQUOTA;
kqid.gid = gid;
return kqid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct kqid make_kqid_projid(kprojid_t projid)
{
struct kqid kqid;
kqid.type = PRJQUOTA;
kqid.projid = projid;
return kqid;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool qid_has_mapping(struct user_namespace *ns, struct kqid qid)
{
return from_kqid(ns, qid) != (qid_t) -1;
}
extern spinlock_t dq_data_lock;
# 205 "./include/linux/quota.h"
struct mem_dqblk {
qsize_t dqb_bhardlimit;
qsize_t dqb_bsoftlimit;
qsize_t dqb_curspace;
qsize_t dqb_rsvspace;
qsize_t dqb_ihardlimit;
qsize_t dqb_isoftlimit;
qsize_t dqb_curinodes;
time64_t dqb_btime;
time64_t dqb_itime;
};
struct quota_format_type;
struct mem_dqinfo {
struct quota_format_type *dqi_format;
int dqi_fmt_id;
struct list_head dqi_dirty_list;
unsigned long dqi_flags;
unsigned int dqi_bgrace;
unsigned int dqi_igrace;
qsize_t dqi_max_spc_limit;
qsize_t dqi_max_ino_limit;
void *dqi_priv;
};
struct super_block;
enum {
DQF_INFO_DIRTY_B = DQF_PRIVATE,
};
extern void mark_info_dirty(struct super_block *sb, int type);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int info_dirty(struct mem_dqinfo *info)
{
return test_bit(DQF_INFO_DIRTY_B, &info->dqi_flags);
}
enum {
DQST_LOOKUPS,
DQST_DROPS,
DQST_READS,
DQST_WRITES,
DQST_CACHE_HITS,
DQST_ALLOC_DQUOTS,
DQST_FREE_DQUOTS,
DQST_SYNCS,
_DQST_DQSTAT_LAST
};
struct dqstats {
unsigned long stat[_DQST_DQSTAT_LAST];
struct percpu_counter counter[_DQST_DQSTAT_LAST];
};
extern struct dqstats *dqstats_pcpu;
extern struct dqstats dqstats;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dqstats_inc(unsigned int type)
{
percpu_counter_inc(&dqstats.counter[type]);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dqstats_dec(unsigned int type)
{
percpu_counter_dec(&dqstats.counter[type]);
}
# 295 "./include/linux/quota.h"
struct dquot {
struct hlist_node dq_hash;
struct list_head dq_inuse;
struct list_head dq_free;
struct list_head dq_dirty;
struct mutex dq_lock;
spinlock_t dq_dqb_lock;
atomic_t dq_count;
struct super_block *dq_sb;
struct kqid dq_id;
loff_t dq_off;
unsigned long dq_flags;
struct mem_dqblk dq_dqb;
};
struct quota_format_ops {
int (*check_quota_file)(struct super_block *sb, int type);
int (*read_file_info)(struct super_block *sb, int type);
int (*write_file_info)(struct super_block *sb, int type);
int (*free_file_info)(struct super_block *sb, int type);
int (*read_dqblk)(struct dquot *dquot);
int (*commit_dqblk)(struct dquot *dquot);
int (*release_dqblk)(struct dquot *dquot);
int (*get_next_id)(struct super_block *sb, struct kqid *qid);
};
struct dquot_operations {
int (*write_dquot) (struct dquot *);
struct dquot *(*alloc_dquot)(struct super_block *, int);
void (*destroy_dquot)(struct dquot *);
int (*acquire_dquot) (struct dquot *);
int (*release_dquot) (struct dquot *);
int (*mark_dirty) (struct dquot *);
int (*write_info) (struct super_block *, int);
qsize_t *(*get_reserved_space) (struct inode *);
int (*get_projid) (struct inode *, kprojid_t *);
int (*get_inode_usage) (struct inode *, qsize_t *);
int (*get_next_id) (struct super_block *sb, struct kqid *qid);
};
struct path;
struct qc_dqblk {
int d_fieldmask;
u64 d_spc_hardlimit;
u64 d_spc_softlimit;
u64 d_ino_hardlimit;
u64 d_ino_softlimit;
u64 d_space;
u64 d_ino_count;
s64 d_ino_timer;
s64 d_spc_timer;
int d_ino_warns;
int d_spc_warns;
u64 d_rt_spc_hardlimit;
u64 d_rt_spc_softlimit;
u64 d_rt_space;
s64 d_rt_spc_timer;
int d_rt_spc_warns;
};
# 396 "./include/linux/quota.h"
struct qc_type_state {
unsigned int flags;
unsigned int spc_timelimit;
unsigned int ino_timelimit;
unsigned int rt_spc_timelimit;
unsigned int spc_warnlimit;
unsigned int ino_warnlimit;
unsigned int rt_spc_warnlimit;
unsigned long long ino;
blkcnt_t blocks;
blkcnt_t nextents;
};
struct qc_state {
unsigned int s_incoredqs;
struct qc_type_state s_state[3];
};
struct qc_info {
int i_fieldmask;
unsigned int i_flags;
unsigned int i_spc_timelimit;
unsigned int i_ino_timelimit;
unsigned int i_rt_spc_timelimit;
unsigned int i_spc_warnlimit;
unsigned int i_ino_warnlimit;
unsigned int i_rt_spc_warnlimit;
};
struct quotactl_ops {
int (*quota_on)(struct super_block *, int, int, const struct path *);
int (*quota_off)(struct super_block *, int);
int (*quota_enable)(struct super_block *, unsigned int);
int (*quota_disable)(struct super_block *, unsigned int);
int (*quota_sync)(struct super_block *, int);
int (*set_info)(struct super_block *, int, struct qc_info *);
int (*get_dqblk)(struct super_block *, struct kqid, struct qc_dqblk *);
int (*get_nextdqblk)(struct super_block *, struct kqid *,
struct qc_dqblk *);
int (*set_dqblk)(struct super_block *, struct kqid, struct qc_dqblk *);
int (*get_state)(struct super_block *, struct qc_state *);
int (*rm_xquota)(struct super_block *, unsigned int);
};
struct quota_format_type {
int qf_fmt_id;
const struct quota_format_ops *qf_ops;
struct module *qf_owner;
struct quota_format_type *qf_next;
};
# 470 "./include/linux/quota.h"
enum {
_DQUOT_USAGE_ENABLED = 0,
_DQUOT_LIMITS_ENABLED,
_DQUOT_SUSPENDED,
_DQUOT_STATE_FLAGS
};
# 497 "./include/linux/quota.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int dquot_state_flag(unsigned int flags, int type)
{
return flags << type;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int dquot_generic_flag(unsigned int flags, int type)
{
return (flags >> type) & ((1 << _DQUOT_USAGE_ENABLED * 3) | (1 << _DQUOT_LIMITS_ENABLED * 3) | (1 << _DQUOT_SUSPENDED * 3));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned dquot_state_types(unsigned flags, unsigned flag)
{
do { bool __cond = !(!((flag) == 0 || (((flag) & ((flag) - 1)) != 0))); extern void __compiletime_assert_40(void) ; if (__cond) __compiletime_assert_40(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return (flags / flag) & ((1 << 3) - 1);
}
extern void quota_send_warning(struct kqid qid, dev_t dev,
const char warntype);
# 525 "./include/linux/quota.h"
struct quota_info {
unsigned int flags;
struct rw_semaphore dqio_sem;
struct inode *files[3];
struct mem_dqinfo info[3];
const struct quota_format_ops *ops[3];
};
int register_quota_format(struct quota_format_type *fmt);
void unregister_quota_format(struct quota_format_type *fmt);
struct quota_module_name {
int qm_fmt_id;
char *qm_mod_name;
};
# 231 "./include/linux/fs.h" 2
# 264 "./include/linux/fs.h"
enum positive_aop_returns {
AOP_WRITEPAGE_ACTIVATE = 0x80000,
AOP_TRUNCATED_PAGE = 0x80001,
};
# 277 "./include/linux/fs.h"
struct page;
struct address_space;
struct writeback_control;
enum rw_hint {
WRITE_LIFE_NOT_SET = 0,
WRITE_LIFE_NONE = 1,
WRITE_LIFE_SHORT = 2,
WRITE_LIFE_MEDIUM = 3,
WRITE_LIFE_LONG = 4,
WRITE_LIFE_EXTREME = 5,
};
# 302 "./include/linux/fs.h"
struct kiocb {
struct file *ki_filp;
loff_t ki_pos;
void (*ki_complete)(struct kiocb *iocb, long ret, long ret2);
void *private;
int ki_flags;
enum rw_hint ki_hint;
} ;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_sync_kiocb(struct kiocb *kiocb)
{
return kiocb->ki_complete == ((void *)0);
}
# 325 "./include/linux/fs.h"
typedef struct {
size_t written;
size_t count;
union {
char *buf;
void *data;
} arg;
int error;
} read_descriptor_t;
typedef int (*read_actor_t)(read_descriptor_t *, struct page *,
unsigned long, unsigned long);
struct address_space_operations {
int (*writepage)(struct page *page, struct writeback_control *wbc);
int (*readpage)(struct file *, struct page *);
int (*writepages)(struct address_space *, struct writeback_control *);
int (*set_page_dirty)(struct page *page);
int (*readpages)(struct file *filp, struct address_space *mapping,
struct list_head *pages, unsigned nr_pages);
int (*write_begin)(struct file *, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata);
int (*write_end)(struct file *, struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied,
struct page *page, void *fsdata);
sector_t (*bmap)(struct address_space *, sector_t);
void (*invalidatepage) (struct page *, unsigned int, unsigned int);
int (*releasepage) (struct page *, gfp_t);
void (*freepage)(struct page *);
ssize_t (*direct_IO)(struct kiocb *, struct iov_iter *iter);
int (*migratepage) (struct address_space *,
struct page *, struct page *, enum migrate_mode);
bool (*isolate_page)(struct page *, isolate_mode_t);
void (*putback_page)(struct page *);
int (*launder_page) (struct page *);
int (*is_partially_uptodate) (struct page *, unsigned long,
unsigned long);
void (*is_dirty_writeback) (struct page *, bool *, bool *);
int (*error_remove_page)(struct address_space *, struct page *);
int (*swap_activate)(struct swap_info_struct *sis, struct file *file,
sector_t *span);
void (*swap_deactivate)(struct file *file);
};
extern const struct address_space_operations empty_aops;
int pagecache_write_begin(struct file *, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata);
int pagecache_write_end(struct file *, struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied,
struct page *page, void *fsdata);
struct address_space {
struct inode *host;
struct radix_tree_root page_tree;
spinlock_t tree_lock;
atomic_t i_mmap_writable;
struct rb_root_cached i_mmap;
struct rw_semaphore i_mmap_rwsem;
unsigned long nrpages;
unsigned long nrexceptional;
unsigned long writeback_index;
const struct address_space_operations *a_ops;
unsigned long flags;
spinlock_t private_lock;
gfp_t gfp_mask;
struct list_head private_list;
void *private_data;
errseq_t wb_err;
} __attribute__((aligned(sizeof(long)))) ;
struct request_queue;
struct block_device {
dev_t bd_dev;
int bd_openers;
struct inode * bd_inode;
struct super_block * bd_super;
struct mutex bd_mutex;
void * bd_claiming;
void * bd_holder;
int bd_holders;
bool bd_write_holder;
struct list_head bd_holder_disks;
struct block_device * bd_contains;
unsigned bd_block_size;
u8 bd_partno;
struct hd_struct * bd_part;
unsigned bd_part_count;
int bd_invalidated;
struct gendisk * bd_disk;
struct request_queue * bd_queue;
struct backing_dev_info *bd_bdi;
struct list_head bd_list;
unsigned long bd_private;
int bd_fsfreeze_count;
struct mutex bd_fsfreeze_mutex;
} ;
# 471 "./include/linux/fs.h"
int mapping_tagged(struct address_space *mapping, int tag);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_mmap_lock_write(struct address_space *mapping)
{
down_write(&mapping->i_mmap_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_mmap_unlock_write(struct address_space *mapping)
{
up_write(&mapping->i_mmap_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_mmap_lock_read(struct address_space *mapping)
{
down_read(&mapping->i_mmap_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_mmap_unlock_read(struct address_space *mapping)
{
up_read(&mapping->i_mmap_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mapping_mapped(struct address_space *mapping)
{
return !(({ union { typeof((&mapping->i_mmap.rb_root)->rb_node) __val; char __c[1]; } __u; if (1) __read_once_size(&((&mapping->i_mmap.rb_root)->rb_node), __u.__c, sizeof((&mapping->i_mmap.rb_root)->rb_node)); else __read_once_size_nocheck(&((&mapping->i_mmap.rb_root)->rb_node), __u.__c, sizeof((&mapping->i_mmap.rb_root)->rb_node)); do { } while (0); __u.__val; }) == ((void *)0));
}
# 510 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mapping_writably_mapped(struct address_space *mapping)
{
return ({ union { typeof((&mapping->i_mmap_writable)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&mapping->i_mmap_writable)->counter), __u.__c, sizeof((&mapping->i_mmap_writable)->counter)); else __read_once_size_nocheck(&((&mapping->i_mmap_writable)->counter), __u.__c, sizeof((&mapping->i_mmap_writable)->counter)); do { } while (0); __u.__val; }) > 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mapping_map_writable(struct address_space *mapping)
{
return atomic_inc_unless_negative(&mapping->i_mmap_writable) ?
0 : -1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_unmap_writable(struct address_space *mapping)
{
atomic_sub(1, &mapping->i_mmap_writable);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mapping_deny_writable(struct address_space *mapping)
{
return atomic_dec_unless_positive(&mapping->i_mmap_writable) ?
0 : -16;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_allow_writable(struct address_space *mapping)
{
atomic_add(1, &mapping->i_mmap_writable);
}
# 548 "./include/linux/fs.h"
struct posix_acl;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct posix_acl *
uncached_acl_sentinel(struct task_struct *task)
{
return (void *)task + 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool
is_uncached_acl(struct posix_acl *acl)
{
return (long)acl & 1;
}
struct fsnotify_mark_connector;
struct inode {
umode_t i_mode;
unsigned short i_opflags;
kuid_t i_uid;
kgid_t i_gid;
unsigned int i_flags;
struct posix_acl *i_acl;
struct posix_acl *i_default_acl;
const struct inode_operations *i_op;
struct super_block *i_sb;
struct address_space *i_mapping;
void *i_security;
unsigned long i_ino;
union {
const unsigned int i_nlink;
unsigned int __i_nlink;
};
dev_t i_rdev;
loff_t i_size;
struct timespec i_atime;
struct timespec i_mtime;
struct timespec i_ctime;
spinlock_t i_lock;
unsigned short i_bytes;
unsigned int i_blkbits;
enum rw_hint i_write_hint;
blkcnt_t i_blocks;
seqcount_t i_size_seqcount;
unsigned long i_state;
struct rw_semaphore i_rwsem;
unsigned long dirtied_when;
unsigned long dirtied_time_when;
struct hlist_node i_hash;
struct list_head i_io_list;
struct bdi_writeback *i_wb;
int i_wb_frn_winner;
u16 i_wb_frn_avg_time;
u16 i_wb_frn_history;
struct list_head i_lru;
struct list_head i_sb_list;
struct list_head i_wb_list;
union {
struct hlist_head i_dentry;
struct callback_head i_rcu;
};
u64 i_version;
atomic64_t i_sequence;
atomic_t i_count;
atomic_t i_dio_count;
atomic_t i_writecount;
const struct file_operations *i_fop;
struct file_lock_context *i_flctx;
struct address_space i_data;
struct list_head i_devices;
union {
struct pipe_inode_info *i_pipe;
struct block_device *i_bdev;
struct cdev *i_cdev;
char *i_link;
unsigned i_dir_seq;
};
__u32 i_generation;
__u32 i_fsnotify_mask;
struct fsnotify_mark_connector *i_fsnotify_marks;
struct fscrypt_info *i_crypt_info;
struct fsverity_info *i_verity_info;
void *i_private;
} ;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int i_blocksize(const struct inode *node)
{
return (1 << node->i_blkbits);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int inode_unhashed(struct inode *inode)
{
return hlist_unhashed(&inode->i_hash);
}
# 713 "./include/linux/fs.h"
enum inode_i_mutex_lock_class
{
I_MUTEX_NORMAL,
I_MUTEX_PARENT,
I_MUTEX_CHILD,
I_MUTEX_XATTR,
I_MUTEX_NONDIR2,
I_MUTEX_PARENT2,
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_lock(struct inode *inode)
{
down_write(&inode->i_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_unlock(struct inode *inode)
{
up_write(&inode->i_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_lock_shared(struct inode *inode)
{
down_read(&inode->i_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_unlock_shared(struct inode *inode)
{
up_read(&inode->i_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int inode_trylock(struct inode *inode)
{
return down_write_trylock(&inode->i_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int inode_trylock_shared(struct inode *inode)
{
return down_read_trylock(&inode->i_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int inode_is_locked(struct inode *inode)
{
return rwsem_is_locked(&inode->i_rwsem);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_lock_nested(struct inode *inode, unsigned subclass)
{
down_write(&inode->i_rwsem);
}
void lock_two_nondirectories(struct inode *, struct inode*);
void unlock_two_nondirectories(struct inode *, struct inode*);
# 776 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) loff_t i_size_read(const struct inode *inode)
{
loff_t i_size;
unsigned int seq;
do {
seq = read_seqcount_begin(&inode->i_size_seqcount);
i_size = inode->i_size;
} while (read_seqcount_retry(&inode->i_size_seqcount, seq));
return i_size;
# 797 "./include/linux/fs.h"
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_size_write(struct inode *inode, loff_t i_size)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
write_seqcount_begin(&inode->i_size_seqcount);
inode->i_size = i_size;
write_seqcount_end(&inode->i_size_seqcount);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned iminor(const struct inode *inode)
{
return ((unsigned int) ((inode->i_rdev) & ((1U << 20) - 1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned imajor(const struct inode *inode)
{
return ((unsigned int) ((inode->i_rdev) >> 20));
}
extern struct block_device *I_BDEV(struct inode *inode);
struct fown_struct {
rwlock_t lock;
struct pid *pid;
enum pid_type pid_type;
kuid_t uid, euid;
int signum;
};
struct file_ra_state {
unsigned long start;
unsigned int size;
unsigned int async_size;
unsigned int ra_pages;
unsigned int mmap_miss;
loff_t prev_pos;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int ra_has_index(struct file_ra_state *ra, unsigned long index)
{
return (index >= ra->start &&
index < ra->start + ra->size);
}
struct file {
union {
struct llist_node fu_llist;
struct callback_head fu_rcuhead;
} f_u;
struct path f_path;
struct inode *f_inode;
const struct file_operations *f_op;
spinlock_t f_lock;
enum rw_hint f_write_hint;
atomic_long_t f_count;
unsigned int f_flags;
fmode_t f_mode;
struct mutex f_pos_lock;
loff_t f_pos;
struct fown_struct f_owner;
const struct cred *f_cred;
struct file_ra_state f_ra;
u64 f_version;
void *f_security;
void *private_data;
struct list_head f_ep_links;
struct list_head f_tfile_llink;
struct address_space *f_mapping;
errseq_t f_wb_err;
}
__attribute__((aligned(4)));
struct file_handle {
__u32 handle_bytes;
int handle_type;
unsigned char f_handle[];
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct file *get_file(struct file *f)
{
atomic_long_inc(&f->f_count);
return f;
}
# 953 "./include/linux/fs.h"
typedef void *fl_owner_t;
struct file_lock;
struct file_lock_operations {
void (*fl_copy_lock)(struct file_lock *, struct file_lock *);
void (*fl_release_private)(struct file_lock *);
};
struct lock_manager_operations {
int (*lm_compare_owner)(struct file_lock *, struct file_lock *);
unsigned long (*lm_owner_key)(struct file_lock *);
fl_owner_t (*lm_get_owner)(fl_owner_t);
void (*lm_put_owner)(fl_owner_t);
void (*lm_notify)(struct file_lock *);
int (*lm_grant)(struct file_lock *, int);
bool (*lm_break)(struct file_lock *);
int (*lm_change)(struct file_lock *, int, struct list_head *);
void (*lm_setup)(struct file_lock *, void **);
};
struct lock_manager {
struct list_head list;
bool block_opens;
};
struct net;
void locks_start_grace(struct net *, struct lock_manager *);
void locks_end_grace(struct lock_manager *);
int locks_in_grace(struct net *);
int opens_in_grace(struct net *);
# 1 "./include/linux/nfs_fs_i.h" 1
struct nlm_lockowner;
struct nfs_lock_info {
u32 state;
struct nlm_lockowner *owner;
struct list_head list;
};
struct nfs4_lock_state;
struct nfs4_lock_info {
struct nfs4_lock_state *owner;
};
# 991 "./include/linux/fs.h" 2
# 1009 "./include/linux/fs.h"
struct file_lock {
struct file_lock *fl_next;
struct list_head fl_list;
struct hlist_node fl_link;
struct list_head fl_block;
fl_owner_t fl_owner;
unsigned int fl_flags;
unsigned char fl_type;
unsigned int fl_pid;
int fl_link_cpu;
wait_queue_head_t fl_wait;
struct file *fl_file;
loff_t fl_start;
loff_t fl_end;
struct fasync_struct * fl_fasync;
unsigned long fl_break_time;
unsigned long fl_downgrade_time;
const struct file_lock_operations *fl_ops;
const struct lock_manager_operations *fl_lmops;
union {
struct nfs_lock_info nfs_fl;
struct nfs4_lock_info nfs4_fl;
struct {
struct list_head link;
int state;
} afs;
} fl_u;
} ;
struct file_lock_context {
spinlock_t flc_lock;
struct list_head flc_flock;
struct list_head flc_posix;
struct list_head flc_lease;
};
# 1055 "./include/linux/fs.h"
extern void send_sigio(struct fown_struct *fown, int fd, int band);
# 1064 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct inode *locks_inode(const struct file *f)
{
return f->f_path.dentry->d_inode;
}
extern int fcntl_getlk(struct file *, unsigned int, struct flock *);
extern int fcntl_setlk(unsigned int, struct file *, unsigned int,
struct flock *);
extern int fcntl_getlk64(struct file *, unsigned int, struct flock64 *);
extern int fcntl_setlk64(unsigned int, struct file *, unsigned int,
struct flock64 *);
extern int fcntl_setlease(unsigned int fd, struct file *filp, long arg);
extern int fcntl_getlease(struct file *filp);
void locks_free_lock_context(struct inode *inode);
void locks_free_lock(struct file_lock *fl);
extern void locks_init_lock(struct file_lock *);
extern struct file_lock * locks_alloc_lock(void);
extern void locks_copy_lock(struct file_lock *, struct file_lock *);
extern void locks_copy_conflock(struct file_lock *, struct file_lock *);
extern void locks_remove_posix(struct file *, fl_owner_t);
extern void locks_remove_file(struct file *);
extern void locks_release_private(struct file_lock *);
extern void posix_test_lock(struct file *, struct file_lock *);
extern int posix_lock_file(struct file *, struct file_lock *, struct file_lock *);
extern int posix_unblock_lock(struct file_lock *);
extern int vfs_test_lock(struct file *, struct file_lock *);
extern int vfs_lock_file(struct file *, unsigned int, struct file_lock *, struct file_lock *);
extern int vfs_cancel_lock(struct file *filp, struct file_lock *fl);
extern int locks_lock_inode_wait(struct inode *inode, struct file_lock *fl);
extern int __break_lease(struct inode *inode, unsigned int flags, unsigned int type);
extern void lease_get_mtime(struct inode *, struct timespec *time);
extern int generic_setlease(struct file *, long, struct file_lock **, void **priv);
extern int vfs_setlease(struct file *, long, struct file_lock **, void **);
extern int lease_modify(struct file_lock *, int, struct list_head *);
struct files_struct;
extern void show_fd_locks(struct seq_file *f,
struct file *filp, struct files_struct *files);
# 1244 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct inode *file_inode(const struct file *f)
{
return f->f_inode;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dentry *file_dentry(const struct file *file)
{
return d_real(file->f_path.dentry, file_inode(file), 0, 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int locks_lock_file_wait(struct file *filp, struct file_lock *fl)
{
return locks_lock_inode_wait(locks_inode(filp), fl);
}
struct fasync_struct {
spinlock_t fa_lock;
int magic;
int fa_fd;
struct fasync_struct *fa_next;
struct file *fa_file;
struct callback_head fa_rcu;
};
extern int fasync_helper(int, struct file *, int, struct fasync_struct **);
extern struct fasync_struct *fasync_insert_entry(int, struct file *, struct fasync_struct **, struct fasync_struct *);
extern int fasync_remove_entry(struct file *, struct fasync_struct **);
extern struct fasync_struct *fasync_alloc(void);
extern void fasync_free(struct fasync_struct *);
extern void kill_fasync(struct fasync_struct **, int, int);
extern void __f_setown(struct file *filp, struct pid *, enum pid_type, int force);
extern int f_setown(struct file *filp, unsigned long arg, int force);
extern void f_delown(struct file *filp);
extern pid_t f_getown(struct file *filp);
extern int send_sigurg(struct fown_struct *fown);
# 1339 "./include/linux/fs.h"
enum {
SB_UNFROZEN = 0,
SB_FREEZE_WRITE = 1,
SB_FREEZE_PAGEFAULT = 2,
SB_FREEZE_FS = 3,
SB_FREEZE_COMPLETE = 4,
};
struct sb_writers {
int frozen;
wait_queue_head_t wait_unfrozen;
struct percpu_rw_semaphore rw_sem[(SB_FREEZE_COMPLETE - 1)];
};
struct super_block {
struct list_head s_list;
dev_t s_dev;
unsigned char s_blocksize_bits;
unsigned long s_blocksize;
loff_t s_maxbytes;
struct file_system_type *s_type;
const struct super_operations *s_op;
const struct dquot_operations *dq_op;
const struct quotactl_ops *s_qcop;
const struct export_operations *s_export_op;
unsigned long s_flags;
unsigned long s_iflags;
unsigned long s_magic;
struct dentry *s_root;
struct rw_semaphore s_umount;
int s_count;
atomic_t s_active;
void *s_security;
const struct xattr_handler **s_xattr;
const struct fscrypt_operations *s_cop;
struct key *s_master_keys;
const struct fsverity_operations *s_vop;
struct hlist_bl_head s_anon;
struct unicode_map *s_encoding;
__u16 s_encoding_flags;
struct list_head s_mounts;
struct block_device *s_bdev;
struct backing_dev_info *s_bdi;
struct mtd_info *s_mtd;
struct hlist_node s_instances;
unsigned int s_quota_types;
struct quota_info s_dquot;
struct sb_writers s_writers;
char s_id[32];
uuid_t s_uuid;
void *s_fs_info;
unsigned int s_max_links;
fmode_t s_mode;
u32 s_time_gran;
struct mutex s_vfs_rename_mutex;
char *s_subtype;
const struct dentry_operations *s_d_op;
int cleancache_poolid;
struct shrinker s_shrink;
atomic_long_t s_remove_count;
int s_readonly_remount;
struct workqueue_struct *s_dio_done_wq;
struct hlist_head s_pins;
struct user_namespace *s_user_ns;
struct list_lru s_dentry_lru __attribute__((__aligned__((1 << 6))));
struct list_lru s_inode_lru __attribute__((__aligned__((1 << 6))));
struct callback_head rcu;
struct work_struct destroy_work;
struct mutex s_sync_lock;
int s_stack_depth;
spinlock_t s_inode_list_lock __attribute__((__aligned__((1 << 6))));
struct list_head s_inodes;
spinlock_t s_inode_wblist_lock;
struct list_head s_inodes_wb;
} ;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) uid_t i_uid_read(const struct inode *inode)
{
return from_kuid(inode->i_sb->s_user_ns, inode->i_uid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) gid_t i_gid_read(const struct inode *inode)
{
return from_kgid(inode->i_sb->s_user_ns, inode->i_gid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_uid_write(struct inode *inode, uid_t uid)
{
inode->i_uid = make_kuid(inode->i_sb->s_user_ns, uid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_gid_write(struct inode *inode, gid_t gid)
{
inode->i_gid = make_kgid(inode->i_sb->s_user_ns, gid);
}
extern struct timespec current_time(struct inode *inode);
void __sb_end_write(struct super_block *sb, int level);
int __sb_start_write(struct super_block *sb, int level, bool wait);
# 1520 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sb_end_write(struct super_block *sb)
{
__sb_end_write(sb, SB_FREEZE_WRITE);
}
# 1532 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sb_end_pagefault(struct super_block *sb)
{
__sb_end_write(sb, SB_FREEZE_PAGEFAULT);
}
# 1544 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sb_end_intwrite(struct super_block *sb)
{
__sb_end_write(sb, SB_FREEZE_FS);
}
# 1568 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sb_start_write(struct super_block *sb)
{
__sb_start_write(sb, SB_FREEZE_WRITE, true);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int sb_start_write_trylock(struct super_block *sb)
{
return __sb_start_write(sb, SB_FREEZE_WRITE, false);
}
# 1597 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sb_start_pagefault(struct super_block *sb)
{
__sb_start_write(sb, SB_FREEZE_PAGEFAULT, true);
}
# 1615 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sb_start_intwrite(struct super_block *sb)
{
__sb_start_write(sb, SB_FREEZE_FS, true);
}
extern bool inode_owner_or_capable(const struct inode *inode);
extern int vfs_create(struct inode *, struct dentry *, umode_t, bool);
extern int vfs_create2(struct vfsmount *, struct inode *, struct dentry *, umode_t, bool);
extern int vfs_mkdir(struct inode *, struct dentry *, umode_t);
extern int vfs_mkdir2(struct vfsmount *, struct inode *, struct dentry *, umode_t);
extern int vfs_mknod(struct inode *, struct dentry *, umode_t, dev_t);
extern int vfs_mknod2(struct vfsmount *, struct inode *, struct dentry *, umode_t, dev_t);
extern int vfs_symlink(struct inode *, struct dentry *, const char *);
extern int vfs_symlink2(struct vfsmount *, struct inode *, struct dentry *, const char *);
extern int vfs_link(struct dentry *, struct inode *, struct dentry *, struct inode **);
extern int vfs_link2(struct vfsmount *, struct dentry *, struct inode *, struct dentry *, struct inode **);
extern int vfs_rmdir(struct inode *, struct dentry *);
extern int vfs_rmdir2(struct vfsmount *, struct inode *, struct dentry *);
extern int vfs_unlink(struct inode *, struct dentry *, struct inode **);
extern int vfs_unlink2(struct vfsmount *, struct inode *, struct dentry *, struct inode **);
extern int vfs_rename(struct inode *, struct dentry *, struct inode *, struct dentry *, struct inode **, unsigned int);
extern int vfs_rename2(struct vfsmount *, struct inode *, struct dentry *, struct inode *, struct dentry *, struct inode **, unsigned int);
extern int vfs_whiteout(struct inode *, struct dentry *);
extern struct dentry *vfs_tmpfile(struct dentry *dentry, umode_t mode,
int open_flag);
extern void inode_init_owner(struct inode *inode, const struct inode *dir,
umode_t mode);
extern bool may_open_dev(const struct path *path);
struct fiemap_extent_info {
unsigned int fi_flags;
unsigned int fi_extents_mapped;
unsigned int fi_extents_max;
struct fiemap_extent *fi_extents_start;
};
int fiemap_fill_next_extent(struct fiemap_extent_info *info, u64 logical,
u64 phys, u64 len, u32 flags);
int fiemap_check_flags(struct fiemap_extent_info *fieinfo, u32 fs_flags);
# 1689 "./include/linux/fs.h"
struct dir_context;
typedef int (*filldir_t)(struct dir_context *, const char *, int, loff_t, u64,
unsigned);
struct dir_context {
const filldir_t actor;
loff_t pos;
};
struct block_device_operations;
# 1726 "./include/linux/fs.h"
struct iov_iter;
struct file_operations {
struct module *owner;
loff_t (*llseek) (struct file *, loff_t, int);
ssize_t (*read) (struct file *, char *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char *, size_t, loff_t *);
ssize_t (*read_iter) (struct kiocb *, struct iov_iter *);
ssize_t (*write_iter) (struct kiocb *, struct iov_iter *);
int (*iterate) (struct file *, struct dir_context *);
int (*iterate_shared) (struct file *, struct dir_context *);
unsigned int (*poll) (struct file *, struct poll_table_struct *);
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
int (*fsync) (struct file *, loff_t, loff_t, int datasync);
int (*fasync) (int, struct file *, int);
int (*lock) (struct file *, int, struct file_lock *);
ssize_t (*sendpage) (struct file *, struct page *, int, size_t, loff_t *, int);
unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
int (*check_flags)(int);
int (*flock) (struct file *, int, struct file_lock *);
ssize_t (*splice_write)(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int);
ssize_t (*splice_read)(struct file *, loff_t *, struct pipe_inode_info *, size_t, unsigned int);
int (*setlease)(struct file *, long, struct file_lock **, void **);
long (*fallocate)(struct file *file, int mode, loff_t offset,
loff_t len);
void (*show_fdinfo)(struct seq_file *m, struct file *f);
ssize_t (*copy_file_range)(struct file *, loff_t, struct file *,
loff_t, size_t, unsigned int);
int (*clone_file_range)(struct file *, loff_t, struct file *, loff_t,
u64);
ssize_t (*dedupe_file_range)(struct file *, u64, u64, struct file *,
u64);
} ;
struct inode_operations {
struct dentry * (*lookup) (struct inode *,struct dentry *, unsigned int);
const char * (*get_link) (struct dentry *, struct inode *, struct delayed_call *);
int (*permission) (struct inode *, int);
int (*permission2) (struct vfsmount *, struct inode *, int);
struct posix_acl * (*get_acl)(struct inode *, int);
int (*readlink) (struct dentry *, char *,int);
int (*create) (struct inode *,struct dentry *, umode_t, bool);
int (*link) (struct dentry *,struct inode *,struct dentry *);
int (*unlink) (struct inode *,struct dentry *);
int (*symlink) (struct inode *,struct dentry *,const char *);
int (*mkdir) (struct inode *,struct dentry *,umode_t);
int (*rmdir) (struct inode *,struct dentry *);
int (*mknod) (struct inode *,struct dentry *,umode_t,dev_t);
int (*rename) (struct inode *, struct dentry *,
struct inode *, struct dentry *, unsigned int);
int (*setattr) (struct dentry *, struct iattr *);
int (*setattr2) (struct vfsmount *, struct dentry *, struct iattr *);
int (*getattr) (const struct path *, struct kstat *, u32, unsigned int);
ssize_t (*listxattr) (struct dentry *, char *, size_t);
int (*fiemap)(struct inode *, struct fiemap_extent_info *, u64 start,
u64 len);
int (*update_time)(struct inode *, struct timespec *, int);
int (*atomic_open)(struct inode *, struct dentry *,
struct file *, unsigned open_flag,
umode_t create_mode, int *opened);
int (*tmpfile) (struct inode *, struct dentry *, umode_t);
int (*set_acl)(struct inode *, struct posix_acl *, int);
} __attribute__((__aligned__((1 << 6))));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ssize_t call_read_iter(struct file *file, struct kiocb *kio,
struct iov_iter *iter)
{
return file->f_op->read_iter(kio, iter);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ssize_t call_write_iter(struct file *file, struct kiocb *kio,
struct iov_iter *iter)
{
return file->f_op->write_iter(kio, iter);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int call_mmap(struct file *file, struct vm_area_struct *vma)
{
return file->f_op->mmap(file, vma);
}
ssize_t rw_copy_check_uvector(int type, const struct iovec * uvector,
unsigned long nr_segs, unsigned long fast_segs,
struct iovec *fast_pointer,
struct iovec **ret_pointer);
extern ssize_t __vfs_read(struct file *, char *, size_t, loff_t *);
extern ssize_t vfs_read(struct file *, char *, size_t, loff_t *);
extern ssize_t vfs_write(struct file *, const char *, size_t, loff_t *);
extern ssize_t vfs_readv(struct file *, const struct iovec *,
unsigned long, loff_t *, rwf_t);
extern ssize_t vfs_copy_file_range(struct file *, loff_t , struct file *,
loff_t, size_t, unsigned int);
extern int vfs_clone_file_prep_inodes(struct inode *inode_in, loff_t pos_in,
struct inode *inode_out, loff_t pos_out,
u64 *len, bool is_dedupe);
extern int do_clone_file_range(struct file *file_in, loff_t pos_in,
struct file *file_out, loff_t pos_out, u64 len);
extern int vfs_clone_file_range(struct file *file_in, loff_t pos_in,
struct file *file_out, loff_t pos_out, u64 len);
extern int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
struct inode *dest, loff_t destoff,
loff_t len, bool *is_same);
extern int vfs_dedupe_file_range(struct file *file,
struct file_dedupe_range *same);
struct super_operations {
struct inode *(*alloc_inode)(struct super_block *sb);
void (*destroy_inode)(struct inode *);
void (*dirty_inode) (struct inode *, int flags);
int (*write_inode) (struct inode *, struct writeback_control *wbc);
int (*drop_inode) (struct inode *);
void (*evict_inode) (struct inode *);
void (*put_super) (struct super_block *);
int (*sync_fs)(struct super_block *sb, int wait);
int (*freeze_super) (struct super_block *);
int (*freeze_fs) (struct super_block *);
int (*thaw_super) (struct super_block *);
int (*unfreeze_fs) (struct super_block *);
int (*statfs) (struct dentry *, struct kstatfs *);
int (*remount_fs) (struct super_block *, int *, char *);
int (*remount_fs2) (struct vfsmount *, struct super_block *, int *, char *);
void *(*clone_mnt_data) (void *);
void (*copy_mnt_data) (void *, void *);
void (*umount_begin) (struct super_block *);
int (*show_options)(struct seq_file *, struct dentry *);
int (*show_options2)(struct vfsmount *,struct seq_file *, struct dentry *);
int (*show_devname)(struct seq_file *, struct dentry *);
int (*show_path)(struct seq_file *, struct dentry *);
int (*show_stats)(struct seq_file *, struct dentry *);
ssize_t (*quota_read)(struct super_block *, int, char *, size_t, loff_t);
ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t);
struct dquot **(*get_dquots)(struct inode *);
int (*bdev_try_to_free_page)(struct super_block*, struct page*, gfp_t);
long (*nr_cached_objects)(struct super_block *,
struct shrink_control *);
long (*free_cached_objects)(struct super_block *,
struct shrink_control *);
};
# 1920 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool sb_rdonly(const struct super_block *sb) { return sb->s_flags & 1; }
# 1950 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool HAS_UNMAPPED_ID(struct inode *inode)
{
return !uid_valid(inode->i_uid) || !gid_valid(inode->i_gid);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) enum rw_hint file_write_hint(struct file *file)
{
if (file->f_write_hint != WRITE_LIFE_NOT_SET)
return file->f_write_hint;
return file_inode(file)->i_write_hint;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int iocb_flags(struct file *file);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_sync_kiocb(struct kiocb *kiocb, struct file *filp)
{
*kiocb = (struct kiocb) {
.ki_filp = filp,
.ki_flags = iocb_flags(filp),
.ki_hint = file_write_hint(filp),
};
}
# 2061 "./include/linux/fs.h"
extern void __mark_inode_dirty(struct inode *, int);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mark_inode_dirty(struct inode *inode)
{
__mark_inode_dirty(inode, ((1 << 0) | (1 << 1) | (1 << 2)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mark_inode_dirty_sync(struct inode *inode)
{
__mark_inode_dirty(inode, (1 << 0));
}
extern void inc_nlink(struct inode *inode);
extern void drop_nlink(struct inode *inode);
extern void clear_nlink(struct inode *inode);
extern void set_nlink(struct inode *inode, unsigned int nlink);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_inc_link_count(struct inode *inode)
{
inc_nlink(inode);
mark_inode_dirty(inode);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_dec_link_count(struct inode *inode)
{
drop_nlink(inode);
mark_inode_dirty(inode);
}
# 2097 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_inc_iversion(struct inode *inode)
{
spin_lock(&inode->i_lock);
inode->i_version++;
spin_unlock(&inode->i_lock);
}
enum file_time_flags {
S_ATIME = 1,
S_MTIME = 2,
S_CTIME = 4,
S_VERSION = 8,
};
extern void touch_atime(const struct path *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void file_accessed(struct file *file)
{
if (!(file->f_flags & 01000000))
touch_atime(&file->f_path);
}
int sync_inode(struct inode *inode, struct writeback_control *wbc);
int sync_inode_metadata(struct inode *inode, int wait);
struct file_system_type {
const char *name;
int fs_flags;
struct dentry *(*mount) (struct file_system_type *, int,
const char *, void *);
struct dentry *(*mount2) (struct vfsmount *, struct file_system_type *, int,
const char *, void *);
void *(*alloc_mnt_data) (void);
void (*kill_sb) (struct super_block *);
struct module *owner;
struct file_system_type * next;
struct hlist_head fs_supers;
struct lock_class_key s_lock_key;
struct lock_class_key s_umount_key;
struct lock_class_key s_vfs_rename_key;
struct lock_class_key s_writers_key[(SB_FREEZE_COMPLETE - 1)];
struct lock_class_key i_lock_key;
struct lock_class_key i_mutex_key;
struct lock_class_key i_mutex_dir_key;
};
extern struct dentry *mount_ns(struct file_system_type *fs_type,
int flags, void *data, void *ns, struct user_namespace *user_ns,
int (*fill_super)(struct super_block *, void *, int));
extern struct dentry *mount_bdev(struct file_system_type *fs_type,
int flags, const char *dev_name, void *data,
int (*fill_super)(struct super_block *, void *, int));
extern struct dentry *mount_single(struct file_system_type *fs_type,
int flags, void *data,
int (*fill_super)(struct super_block *, void *, int));
extern struct dentry *mount_nodev(struct file_system_type *fs_type,
int flags, void *data,
int (*fill_super)(struct super_block *, void *, int));
extern struct dentry *mount_subtree(struct vfsmount *mnt, const char *path);
void generic_shutdown_super(struct super_block *sb);
void kill_block_super(struct super_block *sb);
void kill_anon_super(struct super_block *sb);
void kill_litter_super(struct super_block *sb);
void deactivate_super(struct super_block *sb);
void deactivate_locked_super(struct super_block *sb);
int set_anon_super(struct super_block *s, void *data);
int get_anon_bdev(dev_t *);
void free_anon_bdev(dev_t);
struct super_block *sget_userns(struct file_system_type *type,
int (*test)(struct super_block *,void *),
int (*set)(struct super_block *,void *),
int flags, struct user_namespace *user_ns,
void *data);
struct super_block *sget(struct file_system_type *type,
int (*test)(struct super_block *,void *),
int (*set)(struct super_block *,void *),
int flags, void *data);
extern struct dentry *mount_pseudo_xattr(struct file_system_type *, char *,
const struct super_operations *ops,
const struct xattr_handler **xattr,
const struct dentry_operations *dops,
unsigned long);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dentry *
mount_pseudo(struct file_system_type *fs_type, char *name,
const struct super_operations *ops,
const struct dentry_operations *dops, unsigned long magic)
{
return mount_pseudo_xattr(fs_type, name, ops, ((void *)0), dops, magic);
}
# 2213 "./include/linux/fs.h"
extern int register_filesystem(struct file_system_type *);
extern int unregister_filesystem(struct file_system_type *);
extern struct vfsmount *kern_mount_data(struct file_system_type *, void *data);
extern void kern_unmount(struct vfsmount *mnt);
extern int may_umount_tree(struct vfsmount *);
extern int may_umount(struct vfsmount *);
extern long do_mount(const char *, const char *,
const char *, unsigned long, void *);
extern struct vfsmount *collect_mounts(const struct path *);
extern void drop_collected_mounts(struct vfsmount *);
extern int iterate_mounts(int (*)(struct vfsmount *, void *), void *,
struct vfsmount *);
extern int vfs_statfs(const struct path *, struct kstatfs *);
extern int user_statfs(const char *, struct kstatfs *);
extern int fd_statfs(int, struct kstatfs *);
extern int vfs_ustat(dev_t, struct kstatfs *);
extern int freeze_super(struct super_block *super);
extern int thaw_super(struct super_block *super);
extern bool our_mnt(struct vfsmount *mnt);
extern __attribute__((format(printf, 2, 3)))
int super_setup_bdi_name(struct super_block *sb, char *fmt, ...);
extern int super_setup_bdi(struct super_block *sb);
extern int current_umask(void);
extern void ihold(struct inode * inode);
extern void iput(struct inode *);
extern int generic_update_time(struct inode *, struct timespec *, int);
extern struct kobject *fs_kobj;
extern int locks_mandatory_locked(struct file *);
extern int locks_mandatory_area(struct inode *, struct file *, loff_t, loff_t, unsigned char);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __mandatory_lock(struct inode *ino)
{
return (ino->i_mode & (0002000 | 00010)) == 0002000;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mandatory_lock(struct inode *ino)
{
return ((ino)->i_sb->s_flags & (64)) && __mandatory_lock(ino);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int locks_verify_locked(struct file *file)
{
if (mandatory_lock(locks_inode(file)))
return locks_mandatory_locked(file);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int locks_verify_truncate(struct inode *inode,
struct file *f,
loff_t size)
{
if (!inode->i_flctx || !mandatory_lock(inode))
return 0;
if (size < inode->i_size) {
return locks_mandatory_area(inode, f, size, inode->i_size - 1,
1);
} else {
return locks_mandatory_area(inode, f, inode->i_size, size - 1,
1);
}
}
# 2333 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int break_lease(struct inode *inode, unsigned int mode)
{
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
return __break_lease(inode, mode, 32);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int break_deleg(struct inode *inode, unsigned int mode)
{
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
return __break_lease(inode, mode, 4);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int try_break_deleg(struct inode *inode, struct inode **delegated_inode)
{
int ret;
ret = break_deleg(inode, 00000001|00004000);
if (ret == -11 && delegated_inode) {
*delegated_inode = inode;
ihold(inode);
}
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int break_deleg_wait(struct inode **delegated_inode)
{
int ret;
ret = break_deleg(*delegated_inode, 00000001);
iput(*delegated_inode);
*delegated_inode = ((void *)0);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int break_layout(struct inode *inode, bool wait)
{
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
return __break_lease(inode,
wait ? 00000001 : 00000001 | 00004000,
2048);
return 0;
}
# 2423 "./include/linux/fs.h"
struct audit_names;
struct filename {
const char *name;
const char *uptr;
struct audit_names *aname;
int refcnt;
const char iname[];
};
extern long vfs_truncate(const struct path *, loff_t);
extern int do_truncate(struct dentry *, loff_t start, unsigned int time_attrs,
struct file *filp);
extern int do_truncate2(struct vfsmount *, struct dentry *, loff_t start,
unsigned int time_attrs, struct file *filp);
extern int vfs_fallocate(struct file *file, int mode, loff_t offset,
loff_t len);
extern long do_sys_open(int dfd, const char *filename, int flags,
umode_t mode);
extern struct file *file_open_name(struct filename *, int, umode_t);
extern struct file *filp_open(const char *, int, umode_t);
extern struct file *file_open_root(struct dentry *, struct vfsmount *,
const char *, int, umode_t);
extern struct file * dentry_open(const struct path *, int, const struct cred *);
extern int filp_close(struct file *, fl_owner_t id);
extern struct filename *getname_flags(const char *, int, int *);
extern struct filename *getname(const char *);
extern struct filename *getname_kernel(const char *);
extern void putname(struct filename *name);
enum {
FILE_CREATED = 1,
FILE_OPENED = 2
};
extern int finish_open(struct file *file, struct dentry *dentry,
int (*open)(struct inode *, struct file *),
int *opened);
extern int finish_no_open(struct file *file, struct dentry *dentry);
extern int ioctl_preallocate(struct file *filp, void *argp);
extern void __attribute__ ((__section__(".init.text"))) vfs_caches_init_early(void);
extern void __attribute__ ((__section__(".init.text"))) vfs_caches_init(void);
extern struct kmem_cache *names_cachep;
extern int register_blkdev(unsigned int, const char *);
extern void unregister_blkdev(unsigned int, const char *);
extern void bdev_unhash_inode(dev_t dev);
extern struct block_device *bdget(dev_t);
extern struct block_device *bdgrab(struct block_device *bdev);
extern void bd_set_size(struct block_device *, loff_t size);
extern void bd_forget(struct inode *inode);
extern void bdput(struct block_device *);
extern void invalidate_bdev(struct block_device *);
extern void iterate_bdevs(void (*)(struct block_device *, void *), void *);
extern int sync_blockdev(struct block_device *bdev);
extern void kill_bdev(struct block_device *);
extern struct super_block *freeze_bdev(struct block_device *);
extern void emergency_thaw_all(void);
extern int thaw_bdev(struct block_device *bdev, struct super_block *sb);
extern int fsync_bdev(struct block_device *);
extern struct super_block *blockdev_superblock;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool sb_is_blkdev_sb(struct super_block *sb)
{
return sb == blockdev_superblock;
}
# 2524 "./include/linux/fs.h"
extern int sync_filesystem(struct super_block *);
extern const struct file_operations def_blk_fops;
extern const struct file_operations def_chr_fops;
extern int ioctl_by_bdev(struct block_device *, unsigned, unsigned long);
extern int blkdev_ioctl(struct block_device *, fmode_t, unsigned, unsigned long);
extern long compat_blkdev_ioctl(struct file *, unsigned, unsigned long);
extern int blkdev_get(struct block_device *bdev, fmode_t mode, void *holder);
extern struct block_device *blkdev_get_by_path(const char *path, fmode_t mode,
void *holder);
extern struct block_device *blkdev_get_by_dev(dev_t dev, fmode_t mode,
void *holder);
extern void blkdev_put(struct block_device *bdev, fmode_t mode);
extern int __blkdev_reread_part(struct block_device *bdev);
extern int blkdev_reread_part(struct block_device *bdev);
extern int bd_link_disk_holder(struct block_device *bdev, struct gendisk *disk);
extern void bd_unlink_disk_holder(struct block_device *bdev,
struct gendisk *disk);
# 2565 "./include/linux/fs.h"
extern int alloc_chrdev_region(dev_t *, unsigned, unsigned, const char *);
extern int register_chrdev_region(dev_t, unsigned, const char *);
extern int __register_chrdev(unsigned int major, unsigned int baseminor,
unsigned int count, const char *name,
const struct file_operations *fops);
extern void __unregister_chrdev(unsigned int major, unsigned int baseminor,
unsigned int count, const char *name);
extern void unregister_chrdev_region(dev_t, unsigned);
extern void chrdev_show(struct seq_file *,off_t);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int register_chrdev(unsigned int major, const char *name,
const struct file_operations *fops)
{
return __register_chrdev(major, 0, 256, name, fops);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void unregister_chrdev(unsigned int major, const char *name)
{
__unregister_chrdev(major, 0, 256, name);
}
extern const char *__bdevname(dev_t, char *buffer);
extern const char *bdevname(struct block_device *bdev, char *buffer);
extern struct block_device *lookup_bdev(const char *);
extern void blkdev_show(struct seq_file *,off_t);
extern void init_special_inode(struct inode *, umode_t, dev_t);
extern void make_bad_inode(struct inode *);
extern bool is_bad_inode(struct inode *);
extern void check_disk_size_change(struct gendisk *disk,
struct block_device *bdev);
extern int revalidate_disk(struct gendisk *);
extern int check_disk_change(struct block_device *);
extern int __invalidate_device(struct block_device *, bool);
extern int invalidate_partition(struct gendisk *, int);
unsigned long invalidate_mapping_pages(struct address_space *mapping,
unsigned long start, unsigned long end);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void invalidate_remote_inode(struct inode *inode)
{
if ((((inode->i_mode) & 00170000) == 0100000) || (((inode->i_mode) & 00170000) == 0040000) ||
(((inode->i_mode) & 00170000) == 0120000))
invalidate_mapping_pages(inode->i_mapping, 0, -1);
}
extern int invalidate_inode_pages2(struct address_space *mapping);
extern int invalidate_inode_pages2_range(struct address_space *mapping,
unsigned long start, unsigned long end);
extern int write_inode_now(struct inode *, int);
extern int filemap_fdatawrite(struct address_space *);
extern int filemap_flush(struct address_space *);
extern int filemap_fdatawait_keep_errors(struct address_space *mapping);
extern int filemap_fdatawait_range(struct address_space *, loff_t lstart,
loff_t lend);
extern int filemap_fdatawait_range_keep_errors(struct address_space *mapping,
loff_t start_byte, loff_t end_byte);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int filemap_fdatawait(struct address_space *mapping)
{
return filemap_fdatawait_range(mapping, 0, ((long long)(~0ULL>>1)));
}
extern bool filemap_range_has_page(struct address_space *, loff_t lstart,
loff_t lend);
extern int __attribute__((warn_unused_result)) file_fdatawait_range(struct file *file, loff_t lstart,
loff_t lend);
extern int filemap_write_and_wait(struct address_space *mapping);
extern int filemap_write_and_wait_range(struct address_space *mapping,
loff_t lstart, loff_t lend);
extern int __filemap_fdatawrite_range(struct address_space *mapping,
loff_t start, loff_t end, int sync_mode);
extern int filemap_fdatawrite_range(struct address_space *mapping,
loff_t start, loff_t end);
extern int filemap_check_errors(struct address_space *mapping);
extern void __filemap_set_wb_err(struct address_space *mapping, int err);
extern int __attribute__((warn_unused_result)) file_fdatawait_range(struct file *file, loff_t lstart,
loff_t lend);
extern int __attribute__((warn_unused_result)) file_check_and_advance_wb_err(struct file *file);
extern int __attribute__((warn_unused_result)) file_write_and_wait_range(struct file *file,
loff_t start, loff_t end);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int file_write_and_wait(struct file *file)
{
return file_write_and_wait_range(file, 0, ((long long)(~0ULL>>1)));
}
# 2680 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void filemap_set_wb_err(struct address_space *mapping, int err)
{
if (__builtin_expect(!!(err), 0))
__filemap_set_wb_err(mapping, err);
}
# 2697 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int filemap_check_wb_err(struct address_space *mapping,
errseq_t since)
{
return errseq_check(&mapping->wb_err, since);
}
# 2710 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) errseq_t filemap_sample_wb_err(struct address_space *mapping)
{
return errseq_sample(&mapping->wb_err);
}
extern int vfs_fsync_range(struct file *file, loff_t start, loff_t end,
int datasync);
extern int vfs_fsync(struct file *file, int datasync);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ssize_t generic_write_sync(struct kiocb *iocb, ssize_t count)
{
if (iocb->ki_flags & (1 << 4)) {
int ret = vfs_fsync_range(iocb->ki_filp,
iocb->ki_pos - count, iocb->ki_pos - 1,
(iocb->ki_flags & (1 << 5)) ? 0 : 1);
if (ret)
return ret;
}
return count;
}
extern void emergency_sync(void);
extern void emergency_remount(void);
extern sector_t bmap(struct inode *, sector_t);
extern int notify_change(struct dentry *, struct iattr *, struct inode **);
extern int notify_change2(struct vfsmount *, struct dentry *, struct iattr *, struct inode **);
extern int inode_permission(struct inode *, int);
extern int inode_permission2(struct vfsmount *, struct inode *, int);
extern int __inode_permission(struct inode *, int);
extern int __inode_permission2(struct vfsmount *, struct inode *, int);
extern int generic_permission(struct inode *, int);
extern int __check_sticky(struct inode *dir, struct inode *inode);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool execute_ok(struct inode *inode)
{
return (inode->i_mode & (00100|00010|00001)) || (((inode->i_mode) & 00170000) == 0040000);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void file_start_write(struct file *file)
{
if (!(((file_inode(file)->i_mode) & 00170000) == 0100000))
return;
__sb_start_write(file_inode(file)->i_sb, SB_FREEZE_WRITE, true);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool file_start_write_trylock(struct file *file)
{
if (!(((file_inode(file)->i_mode) & 00170000) == 0100000))
return true;
return __sb_start_write(file_inode(file)->i_sb, SB_FREEZE_WRITE, false);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void file_end_write(struct file *file)
{
if (!(((file_inode(file)->i_mode) & 00170000) == 0100000))
return;
__sb_end_write(file_inode(file)->i_sb, SB_FREEZE_WRITE);
}
# 2793 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_write_access(struct inode *inode)
{
return atomic_inc_unless_negative(&inode->i_writecount) ? 0 : -26;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int deny_write_access(struct file *file)
{
struct inode *inode = file_inode(file);
return atomic_dec_unless_positive(&inode->i_writecount) ? 0 : -26;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void put_write_access(struct inode * inode)
{
atomic_sub(1, &inode->i_writecount);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void allow_write_access(struct file *file)
{
if (file)
atomic_add(1, &file_inode(file)->i_writecount);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool inode_is_open_for_write(const struct inode *inode)
{
return ({ union { typeof((&inode->i_writecount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&inode->i_writecount)->counter), __u.__c, sizeof((&inode->i_writecount)->counter)); else __read_once_size_nocheck(&((&inode->i_writecount)->counter), __u.__c, sizeof((&inode->i_writecount)->counter)); do { } while (0); __u.__val; }) > 0;
}
# 2827 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_readcount_dec(struct inode *inode)
{
return;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void i_readcount_inc(struct inode *inode)
{
return;
}
extern int do_pipe_flags(int *, int);
# 2851 "./include/linux/fs.h"
enum kernel_read_file_id {
READING_UNKNOWN, READING_FIRMWARE, READING_FIRMWARE_PREALLOC_BUFFER, READING_MODULE, READING_KEXEC_IMAGE, READING_KEXEC_INITRAMFS, READING_POLICY, READING_MAX_ID,
};
static const char * const kernel_read_file_str[] = {
"unknown", "firmware", "firmware", "kernel-module", "kexec-image", "kexec-initramfs", "security-policy", "",
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const char *kernel_read_file_id_str(enum kernel_read_file_id id)
{
if ((unsigned)id >= READING_MAX_ID)
return kernel_read_file_str[READING_UNKNOWN];
return kernel_read_file_str[id];
}
extern int kernel_read_file(struct file *, void **, loff_t *, loff_t,
enum kernel_read_file_id);
extern int kernel_read_file_from_path(const char *, void **, loff_t *, loff_t,
enum kernel_read_file_id);
extern int kernel_read_file_from_fd(int, void **, loff_t *, loff_t,
enum kernel_read_file_id);
extern ssize_t kernel_read(struct file *, void *, size_t, loff_t *);
extern ssize_t kernel_write(struct file *, const void *, size_t, loff_t *);
extern ssize_t __kernel_write(struct file *, const void *, size_t, loff_t *);
extern struct file * open_exec(const char *);
extern bool is_subdir(struct dentry *, struct dentry *);
extern bool path_is_under(const struct path *, const struct path *);
extern char *file_path(struct file *, char *, int);
extern loff_t default_llseek(struct file *file, loff_t offset, int whence);
extern loff_t vfs_llseek(struct file *file, loff_t offset, int whence);
extern int inode_init_always(struct super_block *, struct inode *);
extern void inode_init_once(struct inode *);
extern void address_space_init_once(struct address_space *mapping);
extern struct inode * igrab(struct inode *);
extern ino_t iunique(struct super_block *, ino_t);
extern int inode_needs_sync(struct inode *inode);
extern int generic_delete_inode(struct inode *inode);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int generic_drop_inode(struct inode *inode)
{
return !inode->i_nlink || inode_unhashed(inode);
}
extern struct inode *ilookup5_nowait(struct super_block *sb,
unsigned long hashval, int (*test)(struct inode *, void *),
void *data);
extern struct inode *ilookup5(struct super_block *sb, unsigned long hashval,
int (*test)(struct inode *, void *), void *data);
extern struct inode *ilookup(struct super_block *sb, unsigned long ino);
extern struct inode * iget5_locked(struct super_block *, unsigned long, int (*test)(struct inode *, void *), int (*set)(struct inode *, void *), void *);
extern struct inode * iget_locked(struct super_block *, unsigned long);
extern struct inode *find_inode_nowait(struct super_block *,
unsigned long,
int (*match)(struct inode *,
unsigned long, void *),
void *data);
extern int insert_inode_locked4(struct inode *, unsigned long, int (*test)(struct inode *, void *), void *);
extern int insert_inode_locked(struct inode *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void lockdep_annotate_inode_mutex_key(struct inode *inode) { };
extern void unlock_new_inode(struct inode *);
extern unsigned int get_next_ino(void);
extern void evict_inodes(struct super_block *sb);
extern void __iget(struct inode * inode);
extern void iget_failed(struct inode *);
extern void clear_inode(struct inode *);
extern void __destroy_inode(struct inode *);
extern struct inode *new_inode_pseudo(struct super_block *sb);
extern struct inode *new_inode(struct super_block *sb);
extern void free_inode_nonrcu(struct inode *inode);
extern int should_remove_suid(struct dentry *);
extern int file_remove_privs(struct file *);
extern void __insert_inode_hash(struct inode *, unsigned long hashval);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void insert_inode_hash(struct inode *inode)
{
__insert_inode_hash(inode, inode->i_ino);
}
extern void __remove_inode_hash(struct inode *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void remove_inode_hash(struct inode *inode)
{
if (!inode_unhashed(inode) && !hlist_fake(&inode->i_hash))
__remove_inode_hash(inode);
}
extern void inode_sb_list_add(struct inode *inode);
extern int bdev_read_only(struct block_device *);
extern int set_blocksize(struct block_device *, int);
extern int sb_set_blocksize(struct super_block *, int);
extern int sb_min_blocksize(struct super_block *, int);
extern int generic_file_mmap(struct file *, struct vm_area_struct *);
extern int generic_file_readonly_mmap(struct file *, struct vm_area_struct *);
extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *);
extern ssize_t generic_file_read_iter(struct kiocb *, struct iov_iter *);
extern ssize_t __generic_file_write_iter(struct kiocb *, struct iov_iter *);
extern ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *);
extern ssize_t generic_file_direct_write(struct kiocb *, struct iov_iter *);
extern ssize_t generic_perform_write(struct file *, struct iov_iter *, loff_t);
ssize_t vfs_iter_read(struct file *file, struct iov_iter *iter, loff_t *ppos,
rwf_t flags);
ssize_t vfs_iter_write(struct file *file, struct iov_iter *iter, loff_t *ppos,
rwf_t flags);
extern ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to);
extern ssize_t blkdev_write_iter(struct kiocb *iocb, struct iov_iter *from);
extern int blkdev_fsync(struct file *filp, loff_t start, loff_t end,
int datasync);
extern void block_sync_page(struct page *page);
extern ssize_t generic_file_splice_read(struct file *, loff_t *,
struct pipe_inode_info *, size_t, unsigned int);
extern ssize_t iter_file_splice_write(struct pipe_inode_info *,
struct file *, loff_t *, size_t, unsigned int);
extern ssize_t generic_splice_sendpage(struct pipe_inode_info *pipe,
struct file *out, loff_t *, size_t len, unsigned int flags);
extern long do_splice_direct(struct file *in, loff_t *ppos, struct file *out,
loff_t *opos, size_t len, unsigned int flags);
extern void
file_ra_state_init(struct file_ra_state *ra, struct address_space *mapping);
extern loff_t noop_llseek(struct file *file, loff_t offset, int whence);
extern loff_t no_llseek(struct file *file, loff_t offset, int whence);
extern loff_t vfs_setpos(struct file *file, loff_t offset, loff_t maxsize);
extern loff_t generic_file_llseek(struct file *file, loff_t offset, int whence);
extern loff_t generic_file_llseek_size(struct file *file, loff_t offset,
int whence, loff_t maxsize, loff_t eof);
extern loff_t fixed_size_llseek(struct file *file, loff_t offset,
int whence, loff_t size);
extern loff_t no_seek_end_llseek_size(struct file *, loff_t, int, loff_t);
extern loff_t no_seek_end_llseek(struct file *, loff_t, int);
extern int generic_file_open(struct inode * inode, struct file * filp);
extern int nonseekable_open(struct inode * inode, struct file * filp);
extern int stream_open(struct inode * inode, struct file * filp);
typedef void (dio_submit_t)(struct bio *bio, struct inode *inode,
loff_t file_offset);
enum {
DIO_LOCKING = 0x01,
DIO_SKIP_HOLES = 0x02,
DIO_ASYNC_EXTEND = 0x04,
DIO_SKIP_DIO_COUNT = 0x08,
};
void dio_end_io(struct bio *bio);
void dio_warn_stale_pagecache(struct file *filp);
ssize_t __blockdev_direct_IO(struct kiocb *iocb, struct inode *inode,
struct block_device *bdev, struct iov_iter *iter,
get_block_t get_block,
dio_iodone_t end_io, dio_submit_t submit_io,
int flags);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ssize_t blockdev_direct_IO(struct kiocb *iocb,
struct inode *inode,
struct iov_iter *iter,
get_block_t get_block)
{
return __blockdev_direct_IO(iocb, inode, inode->i_sb->s_bdev, iter,
get_block, ((void *)0), ((void *)0), DIO_LOCKING | DIO_SKIP_HOLES);
}
void inode_dio_wait(struct inode *inode);
# 3054 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_dio_begin(struct inode *inode)
{
atomic_add(1, &inode->i_dio_count);
}
# 3066 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_dio_end(struct inode *inode)
{
if ((({ typeof(atomic_sub_return_relaxed(1, &inode->i_dio_count)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_sub_return_relaxed(1, &inode->i_dio_count); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == 0))
wake_up_bit(&inode->i_state, 9);
}
extern void inode_set_flags(struct inode *inode, unsigned int flags,
unsigned int mask);
extern const struct file_operations generic_ro_fops;
extern int readlink_copy(char *, int, const char *);
extern int page_readlink(struct dentry *, char *, int);
extern const char *page_get_link(struct dentry *, struct inode *,
struct delayed_call *);
extern void page_put_link(void *);
extern int __page_symlink(struct inode *inode, const char *symname, int len,
int nofs);
extern int page_symlink(struct inode *inode, const char *symname, int len);
extern const struct inode_operations page_symlink_inode_operations;
extern void kfree_link(void *);
extern void generic_fillattr(struct inode *, struct kstat *);
extern int vfs_getattr_nosec(const struct path *, struct kstat *, u32, unsigned int);
extern int vfs_getattr(const struct path *, struct kstat *, u32, unsigned int);
void __inode_add_bytes(struct inode *inode, loff_t bytes);
void inode_add_bytes(struct inode *inode, loff_t bytes);
void __inode_sub_bytes(struct inode *inode, loff_t bytes);
void inode_sub_bytes(struct inode *inode, loff_t bytes);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) loff_t __inode_get_bytes(struct inode *inode)
{
return (((loff_t)inode->i_blocks) << 9) + inode->i_bytes;
}
loff_t inode_get_bytes(struct inode *inode);
void inode_set_bytes(struct inode *inode, loff_t bytes);
const char *simple_get_link(struct dentry *, struct inode *,
struct delayed_call *);
extern const struct inode_operations simple_symlink_inode_operations;
extern int iterate_dir(struct file *, struct dir_context *);
extern int vfs_statx(int, const char *, int, struct kstat *, u32);
extern int vfs_statx_fd(unsigned int, struct kstat *, u32, unsigned int);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int vfs_stat(const char *filename, struct kstat *stat)
{
return vfs_statx(-100, filename, 0x800,
stat, 0x000007ffU);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int vfs_lstat(const char *name, struct kstat *stat)
{
return vfs_statx(-100, name, 0x100 | 0x800,
stat, 0x000007ffU);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int vfs_fstatat(int dfd, const char *filename,
struct kstat *stat, int flags)
{
return vfs_statx(dfd, filename, flags | 0x800,
stat, 0x000007ffU);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int vfs_fstat(int fd, struct kstat *stat)
{
return vfs_statx_fd(fd, stat, 0x000007ffU, 0);
}
extern const char *vfs_get_link(struct dentry *, struct delayed_call *);
extern int vfs_readlink(struct dentry *, char *, int);
extern int __generic_block_fiemap(struct inode *inode,
struct fiemap_extent_info *fieinfo,
loff_t start, loff_t len,
get_block_t *get_block);
extern int generic_block_fiemap(struct inode *inode,
struct fiemap_extent_info *fieinfo, u64 start,
u64 len, get_block_t *get_block);
extern struct file_system_type *get_filesystem(struct file_system_type *fs);
extern void put_filesystem(struct file_system_type *fs);
extern struct file_system_type *get_fs_type(const char *name);
extern struct super_block *get_super(struct block_device *);
extern struct super_block *get_super_thawed(struct block_device *);
extern struct super_block *get_super_exclusive_thawed(struct block_device *bdev);
extern struct super_block *get_active_super(struct block_device *bdev);
extern void drop_super(struct super_block *sb);
extern void drop_super_exclusive(struct super_block *sb);
extern void iterate_supers(void (*)(struct super_block *, void *), void *);
extern void iterate_supers_type(struct file_system_type *,
void (*)(struct super_block *, void *), void *);
extern int dcache_dir_open(struct inode *, struct file *);
extern int dcache_dir_close(struct inode *, struct file *);
extern loff_t dcache_dir_lseek(struct file *, loff_t, int);
extern int dcache_readdir(struct file *, struct dir_context *);
extern int simple_setattr(struct dentry *, struct iattr *);
extern int simple_getattr(const struct path *, struct kstat *, u32, unsigned int);
extern int simple_statfs(struct dentry *, struct kstatfs *);
extern int simple_open(struct inode *inode, struct file *file);
extern int simple_link(struct dentry *, struct inode *, struct dentry *);
extern int simple_unlink(struct inode *, struct dentry *);
extern int simple_rmdir(struct inode *, struct dentry *);
extern int simple_rename(struct inode *, struct dentry *,
struct inode *, struct dentry *, unsigned int);
extern int noop_fsync(struct file *, loff_t, loff_t, int);
extern int simple_empty(struct dentry *);
extern int simple_readpage(struct file *file, struct page *page);
extern int simple_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata);
extern int simple_write_end(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied,
struct page *page, void *fsdata);
extern int always_delete_dentry(const struct dentry *);
extern struct inode *alloc_anon_inode(struct super_block *);
extern int simple_nosetlease(struct file *, long, struct file_lock **, void **);
extern const struct dentry_operations simple_dentry_operations;
extern struct dentry *simple_lookup(struct inode *, struct dentry *, unsigned int flags);
extern ssize_t generic_read_dir(struct file *, char *, size_t, loff_t *);
extern const struct file_operations simple_dir_operations;
extern const struct inode_operations simple_dir_inode_operations;
extern void make_empty_dir_inode(struct inode *inode);
extern bool is_empty_dir_inode(struct inode *inode);
struct tree_descr { const char *name; const struct file_operations *ops; int mode; };
struct dentry *d_alloc_name(struct dentry *, const char *);
extern int simple_fill_super(struct super_block *, unsigned long,
const struct tree_descr *);
extern int simple_pin_fs(struct file_system_type *, struct vfsmount **mount, int *count);
extern void simple_release_fs(struct vfsmount **mount, int *count);
extern ssize_t simple_read_from_buffer(void *to, size_t count,
loff_t *ppos, const void *from, size_t available);
extern ssize_t simple_write_to_buffer(void *to, size_t available, loff_t *ppos,
const void *from, size_t count);
extern int __generic_file_fsync(struct file *, loff_t, loff_t, int);
extern int generic_file_fsync(struct file *, loff_t, loff_t, int);
extern int generic_check_addressable(unsigned, u64);
extern int generic_ci_d_hash(const struct dentry *dentry, struct qstr *str);
extern int generic_ci_d_compare(const struct dentry *dentry, unsigned int len,
const char *str, const struct qstr *name);
extern bool needs_casefold(const struct inode *dir);
extern void generic_set_encrypted_ci_d_ops(struct inode *dir,
struct dentry *dentry);
extern int buffer_migrate_page(struct address_space *,
struct page *, struct page *,
enum migrate_mode);
extern int setattr_prepare(struct dentry *, struct iattr *);
extern int inode_newsize_ok(const struct inode *, loff_t offset);
extern void setattr_copy(struct inode *inode, const struct iattr *attr);
extern int file_update_time(struct file *file);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool io_is_direct(struct file *filp)
{
return (filp->f_flags & 0200000) || ((filp->f_mapping->host)->i_flags & 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool vma_is_dax(struct vm_area_struct *vma)
{
return vma->vm_file && ((vma->vm_file->f_mapping->host)->i_flags & 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool vma_is_fsdax(struct vm_area_struct *vma)
{
struct inode *inode;
if (!vma->vm_file)
return false;
if (!vma_is_dax(vma))
return false;
inode = file_inode(vma->vm_file);
if ((((inode->i_mode) & 00170000) == 0020000))
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int iocb_flags(struct file *file)
{
int res = 0;
if (file->f_flags & 00002000)
res |= (1 << 1);
if (io_is_direct(file))
res |= (1 << 2);
if ((file->f_flags & 00010000) || (((file->f_mapping->host)->i_sb->s_flags & (16)) || ((file->f_mapping->host)->i_flags & 1)))
res |= (1 << 4);
if (file->f_flags & 04000000)
res |= (1 << 5);
return res;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kiocb_set_rw_flags(struct kiocb *ki, rwf_t flags)
{
if (__builtin_expect(!!(flags & ~((( __kernel_rwf_t)0x00000001) | (( __kernel_rwf_t)0x00000002) | (( __kernel_rwf_t)0x00000004) | (( __kernel_rwf_t)0x00000008))), 0))
return -95;
if (flags & (( __kernel_rwf_t)0x00000008)) {
if (!(ki->ki_filp->f_mode & (( fmode_t)0x8000000)))
return -95;
ki->ki_flags |= (1 << 7);
}
if (flags & (( __kernel_rwf_t)0x00000001))
ki->ki_flags |= (1 << 3);
if (flags & (( __kernel_rwf_t)0x00000002))
ki->ki_flags |= (1 << 4);
if (flags & (( __kernel_rwf_t)0x00000004))
ki->ki_flags |= ((1 << 4) | (1 << 5));
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) ino_t parent_ino(struct dentry *dentry)
{
ino_t res;
spin_lock(&dentry->d_lockref.lock);
res = dentry->d_parent->d_inode->i_ino;
spin_unlock(&dentry->d_lockref.lock);
return res;
}
struct simple_transaction_argresp {
ssize_t size;
char data[0];
};
char *simple_transaction_get(struct file *file, const char *buf,
size_t size);
ssize_t simple_transaction_read(struct file *file, char *buf,
size_t size, loff_t *pos);
int simple_transaction_release(struct inode *inode, struct file *file);
void simple_transaction_set(struct file *file, size_t n);
# 3358 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((format(printf, 1, 2)))
void __simple_attr_check_format(const char *fmt, ...)
{
}
int simple_attr_open(struct inode *inode, struct file *file,
int (*get)(void *, u64 *), int (*set)(void *, u64),
const char *fmt);
int simple_attr_release(struct inode *inode, struct file *file);
ssize_t simple_attr_read(struct file *file, char *buf,
size_t len, loff_t *ppos);
ssize_t simple_attr_write(struct file *file, const char *buf,
size_t len, loff_t *ppos);
struct ctl_table;
int proc_nr_files(struct ctl_table *table, int write,
void *buffer, size_t *lenp, loff_t *ppos);
int proc_nr_dentry(struct ctl_table *table, int write,
void *buffer, size_t *lenp, loff_t *ppos);
int proc_nr_inodes(struct ctl_table *table, int write,
void *buffer, size_t *lenp, loff_t *ppos);
int __attribute__ ((__section__(".init.text"))) get_filesystem_list(char *buf);
# 3389 "./include/linux/fs.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_sxid(umode_t mode)
{
return (mode & 0004000) || ((mode & 0002000) && (mode & 00010));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int check_sticky(struct inode *dir, struct inode *inode)
{
if (!(dir->i_mode & 0001000))
return 0;
return __check_sticky(dir, inode);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inode_has_no_xattr(struct inode *inode)
{
if (!is_sxid(inode->i_mode) && (inode->i_sb->s_flags & (1<<28)))
inode->i_flags |= 4096;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_root_inode(struct inode *inode)
{
return inode == inode->i_sb->s_root->d_inode;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool dir_emit(struct dir_context *ctx,
const char *name, int namelen,
u64 ino, unsigned type)
{
return ctx->actor(ctx, name, namelen, ctx->pos, ino, type) == 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool dir_emit_dot(struct file *file, struct dir_context *ctx)
{
return ctx->actor(ctx, ".", 1, ctx->pos,
file->f_path.dentry->d_inode->i_ino, 4) == 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool dir_emit_dotdot(struct file *file, struct dir_context *ctx)
{
return ctx->actor(ctx, "..", 2, ctx->pos,
parent_ino(file->f_path.dentry), 4) == 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool dir_emit_dots(struct file *file, struct dir_context *ctx)
{
if (ctx->pos == 0) {
if (!dir_emit_dot(file, ctx))
return false;
ctx->pos = 1;
}
if (ctx->pos == 1) {
if (!dir_emit_dotdot(file, ctx))
return false;
ctx->pos = 2;
}
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool dir_relax(struct inode *inode)
{
inode_unlock(inode);
inode_lock(inode);
return !((inode)->i_flags & 16);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool dir_relax_shared(struct inode *inode)
{
inode_unlock_shared(inode);
inode_lock_shared(inode);
return !((inode)->i_flags & 16);
}
extern bool path_noexec(const struct path *path);
extern void inode_nohighmem(struct inode *inode);
int vfs_ioc_setflags_prepare(struct inode *inode, unsigned int oldflags,
unsigned int flags);
int vfs_ioc_fssetxattr_check(struct inode *inode, const struct fsxattr *old_fa,
struct fsxattr *fa);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void simple_fill_fsxattr(struct fsxattr *fa, __u32 xflags)
{
({ void *__p = (fa); size_t __n = sizeof(*fa); if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
fa->fsx_xflags = xflags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int inode_drain_writes(struct inode *inode)
{
inode_dio_wait(inode);
return filemap_write_and_wait(inode->i_mapping);
}
# 12 "./include/linux/seq_file.h" 2
# 1 "./include/linux/cred.h" 1
# 17 "./include/linux/cred.h"
# 1 "./include/linux/key.h" 1
# 22 "./include/linux/key.h"
# 1 "./include/linux/sysctl.h" 1
# 30 "./include/linux/sysctl.h"
# 1 "./include/uapi/linux/sysctl.h" 1
# 35 "./include/uapi/linux/sysctl.h"
struct __sysctl_args {
int *name;
int nlen;
void *oldval;
size_t *oldlenp;
void *newval;
size_t newlen;
unsigned long __unused[4];
};
enum
{
CTL_KERN=1,
CTL_VM=2,
CTL_NET=3,
CTL_PROC=4,
CTL_FS=5,
CTL_DEBUG=6,
CTL_DEV=7,
CTL_BUS=8,
CTL_ABI=9,
CTL_CPU=10,
CTL_ARLAN=254,
CTL_S390DBF=5677,
CTL_SUNRPC=7249,
CTL_PM=9899,
CTL_FRV=9898,
};
enum
{
CTL_BUS_ISA=1
};
enum
{
INOTIFY_MAX_USER_INSTANCES=1,
INOTIFY_MAX_USER_WATCHES=2,
INOTIFY_MAX_QUEUED_EVENTS=3
};
enum
{
KERN_OSTYPE=1,
KERN_OSRELEASE=2,
KERN_OSREV=3,
KERN_VERSION=4,
KERN_SECUREMASK=5,
KERN_PROF=6,
KERN_NODENAME=7,
KERN_DOMAINNAME=8,
KERN_PANIC=15,
KERN_REALROOTDEV=16,
KERN_SPARC_REBOOT=21,
KERN_CTLALTDEL=22,
KERN_PRINTK=23,
KERN_NAMETRANS=24,
KERN_PPC_HTABRECLAIM=25,
KERN_PPC_ZEROPAGED=26,
KERN_PPC_POWERSAVE_NAP=27,
KERN_MODPROBE=28,
KERN_SG_BIG_BUFF=29,
KERN_ACCT=30,
KERN_PPC_L2CR=31,
KERN_RTSIGNR=32,
KERN_RTSIGMAX=33,
KERN_SHMMAX=34,
KERN_MSGMAX=35,
KERN_MSGMNB=36,
KERN_MSGPOOL=37,
KERN_SYSRQ=38,
KERN_MAX_THREADS=39,
KERN_RANDOM=40,
KERN_SHMALL=41,
KERN_MSGMNI=42,
KERN_SEM=43,
KERN_SPARC_STOP_A=44,
KERN_SHMMNI=45,
KERN_OVERFLOWUID=46,
KERN_OVERFLOWGID=47,
KERN_SHMPATH=48,
KERN_HOTPLUG=49,
KERN_IEEE_EMULATION_WARNINGS=50,
KERN_S390_USER_DEBUG_LOGGING=51,
KERN_CORE_USES_PID=52,
KERN_TAINTED=53,
KERN_CADPID=54,
KERN_PIDMAX=55,
KERN_CORE_PATTERN=56,
KERN_PANIC_ON_OOPS=57,
KERN_HPPA_PWRSW=58,
KERN_HPPA_UNALIGNED=59,
KERN_PRINTK_RATELIMIT=60,
KERN_PRINTK_RATELIMIT_BURST=61,
KERN_PTY=62,
KERN_NGROUPS_MAX=63,
KERN_SPARC_SCONS_PWROFF=64,
KERN_HZ_TIMER=65,
KERN_UNKNOWN_NMI_PANIC=66,
KERN_BOOTLOADER_TYPE=67,
KERN_RANDOMIZE=68,
KERN_SETUID_DUMPABLE=69,
KERN_SPIN_RETRY=70,
KERN_ACPI_VIDEO_FLAGS=71,
KERN_IA64_UNALIGNED=72,
KERN_COMPAT_LOG=73,
KERN_MAX_LOCK_DEPTH=74,
KERN_NMI_WATCHDOG=75,
KERN_PANIC_ON_NMI=76,
KERN_PANIC_ON_WARN=77,
};
enum
{
VM_UNUSED1=1,
VM_UNUSED2=2,
VM_UNUSED3=3,
VM_UNUSED4=4,
VM_OVERCOMMIT_MEMORY=5,
VM_UNUSED5=6,
VM_UNUSED7=7,
VM_UNUSED8=8,
VM_UNUSED9=9,
VM_PAGE_CLUSTER=10,
VM_DIRTY_BACKGROUND=11,
VM_DIRTY_RATIO=12,
VM_DIRTY_WB_CS=13,
VM_DIRTY_EXPIRE_CS=14,
VM_NR_PDFLUSH_THREADS=15,
VM_OVERCOMMIT_RATIO=16,
VM_PAGEBUF=17,
VM_HUGETLB_PAGES=18,
VM_SWAPPINESS=19,
VM_LOWMEM_RESERVE_RATIO=20,
VM_MIN_FREE_KBYTES=21,
VM_MAX_MAP_COUNT=22,
VM_LAPTOP_MODE=23,
VM_BLOCK_DUMP=24,
VM_HUGETLB_GROUP=25,
VM_VFS_CACHE_PRESSURE=26,
VM_LEGACY_VA_LAYOUT=27,
VM_SWAP_TOKEN_TIMEOUT=28,
VM_DROP_PAGECACHE=29,
VM_PERCPU_PAGELIST_FRACTION=30,
VM_ZONE_RECLAIM_MODE=31,
VM_MIN_UNMAPPED=32,
VM_PANIC_ON_OOM=33,
VM_VDSO_ENABLED=34,
VM_MIN_SLAB=35,
};
enum
{
NET_CORE=1,
NET_ETHER=2,
NET_802=3,
NET_UNIX=4,
NET_IPV4=5,
NET_IPX=6,
NET_ATALK=7,
NET_NETROM=8,
NET_AX25=9,
NET_BRIDGE=10,
NET_ROSE=11,
NET_IPV6=12,
NET_X25=13,
NET_TR=14,
NET_DECNET=15,
NET_ECONET=16,
NET_SCTP=17,
NET_LLC=18,
NET_NETFILTER=19,
NET_DCCP=20,
NET_IRDA=412,
};
enum
{
RANDOM_POOLSIZE=1,
RANDOM_ENTROPY_COUNT=2,
RANDOM_READ_THRESH=3,
RANDOM_WRITE_THRESH=4,
RANDOM_BOOT_ID=5,
RANDOM_UUID=6
};
enum
{
PTY_MAX=1,
PTY_NR=2
};
enum
{
BUS_ISA_MEM_BASE=1,
BUS_ISA_PORT_BASE=2,
BUS_ISA_PORT_SHIFT=3
};
enum
{
NET_CORE_WMEM_MAX=1,
NET_CORE_RMEM_MAX=2,
NET_CORE_WMEM_DEFAULT=3,
NET_CORE_RMEM_DEFAULT=4,
NET_CORE_MAX_BACKLOG=6,
NET_CORE_FASTROUTE=7,
NET_CORE_MSG_COST=8,
NET_CORE_MSG_BURST=9,
NET_CORE_OPTMEM_MAX=10,
NET_CORE_HOT_LIST_LENGTH=11,
NET_CORE_DIVERT_VERSION=12,
NET_CORE_NO_CONG_THRESH=13,
NET_CORE_NO_CONG=14,
NET_CORE_LO_CONG=15,
NET_CORE_MOD_CONG=16,
NET_CORE_DEV_WEIGHT=17,
NET_CORE_SOMAXCONN=18,
NET_CORE_BUDGET=19,
NET_CORE_AEVENT_ETIME=20,
NET_CORE_AEVENT_RSEQTH=21,
NET_CORE_WARNINGS=22,
};
enum
{
NET_UNIX_DESTROY_DELAY=1,
NET_UNIX_DELETE_DELAY=2,
NET_UNIX_MAX_DGRAM_QLEN=3,
};
enum
{
NET_NF_CONNTRACK_MAX=1,
NET_NF_CONNTRACK_TCP_TIMEOUT_SYN_SENT=2,
NET_NF_CONNTRACK_TCP_TIMEOUT_SYN_RECV=3,
NET_NF_CONNTRACK_TCP_TIMEOUT_ESTABLISHED=4,
NET_NF_CONNTRACK_TCP_TIMEOUT_FIN_WAIT=5,
NET_NF_CONNTRACK_TCP_TIMEOUT_CLOSE_WAIT=6,
NET_NF_CONNTRACK_TCP_TIMEOUT_LAST_ACK=7,
NET_NF_CONNTRACK_TCP_TIMEOUT_TIME_WAIT=8,
NET_NF_CONNTRACK_TCP_TIMEOUT_CLOSE=9,
NET_NF_CONNTRACK_UDP_TIMEOUT=10,
NET_NF_CONNTRACK_UDP_TIMEOUT_STREAM=11,
NET_NF_CONNTRACK_ICMP_TIMEOUT=12,
NET_NF_CONNTRACK_GENERIC_TIMEOUT=13,
NET_NF_CONNTRACK_BUCKETS=14,
NET_NF_CONNTRACK_LOG_INVALID=15,
NET_NF_CONNTRACK_TCP_TIMEOUT_MAX_RETRANS=16,
NET_NF_CONNTRACK_TCP_LOOSE=17,
NET_NF_CONNTRACK_TCP_BE_LIBERAL=18,
NET_NF_CONNTRACK_TCP_MAX_RETRANS=19,
NET_NF_CONNTRACK_SCTP_TIMEOUT_CLOSED=20,
NET_NF_CONNTRACK_SCTP_TIMEOUT_COOKIE_WAIT=21,
NET_NF_CONNTRACK_SCTP_TIMEOUT_COOKIE_ECHOED=22,
NET_NF_CONNTRACK_SCTP_TIMEOUT_ESTABLISHED=23,
NET_NF_CONNTRACK_SCTP_TIMEOUT_SHUTDOWN_SENT=24,
NET_NF_CONNTRACK_SCTP_TIMEOUT_SHUTDOWN_RECD=25,
NET_NF_CONNTRACK_SCTP_TIMEOUT_SHUTDOWN_ACK_SENT=26,
NET_NF_CONNTRACK_COUNT=27,
NET_NF_CONNTRACK_ICMPV6_TIMEOUT=28,
NET_NF_CONNTRACK_FRAG6_TIMEOUT=29,
NET_NF_CONNTRACK_FRAG6_LOW_THRESH=30,
NET_NF_CONNTRACK_FRAG6_HIGH_THRESH=31,
NET_NF_CONNTRACK_CHECKSUM=32,
};
enum
{
NET_IPV4_FORWARD=8,
NET_IPV4_DYNADDR=9,
NET_IPV4_CONF=16,
NET_IPV4_NEIGH=17,
NET_IPV4_ROUTE=18,
NET_IPV4_FIB_HASH=19,
NET_IPV4_NETFILTER=20,
NET_IPV4_TCP_TIMESTAMPS=33,
NET_IPV4_TCP_WINDOW_SCALING=34,
NET_IPV4_TCP_SACK=35,
NET_IPV4_TCP_RETRANS_COLLAPSE=36,
NET_IPV4_DEFAULT_TTL=37,
NET_IPV4_AUTOCONFIG=38,
NET_IPV4_NO_PMTU_DISC=39,
NET_IPV4_TCP_SYN_RETRIES=40,
NET_IPV4_IPFRAG_HIGH_THRESH=41,
NET_IPV4_IPFRAG_LOW_THRESH=42,
NET_IPV4_IPFRAG_TIME=43,
NET_IPV4_TCP_MAX_KA_PROBES=44,
NET_IPV4_TCP_KEEPALIVE_TIME=45,
NET_IPV4_TCP_KEEPALIVE_PROBES=46,
NET_IPV4_TCP_RETRIES1=47,
NET_IPV4_TCP_RETRIES2=48,
NET_IPV4_TCP_FIN_TIMEOUT=49,
NET_IPV4_IP_MASQ_DEBUG=50,
NET_TCP_SYNCOOKIES=51,
NET_TCP_STDURG=52,
NET_TCP_RFC1337=53,
NET_TCP_SYN_TAILDROP=54,
NET_TCP_MAX_SYN_BACKLOG=55,
NET_IPV4_LOCAL_PORT_RANGE=56,
NET_IPV4_ICMP_ECHO_IGNORE_ALL=57,
NET_IPV4_ICMP_ECHO_IGNORE_BROADCASTS=58,
NET_IPV4_ICMP_SOURCEQUENCH_RATE=59,
NET_IPV4_ICMP_DESTUNREACH_RATE=60,
NET_IPV4_ICMP_TIMEEXCEED_RATE=61,
NET_IPV4_ICMP_PARAMPROB_RATE=62,
NET_IPV4_ICMP_ECHOREPLY_RATE=63,
NET_IPV4_ICMP_IGNORE_BOGUS_ERROR_RESPONSES=64,
NET_IPV4_IGMP_MAX_MEMBERSHIPS=65,
NET_TCP_TW_RECYCLE=66,
NET_IPV4_ALWAYS_DEFRAG=67,
NET_IPV4_TCP_KEEPALIVE_INTVL=68,
NET_IPV4_INET_PEER_THRESHOLD=69,
NET_IPV4_INET_PEER_MINTTL=70,
NET_IPV4_INET_PEER_MAXTTL=71,
NET_IPV4_INET_PEER_GC_MINTIME=72,
NET_IPV4_INET_PEER_GC_MAXTIME=73,
NET_TCP_ORPHAN_RETRIES=74,
NET_TCP_ABORT_ON_OVERFLOW=75,
NET_TCP_SYNACK_RETRIES=76,
NET_TCP_MAX_ORPHANS=77,
NET_TCP_MAX_TW_BUCKETS=78,
NET_TCP_FACK=79,
NET_TCP_REORDERING=80,
NET_TCP_ECN=81,
NET_TCP_DSACK=82,
NET_TCP_MEM=83,
NET_TCP_WMEM=84,
NET_TCP_RMEM=85,
NET_TCP_APP_WIN=86,
NET_TCP_ADV_WIN_SCALE=87,
NET_IPV4_NONLOCAL_BIND=88,
NET_IPV4_ICMP_RATELIMIT=89,
NET_IPV4_ICMP_RATEMASK=90,
NET_TCP_TW_REUSE=91,
NET_TCP_FRTO=92,
NET_TCP_LOW_LATENCY=93,
NET_IPV4_IPFRAG_SECRET_INTERVAL=94,
NET_IPV4_IGMP_MAX_MSF=96,
NET_TCP_NO_METRICS_SAVE=97,
NET_TCP_DEFAULT_WIN_SCALE=105,
NET_TCP_MODERATE_RCVBUF=106,
NET_TCP_TSO_WIN_DIVISOR=107,
NET_TCP_BIC_BETA=108,
NET_IPV4_ICMP_ERRORS_USE_INBOUND_IFADDR=109,
NET_TCP_CONG_CONTROL=110,
NET_TCP_ABC=111,
NET_IPV4_IPFRAG_MAX_DIST=112,
NET_TCP_MTU_PROBING=113,
NET_TCP_BASE_MSS=114,
NET_IPV4_TCP_WORKAROUND_SIGNED_WINDOWS=115,
NET_TCP_DMA_COPYBREAK=116,
NET_TCP_SLOW_START_AFTER_IDLE=117,
NET_CIPSOV4_CACHE_ENABLE=118,
NET_CIPSOV4_CACHE_BUCKET_SIZE=119,
NET_CIPSOV4_RBM_OPTFMT=120,
NET_CIPSOV4_RBM_STRICTVALID=121,
NET_TCP_AVAIL_CONG_CONTROL=122,
NET_TCP_ALLOWED_CONG_CONTROL=123,
NET_TCP_MAX_SSTHRESH=124,
NET_TCP_FRTO_RESPONSE=125,
};
enum {
NET_IPV4_ROUTE_FLUSH=1,
NET_IPV4_ROUTE_MIN_DELAY=2,
NET_IPV4_ROUTE_MAX_DELAY=3,
NET_IPV4_ROUTE_GC_THRESH=4,
NET_IPV4_ROUTE_MAX_SIZE=5,
NET_IPV4_ROUTE_GC_MIN_INTERVAL=6,
NET_IPV4_ROUTE_GC_TIMEOUT=7,
NET_IPV4_ROUTE_GC_INTERVAL=8,
NET_IPV4_ROUTE_REDIRECT_LOAD=9,
NET_IPV4_ROUTE_REDIRECT_NUMBER=10,
NET_IPV4_ROUTE_REDIRECT_SILENCE=11,
NET_IPV4_ROUTE_ERROR_COST=12,
NET_IPV4_ROUTE_ERROR_BURST=13,
NET_IPV4_ROUTE_GC_ELASTICITY=14,
NET_IPV4_ROUTE_MTU_EXPIRES=15,
NET_IPV4_ROUTE_MIN_PMTU=16,
NET_IPV4_ROUTE_MIN_ADVMSS=17,
NET_IPV4_ROUTE_SECRET_INTERVAL=18,
NET_IPV4_ROUTE_GC_MIN_INTERVAL_MS=19,
};
enum
{
NET_PROTO_CONF_ALL=-2,
NET_PROTO_CONF_DEFAULT=-3
};
enum
{
NET_IPV4_CONF_FORWARDING=1,
NET_IPV4_CONF_MC_FORWARDING=2,
NET_IPV4_CONF_PROXY_ARP=3,
NET_IPV4_CONF_ACCEPT_REDIRECTS=4,
NET_IPV4_CONF_SECURE_REDIRECTS=5,
NET_IPV4_CONF_SEND_REDIRECTS=6,
NET_IPV4_CONF_SHARED_MEDIA=7,
NET_IPV4_CONF_RP_FILTER=8,
NET_IPV4_CONF_ACCEPT_SOURCE_ROUTE=9,
NET_IPV4_CONF_BOOTP_RELAY=10,
NET_IPV4_CONF_LOG_MARTIANS=11,
NET_IPV4_CONF_TAG=12,
NET_IPV4_CONF_ARPFILTER=13,
NET_IPV4_CONF_MEDIUM_ID=14,
NET_IPV4_CONF_NOXFRM=15,
NET_IPV4_CONF_NOPOLICY=16,
NET_IPV4_CONF_FORCE_IGMP_VERSION=17,
NET_IPV4_CONF_ARP_ANNOUNCE=18,
NET_IPV4_CONF_ARP_IGNORE=19,
NET_IPV4_CONF_PROMOTE_SECONDARIES=20,
NET_IPV4_CONF_ARP_ACCEPT=21,
NET_IPV4_CONF_ARP_NOTIFY=22,
};
enum
{
NET_IPV4_NF_CONNTRACK_MAX=1,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_SYN_SENT=2,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_SYN_RECV=3,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_ESTABLISHED=4,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_FIN_WAIT=5,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_CLOSE_WAIT=6,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_LAST_ACK=7,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_TIME_WAIT=8,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_CLOSE=9,
NET_IPV4_NF_CONNTRACK_UDP_TIMEOUT=10,
NET_IPV4_NF_CONNTRACK_UDP_TIMEOUT_STREAM=11,
NET_IPV4_NF_CONNTRACK_ICMP_TIMEOUT=12,
NET_IPV4_NF_CONNTRACK_GENERIC_TIMEOUT=13,
NET_IPV4_NF_CONNTRACK_BUCKETS=14,
NET_IPV4_NF_CONNTRACK_LOG_INVALID=15,
NET_IPV4_NF_CONNTRACK_TCP_TIMEOUT_MAX_RETRANS=16,
NET_IPV4_NF_CONNTRACK_TCP_LOOSE=17,
NET_IPV4_NF_CONNTRACK_TCP_BE_LIBERAL=18,
NET_IPV4_NF_CONNTRACK_TCP_MAX_RETRANS=19,
NET_IPV4_NF_CONNTRACK_SCTP_TIMEOUT_CLOSED=20,
NET_IPV4_NF_CONNTRACK_SCTP_TIMEOUT_COOKIE_WAIT=21,
NET_IPV4_NF_CONNTRACK_SCTP_TIMEOUT_COOKIE_ECHOED=22,
NET_IPV4_NF_CONNTRACK_SCTP_TIMEOUT_ESTABLISHED=23,
NET_IPV4_NF_CONNTRACK_SCTP_TIMEOUT_SHUTDOWN_SENT=24,
NET_IPV4_NF_CONNTRACK_SCTP_TIMEOUT_SHUTDOWN_RECD=25,
NET_IPV4_NF_CONNTRACK_SCTP_TIMEOUT_SHUTDOWN_ACK_SENT=26,
NET_IPV4_NF_CONNTRACK_COUNT=27,
NET_IPV4_NF_CONNTRACK_CHECKSUM=28,
};
enum {
NET_IPV6_CONF=16,
NET_IPV6_NEIGH=17,
NET_IPV6_ROUTE=18,
NET_IPV6_ICMP=19,
NET_IPV6_BINDV6ONLY=20,
NET_IPV6_IP6FRAG_HIGH_THRESH=21,
NET_IPV6_IP6FRAG_LOW_THRESH=22,
NET_IPV6_IP6FRAG_TIME=23,
NET_IPV6_IP6FRAG_SECRET_INTERVAL=24,
NET_IPV6_MLD_MAX_MSF=25,
};
enum {
NET_IPV6_ROUTE_FLUSH=1,
NET_IPV6_ROUTE_GC_THRESH=2,
NET_IPV6_ROUTE_MAX_SIZE=3,
NET_IPV6_ROUTE_GC_MIN_INTERVAL=4,
NET_IPV6_ROUTE_GC_TIMEOUT=5,
NET_IPV6_ROUTE_GC_INTERVAL=6,
NET_IPV6_ROUTE_GC_ELASTICITY=7,
NET_IPV6_ROUTE_MTU_EXPIRES=8,
NET_IPV6_ROUTE_MIN_ADVMSS=9,
NET_IPV6_ROUTE_GC_MIN_INTERVAL_MS=10
};
enum {
NET_IPV6_FORWARDING=1,
NET_IPV6_HOP_LIMIT=2,
NET_IPV6_MTU=3,
NET_IPV6_ACCEPT_RA=4,
NET_IPV6_ACCEPT_REDIRECTS=5,
NET_IPV6_AUTOCONF=6,
NET_IPV6_DAD_TRANSMITS=7,
NET_IPV6_RTR_SOLICITS=8,
NET_IPV6_RTR_SOLICIT_INTERVAL=9,
NET_IPV6_RTR_SOLICIT_DELAY=10,
NET_IPV6_USE_TEMPADDR=11,
NET_IPV6_TEMP_VALID_LFT=12,
NET_IPV6_TEMP_PREFERED_LFT=13,
NET_IPV6_REGEN_MAX_RETRY=14,
NET_IPV6_MAX_DESYNC_FACTOR=15,
NET_IPV6_MAX_ADDRESSES=16,
NET_IPV6_FORCE_MLD_VERSION=17,
NET_IPV6_ACCEPT_RA_DEFRTR=18,
NET_IPV6_ACCEPT_RA_PINFO=19,
NET_IPV6_ACCEPT_RA_RTR_PREF=20,
NET_IPV6_RTR_PROBE_INTERVAL=21,
NET_IPV6_ACCEPT_RA_RT_INFO_MAX_PLEN=22,
NET_IPV6_PROXY_NDP=23,
NET_IPV6_ACCEPT_SOURCE_ROUTE=25,
NET_IPV6_ACCEPT_RA_FROM_LOCAL=26,
NET_IPV6_ACCEPT_RA_RT_INFO_MIN_PLEN=27,
__NET_IPV6_MAX
};
enum {
NET_IPV6_ICMP_RATELIMIT=1
};
enum {
NET_NEIGH_MCAST_SOLICIT=1,
NET_NEIGH_UCAST_SOLICIT=2,
NET_NEIGH_APP_SOLICIT=3,
NET_NEIGH_RETRANS_TIME=4,
NET_NEIGH_REACHABLE_TIME=5,
NET_NEIGH_DELAY_PROBE_TIME=6,
NET_NEIGH_GC_STALE_TIME=7,
NET_NEIGH_UNRES_QLEN=8,
NET_NEIGH_PROXY_QLEN=9,
NET_NEIGH_ANYCAST_DELAY=10,
NET_NEIGH_PROXY_DELAY=11,
NET_NEIGH_LOCKTIME=12,
NET_NEIGH_GC_INTERVAL=13,
NET_NEIGH_GC_THRESH1=14,
NET_NEIGH_GC_THRESH2=15,
NET_NEIGH_GC_THRESH3=16,
NET_NEIGH_RETRANS_TIME_MS=17,
NET_NEIGH_REACHABLE_TIME_MS=18,
};
enum {
NET_DCCP_DEFAULT=1,
};
enum {
NET_IPX_PPROP_BROADCASTING=1,
NET_IPX_FORWARDING=2
};
enum {
NET_LLC2=1,
NET_LLC_STATION=2,
};
enum {
NET_LLC2_TIMEOUT=1,
};
enum {
NET_LLC_STATION_ACK_TIMEOUT=1,
};
enum {
NET_LLC2_ACK_TIMEOUT=1,
NET_LLC2_P_TIMEOUT=2,
NET_LLC2_REJ_TIMEOUT=3,
NET_LLC2_BUSY_TIMEOUT=4,
};
enum {
NET_ATALK_AARP_EXPIRY_TIME=1,
NET_ATALK_AARP_TICK_TIME=2,
NET_ATALK_AARP_RETRANSMIT_LIMIT=3,
NET_ATALK_AARP_RESOLVE_TIME=4
};
enum {
NET_NETROM_DEFAULT_PATH_QUALITY=1,
NET_NETROM_OBSOLESCENCE_COUNT_INITIALISER=2,
NET_NETROM_NETWORK_TTL_INITIALISER=3,
NET_NETROM_TRANSPORT_TIMEOUT=4,
NET_NETROM_TRANSPORT_MAXIMUM_TRIES=5,
NET_NETROM_TRANSPORT_ACKNOWLEDGE_DELAY=6,
NET_NETROM_TRANSPORT_BUSY_DELAY=7,
NET_NETROM_TRANSPORT_REQUESTED_WINDOW_SIZE=8,
NET_NETROM_TRANSPORT_NO_ACTIVITY_TIMEOUT=9,
NET_NETROM_ROUTING_CONTROL=10,
NET_NETROM_LINK_FAILS_COUNT=11,
NET_NETROM_RESET=12
};
enum {
NET_AX25_IP_DEFAULT_MODE=1,
NET_AX25_DEFAULT_MODE=2,
NET_AX25_BACKOFF_TYPE=3,
NET_AX25_CONNECT_MODE=4,
NET_AX25_STANDARD_WINDOW=5,
NET_AX25_EXTENDED_WINDOW=6,
NET_AX25_T1_TIMEOUT=7,
NET_AX25_T2_TIMEOUT=8,
NET_AX25_T3_TIMEOUT=9,
NET_AX25_IDLE_TIMEOUT=10,
NET_AX25_N2=11,
NET_AX25_PACLEN=12,
NET_AX25_PROTOCOL=13,
NET_AX25_DAMA_SLAVE_TIMEOUT=14
};
enum {
NET_ROSE_RESTART_REQUEST_TIMEOUT=1,
NET_ROSE_CALL_REQUEST_TIMEOUT=2,
NET_ROSE_RESET_REQUEST_TIMEOUT=3,
NET_ROSE_CLEAR_REQUEST_TIMEOUT=4,
NET_ROSE_ACK_HOLD_BACK_TIMEOUT=5,
NET_ROSE_ROUTING_CONTROL=6,
NET_ROSE_LINK_FAIL_TIMEOUT=7,
NET_ROSE_MAX_VCS=8,
NET_ROSE_WINDOW_SIZE=9,
NET_ROSE_NO_ACTIVITY_TIMEOUT=10
};
enum {
NET_X25_RESTART_REQUEST_TIMEOUT=1,
NET_X25_CALL_REQUEST_TIMEOUT=2,
NET_X25_RESET_REQUEST_TIMEOUT=3,
NET_X25_CLEAR_REQUEST_TIMEOUT=4,
NET_X25_ACK_HOLD_BACK_TIMEOUT=5,
NET_X25_FORWARD=6
};
enum
{
NET_TR_RIF_TIMEOUT=1
};
enum {
NET_DECNET_NODE_TYPE = 1,
NET_DECNET_NODE_ADDRESS = 2,
NET_DECNET_NODE_NAME = 3,
NET_DECNET_DEFAULT_DEVICE = 4,
NET_DECNET_TIME_WAIT = 5,
NET_DECNET_DN_COUNT = 6,
NET_DECNET_DI_COUNT = 7,
NET_DECNET_DR_COUNT = 8,
NET_DECNET_DST_GC_INTERVAL = 9,
NET_DECNET_CONF = 10,
NET_DECNET_NO_FC_MAX_CWND = 11,
NET_DECNET_MEM = 12,
NET_DECNET_RMEM = 13,
NET_DECNET_WMEM = 14,
NET_DECNET_DEBUG_LEVEL = 255
};
enum {
NET_DECNET_CONF_LOOPBACK = -2,
NET_DECNET_CONF_DDCMP = -3,
NET_DECNET_CONF_PPP = -4,
NET_DECNET_CONF_X25 = -5,
NET_DECNET_CONF_GRE = -6,
NET_DECNET_CONF_ETHER = -7
};
enum {
NET_DECNET_CONF_DEV_PRIORITY = 1,
NET_DECNET_CONF_DEV_T1 = 2,
NET_DECNET_CONF_DEV_T2 = 3,
NET_DECNET_CONF_DEV_T3 = 4,
NET_DECNET_CONF_DEV_FORWARDING = 5,
NET_DECNET_CONF_DEV_BLKSIZE = 6,
NET_DECNET_CONF_DEV_STATE = 7
};
enum {
NET_SCTP_RTO_INITIAL = 1,
NET_SCTP_RTO_MIN = 2,
NET_SCTP_RTO_MAX = 3,
NET_SCTP_RTO_ALPHA = 4,
NET_SCTP_RTO_BETA = 5,
NET_SCTP_VALID_COOKIE_LIFE = 6,
NET_SCTP_ASSOCIATION_MAX_RETRANS = 7,
NET_SCTP_PATH_MAX_RETRANS = 8,
NET_SCTP_MAX_INIT_RETRANSMITS = 9,
NET_SCTP_HB_INTERVAL = 10,
NET_SCTP_PRESERVE_ENABLE = 11,
NET_SCTP_MAX_BURST = 12,
NET_SCTP_ADDIP_ENABLE = 13,
NET_SCTP_PRSCTP_ENABLE = 14,
NET_SCTP_SNDBUF_POLICY = 15,
NET_SCTP_SACK_TIMEOUT = 16,
NET_SCTP_RCVBUF_POLICY = 17,
};
enum {
NET_BRIDGE_NF_CALL_ARPTABLES = 1,
NET_BRIDGE_NF_CALL_IPTABLES = 2,
NET_BRIDGE_NF_CALL_IP6TABLES = 3,
NET_BRIDGE_NF_FILTER_VLAN_TAGGED = 4,
NET_BRIDGE_NF_FILTER_PPPOE_TAGGED = 5,
};
enum {
NET_IRDA_DISCOVERY=1,
NET_IRDA_DEVNAME=2,
NET_IRDA_DEBUG=3,
NET_IRDA_FAST_POLL=4,
NET_IRDA_DISCOVERY_SLOTS=5,
NET_IRDA_DISCOVERY_TIMEOUT=6,
NET_IRDA_SLOT_TIMEOUT=7,
NET_IRDA_MAX_BAUD_RATE=8,
NET_IRDA_MIN_TX_TURN_TIME=9,
NET_IRDA_MAX_TX_DATA_SIZE=10,
NET_IRDA_MAX_TX_WINDOW=11,
NET_IRDA_MAX_NOREPLY_TIME=12,
NET_IRDA_WARN_NOREPLY_TIME=13,
NET_IRDA_LAP_KEEPALIVE_TIME=14,
};
enum
{
FS_NRINODE=1,
FS_STATINODE=2,
FS_MAXINODE=3,
FS_NRDQUOT=4,
FS_MAXDQUOT=5,
FS_NRFILE=6,
FS_MAXFILE=7,
FS_DENTRY=8,
FS_NRSUPER=9,
FS_MAXSUPER=10,
FS_OVERFLOWUID=11,
FS_OVERFLOWGID=12,
FS_LEASES=13,
FS_DIR_NOTIFY=14,
FS_LEASE_TIME=15,
FS_DQSTATS=16,
FS_XFS=17,
FS_AIO_NR=18,
FS_AIO_MAX_NR=19,
FS_INOTIFY=20,
FS_OCFS2=988,
};
enum {
FS_DQ_LOOKUPS = 1,
FS_DQ_DROPS = 2,
FS_DQ_READS = 3,
FS_DQ_WRITES = 4,
FS_DQ_CACHE_HITS = 5,
FS_DQ_ALLOCATED = 6,
FS_DQ_FREE = 7,
FS_DQ_SYNCS = 8,
FS_DQ_WARNINGS = 9,
};
enum {
DEV_CDROM=1,
DEV_HWMON=2,
DEV_PARPORT=3,
DEV_RAID=4,
DEV_MAC_HID=5,
DEV_SCSI=6,
DEV_IPMI=7,
};
enum {
DEV_CDROM_INFO=1,
DEV_CDROM_AUTOCLOSE=2,
DEV_CDROM_AUTOEJECT=3,
DEV_CDROM_DEBUG=4,
DEV_CDROM_LOCK=5,
DEV_CDROM_CHECK_MEDIA=6
};
enum {
DEV_PARPORT_DEFAULT=-3
};
enum {
DEV_RAID_SPEED_LIMIT_MIN=1,
DEV_RAID_SPEED_LIMIT_MAX=2
};
enum {
DEV_PARPORT_DEFAULT_TIMESLICE=1,
DEV_PARPORT_DEFAULT_SPINTIME=2
};
enum {
DEV_PARPORT_SPINTIME=1,
DEV_PARPORT_BASE_ADDR=2,
DEV_PARPORT_IRQ=3,
DEV_PARPORT_DMA=4,
DEV_PARPORT_MODES=5,
DEV_PARPORT_DEVICES=6,
DEV_PARPORT_AUTOPROBE=16
};
enum {
DEV_PARPORT_DEVICES_ACTIVE=-3,
};
enum {
DEV_PARPORT_DEVICE_TIMESLICE=1,
};
enum {
DEV_MAC_HID_KEYBOARD_SENDS_LINUX_KEYCODES=1,
DEV_MAC_HID_KEYBOARD_LOCK_KEYCODES=2,
DEV_MAC_HID_MOUSE_BUTTON_EMULATION=3,
DEV_MAC_HID_MOUSE_BUTTON2_KEYCODE=4,
DEV_MAC_HID_MOUSE_BUTTON3_KEYCODE=5,
DEV_MAC_HID_ADB_MOUSE_SENDS_KEYCODES=6
};
enum {
DEV_SCSI_LOGGING_LEVEL=1,
};
enum {
DEV_IPMI_POWEROFF_POWERCYCLE=1,
};
enum
{
ABI_DEFHANDLER_COFF=1,
ABI_DEFHANDLER_ELF=2,
ABI_DEFHANDLER_LCALL7=3,
ABI_DEFHANDLER_LIBCSO=4,
ABI_TRACE=5,
ABI_FAKE_UTSNAME=6,
};
# 31 "./include/linux/sysctl.h" 2
struct completion;
struct ctl_table;
struct nsproxy;
struct ctl_table_root;
struct ctl_table_header;
struct ctl_dir;
typedef int proc_handler (struct ctl_table *ctl, int write,
void *buffer, size_t *lenp, loff_t *ppos);
extern int proc_dostring(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_dointvec(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_douintvec(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_dointvec_minmax(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_douintvec_minmax(struct ctl_table *table, int write,
void *buffer, size_t *lenp,
loff_t *ppos);
extern int proc_dointvec_jiffies(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_dointvec_userhz_jiffies(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_dointvec_ms_jiffies(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_doulongvec_minmax(struct ctl_table *, int,
void *, size_t *, loff_t *);
extern int proc_doulongvec_ms_jiffies_minmax(struct ctl_table *table, int,
void *, size_t *, loff_t *);
extern int proc_do_large_bitmap(struct ctl_table *, int,
void *, size_t *, loff_t *);
# 95 "./include/linux/sysctl.h"
struct ctl_table_poll {
atomic_t event;
wait_queue_head_t wait;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *proc_sys_poll_event(struct ctl_table_poll *poll)
{
return (void *)(unsigned long)({ union { typeof((&poll->event)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&poll->event)->counter), __u.__c, sizeof((&poll->event)->counter)); else __read_once_size_nocheck(&((&poll->event)->counter), __u.__c, sizeof((&poll->event)->counter)); do { } while (0); __u.__val; });
}
# 113 "./include/linux/sysctl.h"
struct ctl_table
{
const char *procname;
void *data;
int maxlen;
umode_t mode;
struct ctl_table *child;
proc_handler *proc_handler;
struct ctl_table_poll *poll;
void *extra1;
void *extra2;
} ;
struct ctl_node {
struct rb_node node;
struct ctl_table_header *header;
};
struct ctl_table_header
{
union {
struct {
struct ctl_table *ctl_table;
int used;
int count;
int nreg;
};
struct callback_head rcu;
};
struct completion *unregistering;
struct ctl_table *ctl_table_arg;
struct ctl_table_root *root;
struct ctl_table_set *set;
struct ctl_dir *parent;
struct ctl_node *node;
struct hlist_head inodes;
};
struct ctl_dir {
struct ctl_table_header header;
struct rb_root root;
};
struct ctl_table_set {
int (*is_seen)(struct ctl_table_set *);
struct ctl_dir dir;
};
struct ctl_table_root {
struct ctl_table_set default_set;
struct ctl_table_set *(*lookup)(struct ctl_table_root *root);
void (*set_ownership)(struct ctl_table_header *head,
struct ctl_table *table,
kuid_t *uid, kgid_t *gid);
int (*permissions)(struct ctl_table_header *head, struct ctl_table *table);
};
struct ctl_path {
const char *procname;
};
void proc_sys_poll_notify(struct ctl_table_poll *poll);
extern void setup_sysctl_set(struct ctl_table_set *p,
struct ctl_table_root *root,
int (*is_seen)(struct ctl_table_set *));
extern void retire_sysctl_set(struct ctl_table_set *set);
struct ctl_table_header *__register_sysctl_table(
struct ctl_table_set *set,
const char *path, struct ctl_table *table);
struct ctl_table_header *__register_sysctl_paths(
struct ctl_table_set *set,
const struct ctl_path *path, struct ctl_table *table);
struct ctl_table_header *register_sysctl(const char *path, struct ctl_table *table);
struct ctl_table_header *register_sysctl_table(struct ctl_table * table);
struct ctl_table_header *register_sysctl_paths(const struct ctl_path *path,
struct ctl_table *table);
void unregister_sysctl_table(struct ctl_table_header * table);
extern int sysctl_init(void);
extern struct ctl_table sysctl_mount_point[];
# 233 "./include/linux/sysctl.h"
int sysctl_max_threads(struct ctl_table *table, int write,
void *buffer, size_t *lenp, loff_t *ppos);
# 23 "./include/linux/key.h" 2
# 1 "./include/linux/assoc_array.h" 1
# 26 "./include/linux/assoc_array.h"
struct assoc_array {
struct assoc_array_ptr *root;
unsigned long nr_leaves_on_tree;
};
struct assoc_array_ops {
unsigned long (*get_key_chunk)(const void *index_key, int level);
unsigned long (*get_object_key_chunk)(const void *object, int level);
bool (*compare_object)(const void *object, const void *index_key);
int (*diff_objects)(const void *object, const void *index_key);
void (*free_object)(void *object);
};
struct assoc_array_edit;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void assoc_array_init(struct assoc_array *array)
{
array->root = ((void *)0);
array->nr_leaves_on_tree = 0;
}
extern int assoc_array_iterate(const struct assoc_array *array,
int (*iterator)(const void *object,
void *iterator_data),
void *iterator_data);
extern void *assoc_array_find(const struct assoc_array *array,
const struct assoc_array_ops *ops,
const void *index_key);
extern void assoc_array_destroy(struct assoc_array *array,
const struct assoc_array_ops *ops);
extern struct assoc_array_edit *assoc_array_insert(struct assoc_array *array,
const struct assoc_array_ops *ops,
const void *index_key,
void *object);
extern void assoc_array_insert_set_object(struct assoc_array_edit *edit,
void *object);
extern struct assoc_array_edit *assoc_array_delete(struct assoc_array *array,
const struct assoc_array_ops *ops,
const void *index_key);
extern struct assoc_array_edit *assoc_array_clear(struct assoc_array *array,
const struct assoc_array_ops *ops);
extern void assoc_array_apply_edit(struct assoc_array_edit *edit);
extern void assoc_array_cancel_edit(struct assoc_array_edit *edit);
extern int assoc_array_gc(struct assoc_array *array,
const struct assoc_array_ops *ops,
bool (*iterator)(void *object, void *iterator_data),
void *iterator_data);
# 26 "./include/linux/key.h" 2
typedef int32_t key_serial_t;
typedef uint32_t key_perm_t;
struct key;
# 77 "./include/linux/key.h"
struct seq_file;
struct user_struct;
struct signal_struct;
struct cred;
struct key_type;
struct key_owner;
struct keyring_list;
struct keyring_name;
struct keyring_index_key {
struct key_type *type;
const char *description;
size_t desc_len;
};
union key_payload {
void *rcu_data0;
void *data[4];
};
# 112 "./include/linux/key.h"
typedef struct __key_reference_with_attributes *key_ref_t;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) key_ref_t make_key_ref(const struct key *key,
bool possession)
{
return (key_ref_t) ((unsigned long) key | possession);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct key *key_ref_to_ptr(const key_ref_t key_ref)
{
return (struct key *) ((unsigned long) key_ref & ~1UL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_key_possessed(const key_ref_t key_ref)
{
return (unsigned long) key_ref & 1UL;
}
typedef int (*key_restrict_link_func_t)(struct key *dest_keyring,
const struct key_type *type,
const union key_payload *payload,
struct key *restriction_key);
struct key_restriction {
key_restrict_link_func_t check;
struct key *key;
struct key_type *keytype;
};
enum key_state {
KEY_IS_UNINSTANTIATED,
KEY_IS_POSITIVE,
};
# 154 "./include/linux/key.h"
struct key {
refcount_t usage;
key_serial_t serial;
union {
struct list_head graveyard_link;
struct rb_node serial_node;
};
struct rw_semaphore sem;
struct key_user *user;
void *security;
union {
time_t expiry;
time_t revoked_at;
};
time_t last_used_at;
kuid_t uid;
kgid_t gid;
key_perm_t perm;
unsigned short quotalen;
unsigned short datalen;
short state;
unsigned long flags;
# 201 "./include/linux/key.h"
union {
struct keyring_index_key index_key;
struct {
struct key_type *type;
char *description;
};
};
union {
union key_payload payload;
struct {
struct list_head name_link;
struct assoc_array keys;
};
};
# 233 "./include/linux/key.h"
struct key_restriction *restrict_link;
};
extern struct key *key_alloc(struct key_type *type,
const char *desc,
kuid_t uid, kgid_t gid,
const struct cred *cred,
key_perm_t perm,
unsigned long flags,
struct key_restriction *restrict_link);
# 252 "./include/linux/key.h"
extern void key_revoke(struct key *key);
extern void key_invalidate(struct key *key);
extern void key_put(struct key *key);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct key *__key_get(struct key *key)
{
refcount_inc(&key->usage);
return key;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct key *key_get(struct key *key)
{
return key ? __key_get(key) : key;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void key_ref_put(key_ref_t key_ref)
{
key_put(key_ref_to_ptr(key_ref));
}
extern struct key *request_key(struct key_type *type,
const char *description,
const char *callout_info);
extern struct key *request_key_with_auxdata(struct key_type *type,
const char *description,
const void *callout_info,
size_t callout_len,
void *aux);
extern struct key *request_key_async(struct key_type *type,
const char *description,
const void *callout_info,
size_t callout_len);
extern struct key *request_key_async_with_auxdata(struct key_type *type,
const char *description,
const void *callout_info,
size_t callout_len,
void *aux);
extern int wait_for_key_construction(struct key *key, bool intr);
extern int key_validate(const struct key *key);
extern key_ref_t key_create_or_update(key_ref_t keyring,
const char *type,
const char *description,
const void *payload,
size_t plen,
key_perm_t perm,
unsigned long flags);
extern int key_update(key_ref_t key,
const void *payload,
size_t plen);
extern int key_link(struct key *keyring,
struct key *key);
extern int key_unlink(struct key *keyring,
struct key *key);
extern struct key *keyring_alloc(const char *description, kuid_t uid, kgid_t gid,
const struct cred *cred,
key_perm_t perm,
unsigned long flags,
struct key_restriction *restrict_link,
struct key *dest);
extern int restrict_link_reject(struct key *keyring,
const struct key_type *type,
const union key_payload *payload,
struct key *restriction_key);
extern int keyring_clear(struct key *keyring);
extern key_ref_t keyring_search(key_ref_t keyring,
struct key_type *type,
const char *description);
extern int keyring_add_key(struct key *keyring,
struct key *key);
extern int keyring_restrict(key_ref_t keyring, const char *type,
const char *restriction);
extern struct key *key_lookup(key_serial_t id);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) key_serial_t key_serial(const struct key *key)
{
return key ? key->serial : 0;
}
extern void key_set_timeout(struct key *, unsigned);
extern key_ref_t lookup_user_key(key_serial_t id, unsigned long flags,
key_perm_t perm);
# 362 "./include/linux/key.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) short key_read_state(const struct key *key)
{
return ({ typeof(*&key->state) ___p1 = ({ union { typeof(*&key->state) __val; char __c[1]; } __u; if (1) __read_once_size(&(*&key->state), __u.__c, sizeof(*&key->state)); else __read_once_size_nocheck(&(*&key->state), __u.__c, sizeof(*&key->state)); do { } while (0); __u.__val; }); do { bool __cond = !((sizeof(*&key->state) == sizeof(char) || sizeof(*&key->state) == sizeof(short) || sizeof(*&key->state) == sizeof(int) || sizeof(*&key->state) == sizeof(long))); extern void __compiletime_assert_41(void) ; if (__cond) __compiletime_assert_41(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); ___p1; });
}
# 375 "./include/linux/key.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool key_is_positive(const struct key *key)
{
return key_read_state(key) == KEY_IS_POSITIVE;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool key_is_negative(const struct key *key)
{
return key_read_state(key) < 0;
}
# 398 "./include/linux/key.h"
extern struct ctl_table key_sysctls[];
extern int install_thread_keyring_to_cred(struct cred *cred);
extern void key_fsuid_changed(struct task_struct *tsk);
extern void key_fsgid_changed(struct task_struct *tsk);
extern void key_init(void);
# 18 "./include/linux/cred.h" 2
# 1 "./include/linux/selinux.h" 1
# 17 "./include/linux/selinux.h"
struct selinux_audit_rule;
struct audit_context;
struct kern_ipc_perm;
bool selinux_is_enabled(void);
# 19 "./include/linux/cred.h" 2
# 1 "./include/linux/sched/user.h" 1
struct key;
struct user_struct {
atomic_t __count;
atomic_t processes;
atomic_t sigpending;
atomic_long_t epoll_watches;
unsigned long locked_shm;
unsigned long unix_inflight;
atomic_long_t pipe_bufs;
struct key *uid_keyring;
struct key *session_keyring;
struct hlist_node uidhash_node;
kuid_t uid;
atomic_long_t locked_vm;
};
extern int uids_sysfs_init(void);
extern struct user_struct *find_user(kuid_t);
extern struct user_struct root_user;
extern struct user_struct * alloc_uid(kuid_t);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct user_struct *get_uid(struct user_struct *u)
{
atomic_add(1, &u->__count);
return u;
}
extern void free_uid(struct user_struct *);
# 23 "./include/linux/cred.h" 2
struct cred;
struct inode;
struct group_info {
atomic_t usage;
int ngroups;
kgid_t gid[0];
} ;
# 45 "./include/linux/cred.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct group_info *get_group_info(struct group_info *gi)
{
atomic_add(1, &gi->usage);
return gi;
}
# 61 "./include/linux/cred.h"
extern struct group_info init_groups;
extern struct group_info *groups_alloc(int);
extern void groups_free(struct group_info *);
extern int in_group_p(kgid_t);
extern int in_egroup_p(kgid_t);
# 82 "./include/linux/cred.h"
extern int set_current_groups(struct group_info *);
extern void set_groups(struct cred *, struct group_info *);
extern int groups_search(const struct group_info *, kgid_t);
extern bool may_setgroups(void);
extern void groups_sort(struct group_info *);
# 111 "./include/linux/cred.h"
struct cred {
atomic_t usage;
kuid_t uid;
kgid_t gid;
kuid_t suid;
kgid_t sgid;
kuid_t euid;
kgid_t egid;
kuid_t fsuid;
kgid_t fsgid;
unsigned securebits;
kernel_cap_t cap_inheritable;
kernel_cap_t cap_permitted;
kernel_cap_t cap_effective;
kernel_cap_t cap_bset;
kernel_cap_t cap_ambient;
unsigned char jit_keyring;
struct key *session_keyring;
struct key *process_keyring;
struct key *thread_keyring;
struct key *request_key_auth;
void *security;
struct user_struct *user;
struct user_namespace *user_ns;
struct group_info *group_info;
union {
int non_rcu;
struct callback_head rcu;
};
} ;
extern void __put_cred(struct cred *);
extern void exit_creds(struct task_struct *);
extern int copy_creds(struct task_struct *, unsigned long);
extern const struct cred *get_task_cred(struct task_struct *);
extern struct cred *cred_alloc_blank(void);
extern struct cred *prepare_creds(void);
extern struct cred *prepare_exec_creds(void);
extern int commit_creds(struct cred *);
extern void abort_creds(struct cred *);
extern const struct cred *override_creds(const struct cred *);
extern void revert_creds(const struct cred *);
extern struct cred *prepare_kernel_cred(struct task_struct *);
extern int change_create_files_as(struct cred *, struct inode *);
extern int set_security_override(struct cred *, u32);
extern int set_security_override_from_ctx(struct cred *, const char *);
extern int set_create_files_as(struct cred *, struct inode *);
extern void __attribute__ ((__section__(".init.text"))) cred_init(void);
# 202 "./include/linux/cred.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void validate_creds(const struct cred *cred)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void validate_creds_for_do_exit(struct task_struct *tsk)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void validate_process_creds(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cap_ambient_invariant_ok(const struct cred *cred)
{
return cap_issubset(cred->cap_ambient,
cap_intersect(cred->cap_permitted,
cred->cap_inheritable));
}
# 227 "./include/linux/cred.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct cred *get_new_cred(struct cred *cred)
{
atomic_add(1, &cred->usage);
return cred;
}
# 246 "./include/linux/cred.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const struct cred *get_cred(const struct cred *cred)
{
struct cred *nonconst_cred = (struct cred *) cred;
validate_creds(cred);
nonconst_cred->non_rcu = 0;
return get_new_cred(nonconst_cred);
}
# 265 "./include/linux/cred.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void put_cred(const struct cred *_cred)
{
struct cred *cred = (struct cred *) _cred;
validate_creds(cred);
if ((({ typeof(atomic_sub_return_relaxed(1, &(cred)->usage)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_sub_return_relaxed(1, &(cred)->usage); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == 0))
__put_cred(cred);
}
# 374 "./include/linux/cred.h"
extern struct user_namespace init_user_ns;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct user_namespace *current_user_ns(void)
{
return &init_user_ns;
}
# 13 "./include/linux/seq_file.h" 2
struct seq_operations;
struct seq_file {
char *buf;
size_t size;
size_t from;
size_t count;
size_t pad_until;
loff_t index;
loff_t read_pos;
u64 version;
struct mutex lock;
const struct seq_operations *op;
int poll_event;
const struct file *file;
void *private;
};
struct seq_operations {
void * (*start) (struct seq_file *m, loff_t *pos);
void (*stop) (struct seq_file *m, void *v);
void * (*next) (struct seq_file *m, void *v, loff_t *pos);
int (*show) (struct seq_file *m, void *v);
};
# 51 "./include/linux/seq_file.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool seq_has_overflowed(struct seq_file *m)
{
return m->count == m->size;
}
# 64 "./include/linux/seq_file.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) size_t seq_get_buf(struct seq_file *m, char **bufp)
{
do { if (__builtin_expect(!!(m->count > m->size), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/seq_file.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "66" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
if (m->count < m->size)
*bufp = m->buf + m->count;
else
*bufp = ((void *)0);
return m->size - m->count;
}
# 84 "./include/linux/seq_file.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void seq_commit(struct seq_file *m, int num)
{
if (num < 0) {
m->count = m->size;
} else {
do { if (__builtin_expect(!!(m->count + num > m->size), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/seq_file.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "89" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
m->count += num;
}
}
# 102 "./include/linux/seq_file.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void seq_setwidth(struct seq_file *m, size_t size)
{
m->pad_until = m->count + size;
}
void seq_pad(struct seq_file *m, char c);
char *mangle_path(char *s, const char *p, const char *esc);
int seq_open(struct file *, const struct seq_operations *);
ssize_t seq_read(struct file *, char *, size_t, loff_t *);
loff_t seq_lseek(struct file *, loff_t, int);
int seq_release(struct inode *, struct file *);
int seq_write(struct seq_file *seq, const void *data, size_t len);
__attribute__((format(printf, 2, 0)))
void seq_vprintf(struct seq_file *m, const char *fmt, va_list args);
__attribute__((format(printf, 2, 3)))
void seq_printf(struct seq_file *m, const char *fmt, ...);
void seq_putc(struct seq_file *m, char c);
void seq_puts(struct seq_file *m, const char *s);
void seq_put_decimal_ull(struct seq_file *m, const char *delimiter,
unsigned long long num);
void seq_put_decimal_ll(struct seq_file *m, const char *delimiter, long long num);
void seq_escape(struct seq_file *m, const char *s, const char *esc);
void seq_hex_dump(struct seq_file *m, const char *prefix_str, int prefix_type,
int rowsize, int groupsize, const void *buf, size_t len,
bool ascii);
int seq_path(struct seq_file *, const struct path *, const char *);
int seq_file_path(struct seq_file *, struct file *, const char *);
int seq_dentry(struct seq_file *, struct dentry *, const char *);
int seq_path_root(struct seq_file *m, const struct path *path,
const struct path *root, const char *esc);
int single_open(struct file *, int (*)(struct seq_file *, void *), void *);
int single_open_size(struct file *, int (*)(struct seq_file *, void *), void *, size_t);
int single_release(struct inode *, struct file *);
void *__seq_open_private(struct file *, const struct seq_operations *, int);
int seq_open_private(struct file *, const struct seq_operations *, int);
int seq_release_private(struct inode *, struct file *);
# 157 "./include/linux/seq_file.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct user_namespace *seq_user_ns(struct seq_file *seq)
{
extern struct user_namespace init_user_ns;
return &init_user_ns;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void seq_show_option(struct seq_file *m, const char *name,
const char *value)
{
seq_putc(m, ',');
seq_escape(m, name, ",= \t\n\\");
if (value) {
seq_putc(m, '=');
seq_escape(m, value, ", \t\n\\");
}
}
# 207 "./include/linux/seq_file.h"
extern struct list_head *seq_list_start(struct list_head *head,
loff_t pos);
extern struct list_head *seq_list_start_head(struct list_head *head,
loff_t pos);
extern struct list_head *seq_list_next(void *v, struct list_head *head,
loff_t *ppos);
extern struct hlist_node *seq_hlist_start(struct hlist_head *head,
loff_t pos);
extern struct hlist_node *seq_hlist_start_head(struct hlist_head *head,
loff_t pos);
extern struct hlist_node *seq_hlist_next(void *v, struct hlist_head *head,
loff_t *ppos);
extern struct hlist_node *seq_hlist_start_rcu(struct hlist_head *head,
loff_t pos);
extern struct hlist_node *seq_hlist_start_head_rcu(struct hlist_head *head,
loff_t pos);
extern struct hlist_node *seq_hlist_next_rcu(void *v,
struct hlist_head *head,
loff_t *ppos);
extern struct hlist_node *seq_hlist_start_percpu(struct hlist_head *head, int *cpu, loff_t pos);
extern struct hlist_node *seq_hlist_next_percpu(void *v, struct hlist_head *head, int *cpu, loff_t *pos);
# 31 "./include/linux/pm.h" 2
extern void (*pm_power_off)(void);
extern void (*pm_power_off_prepare)(void);
struct device;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pm_vt_switch_required(struct device *dev, bool required)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pm_vt_switch_unregister(struct device *dev)
{
}
struct device;
extern const char power_group_name[];
typedef struct pm_message {
int event;
} pm_message_t;
# 291 "./include/linux/pm.h"
struct dev_pm_ops {
int (*prepare)(struct device *dev);
void (*complete)(struct device *dev);
int (*suspend)(struct device *dev);
int (*resume)(struct device *dev);
int (*freeze)(struct device *dev);
int (*thaw)(struct device *dev);
int (*poweroff)(struct device *dev);
int (*restore)(struct device *dev);
int (*suspend_late)(struct device *dev);
int (*resume_early)(struct device *dev);
int (*freeze_late)(struct device *dev);
int (*thaw_early)(struct device *dev);
int (*poweroff_late)(struct device *dev);
int (*restore_early)(struct device *dev);
int (*suspend_noirq)(struct device *dev);
int (*resume_noirq)(struct device *dev);
int (*freeze_noirq)(struct device *dev);
int (*thaw_noirq)(struct device *dev);
int (*poweroff_noirq)(struct device *dev);
int (*restore_noirq)(struct device *dev);
int (*runtime_suspend)(struct device *dev);
int (*runtime_resume)(struct device *dev);
int (*runtime_idle)(struct device *dev);
};
# 509 "./include/linux/pm.h"
enum rpm_status {
RPM_ACTIVE = 0,
RPM_RESUMING,
RPM_SUSPENDED,
RPM_SUSPENDING,
};
# 531 "./include/linux/pm.h"
enum rpm_request {
RPM_REQ_NONE = 0,
RPM_REQ_IDLE,
RPM_REQ_SUSPEND,
RPM_REQ_AUTOSUSPEND,
RPM_REQ_RESUME,
};
struct wakeup_source;
struct wake_irq;
struct pm_domain_data;
struct pm_subsys_data {
spinlock_t lock;
unsigned int refcount;
struct list_head clock_list;
};
struct dev_pm_info {
pm_message_t power_state;
unsigned int can_wakeup:1;
unsigned int async_suspend:1;
bool in_dpm_list:1;
bool is_prepared:1;
bool is_suspended:1;
bool is_noirq_suspended:1;
bool is_late_suspended:1;
bool no_pm:1;
bool early_init:1;
bool direct_complete:1;
spinlock_t lock;
struct list_head entry;
struct completion completion;
struct wakeup_source *wakeup;
bool wakeup_path:1;
bool syscore:1;
bool is_userresume:1;
bool no_pm_callbacks:1;
struct timer_list suspend_timer;
unsigned long timer_expires;
struct work_struct work;
wait_queue_head_t wait_queue;
struct wake_irq *wakeirq;
atomic_t usage_count;
atomic_t child_count;
unsigned int disable_depth:3;
unsigned int idle_notification:1;
unsigned int request_pending:1;
unsigned int deferred_resume:1;
unsigned int runtime_auto:1;
bool ignore_children:1;
unsigned int no_callbacks:1;
unsigned int irq_safe:1;
unsigned int use_autosuspend:1;
unsigned int timer_autosuspends:1;
unsigned int memalloc_noio:1;
unsigned int links_count;
enum rpm_request request;
enum rpm_status runtime_status;
int runtime_error;
int autosuspend_delay;
unsigned long last_busy;
unsigned long active_jiffies;
unsigned long suspended_jiffies;
unsigned long accounting_timestamp;
unsigned long resume_time;
struct pm_subsys_data *subsys_data;
void (*set_latency_tolerance)(struct device *, s32);
struct dev_pm_qos *qos;
};
extern void update_pm_runtime_accounting(struct device *dev);
extern int dev_pm_get_subsys_data(struct device *dev);
extern void dev_pm_put_subsys_data(struct device *dev);
# 631 "./include/linux/pm.h"
struct dev_pm_domain {
struct dev_pm_ops ops;
void (*detach)(struct device *dev, bool power_off);
int (*activate)(struct device *dev);
void (*sync)(struct device *dev);
void (*dismiss)(struct device *dev);
};
# 693 "./include/linux/pm.h"
extern void dpm_show_dev_resume_list(struct seq_file *buf);
extern void device_pm_lock(void);
extern void dpm_resume_start(pm_message_t state);
extern void dpm_resume_end(pm_message_t state);
extern void dpm_noirq_resume_devices(pm_message_t state);
extern void dpm_noirq_end(void);
extern void dpm_resume_noirq(pm_message_t state);
extern void dpm_resume_early(pm_message_t state);
extern void dpm_resume(pm_message_t state);
extern void dpm_resume_user(pm_message_t state);
extern int dpm_show_userresume_list(char *buf);
extern bool is_userresume_done(void);
extern void dpm_complete(pm_message_t state);
extern void device_pm_unlock(void);
extern int dpm_suspend_end(pm_message_t state);
extern int dpm_suspend_start(pm_message_t state);
extern void dpm_noirq_begin(void);
extern int dpm_noirq_suspend_devices(pm_message_t state);
extern int dpm_suspend_noirq(pm_message_t state);
extern int dpm_suspend_late(pm_message_t state);
extern int dpm_suspend(pm_message_t state);
extern int dpm_prepare(pm_message_t state);
extern void __suspend_report_result(const char *function, void *fn, int ret);
extern int device_pm_wait_for_dev(struct device *sub, struct device *dev);
extern void dpm_for_each_dev(void *data, void (*fn)(struct device *, void *));
extern int pm_generic_prepare(struct device *dev);
extern int pm_generic_suspend_late(struct device *dev);
extern int pm_generic_suspend_noirq(struct device *dev);
extern int pm_generic_suspend(struct device *dev);
extern int pm_generic_resume_early(struct device *dev);
extern int pm_generic_resume_noirq(struct device *dev);
extern int pm_generic_resume(struct device *dev);
extern int pm_generic_freeze_noirq(struct device *dev);
extern int pm_generic_freeze_late(struct device *dev);
extern int pm_generic_freeze(struct device *dev);
extern int pm_generic_thaw_noirq(struct device *dev);
extern int pm_generic_thaw_early(struct device *dev);
extern int pm_generic_thaw(struct device *dev);
extern int pm_generic_restore_noirq(struct device *dev);
extern int pm_generic_restore_early(struct device *dev);
extern int pm_generic_restore(struct device *dev);
extern int pm_generic_poweroff_noirq(struct device *dev);
extern int pm_generic_poweroff_late(struct device *dev);
extern int pm_generic_poweroff(struct device *dev);
extern void pm_generic_complete(struct device *dev);
extern void pm_complete_with_resume_check(struct device *dev);
# 796 "./include/linux/pm.h"
enum dpm_order {
DPM_ORDER_NONE,
DPM_ORDER_DEV_AFTER_PARENT,
DPM_ORDER_PARENT_BEFORE_DEV,
DPM_ORDER_DEV_LAST,
};
enum power_on_type {
POWER_ON_AC = 0,
POWER_ON_DC,
};
enum pm_stage_type {
PM_STAGE_BOOTCODE = 0,
PM_STAGE_KERNEL,
};
struct pm_resume_info {
unsigned long long elapsed;
unsigned long long start;
unsigned long long end;
};
void pm_resume_set_starttime(enum power_on_type powertype,
enum pm_stage_type stagetype, unsigned long long time);
void pm_resume_set_endtime(enum power_on_type powertype,
enum pm_stage_type stagetype, unsigned long long time);
void pm_resume_set_elapsedtime(enum power_on_type powertype,
enum pm_stage_type stagetype, unsigned long long time);
# 26 "./include/linux/device.h" 2
# 1 "./include/linux/ratelimit.h" 1
# 15 "./include/linux/ratelimit.h"
struct ratelimit_state {
raw_spinlock_t lock;
int interval;
int burst;
int printed;
int missed;
unsigned long begin;
unsigned long flags;
};
# 40 "./include/linux/ratelimit.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ratelimit_state_init(struct ratelimit_state *rs,
int interval, int burst)
{
({ void *__p = (rs); size_t __n = sizeof(*rs); if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
do { *(&rs->lock) = (raw_spinlock_t) { .raw_lock = { { 0 } }, }; } while (0);
rs->interval = interval;
rs->burst = burst;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ratelimit_default_init(struct ratelimit_state *rs)
{
return ratelimit_state_init(rs, (5 * 100),
10);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ratelimit_state_exit(struct ratelimit_state *rs)
{
if (!(rs->flags & (1UL << (0))))
return;
if (rs->missed) {
printk("\001" "4" "%s: %d output lines suppressed due to ratelimiting\n", (current_thread_info()->task)->comm, rs->missed);
rs->missed = 0;
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
ratelimit_set_flags(struct ratelimit_state *rs, unsigned long flags)
{
rs->flags = flags;
}
extern struct ratelimit_state printk_ratelimit_state;
extern int ___ratelimit(struct ratelimit_state *rs, const char *func);
# 28 "./include/linux/device.h" 2
# 1 "./arch/arm/include/asm/device.h" 1
struct dev_archdata {
# 22 "./arch/arm/include/asm/device.h"
unsigned int dma_coherent:1;
unsigned int dma_ops_setup:1;
};
struct omap_device;
struct pdev_archdata {
};
# 31 "./include/linux/device.h" 2
struct device;
struct device_private;
struct device_driver;
struct driver_private;
struct module;
struct class;
struct subsys_private;
struct bus_type;
struct device_node;
struct fwnode_handle;
struct iommu_ops;
struct iommu_group;
struct iommu_fwspec;
struct bus_attribute {
struct attribute attr;
ssize_t (*show)(struct bus_type *bus, char *buf);
ssize_t (*store)(struct bus_type *bus, const char *buf, size_t count);
};
# 59 "./include/linux/device.h"
extern int __attribute__((warn_unused_result)) bus_create_file(struct bus_type *,
struct bus_attribute *);
extern void bus_remove_file(struct bus_type *, struct bus_attribute *);
# 110 "./include/linux/device.h"
struct bus_type {
const char *name;
const char *dev_name;
struct device *dev_root;
const struct attribute_group **bus_groups;
const struct attribute_group **dev_groups;
const struct attribute_group **drv_groups;
int (*match)(struct device *dev, struct device_driver *drv);
int (*uevent)(struct device *dev, struct kobj_uevent_env *env);
int (*probe)(struct device *dev);
int (*remove)(struct device *dev);
void (*shutdown)(struct device *dev);
int (*online)(struct device *dev);
int (*offline)(struct device *dev);
int (*suspend)(struct device *dev, pm_message_t state);
int (*resume)(struct device *dev);
int (*num_vf)(struct device *dev);
const struct dev_pm_ops *pm;
const struct iommu_ops *iommu_ops;
struct subsys_private *p;
struct lock_class_key lock_key;
};
extern int __attribute__((warn_unused_result)) bus_register(struct bus_type *bus);
extern void bus_unregister(struct bus_type *bus);
extern int __attribute__((warn_unused_result)) bus_rescan_devices(struct bus_type *bus);
struct subsys_dev_iter {
struct klist_iter ki;
const struct device_type *type;
};
void subsys_dev_iter_init(struct subsys_dev_iter *iter,
struct bus_type *subsys,
struct device *start,
const struct device_type *type);
struct device *subsys_dev_iter_next(struct subsys_dev_iter *iter);
void subsys_dev_iter_exit(struct subsys_dev_iter *iter);
int bus_for_each_dev(struct bus_type *bus, struct device *start, void *data,
int (*fn)(struct device *dev, void *data));
struct device *bus_find_device(struct bus_type *bus, struct device *start,
void *data,
int (*match)(struct device *dev, void *data));
struct device *bus_find_device_by_name(struct bus_type *bus,
struct device *start,
const char *name);
struct device *subsys_find_device_by_id(struct bus_type *bus, unsigned int id,
struct device *hint);
int bus_for_each_drv(struct bus_type *bus, struct device_driver *start,
void *data, int (*fn)(struct device_driver *, void *));
void bus_sort_breadthfirst(struct bus_type *bus,
int (*compare)(const struct device *a,
const struct device *b));
struct notifier_block;
extern int bus_register_notifier(struct bus_type *bus,
struct notifier_block *nb);
extern int bus_unregister_notifier(struct bus_type *bus,
struct notifier_block *nb);
# 202 "./include/linux/device.h"
extern struct kset *bus_get_kset(struct bus_type *bus);
extern struct klist *bus_get_device_klist(struct bus_type *bus);
# 227 "./include/linux/device.h"
enum probe_type {
PROBE_DEFAULT_STRATEGY,
PROBE_PREFER_ASYNCHRONOUS,
PROBE_FORCE_SYNCHRONOUS,
};
# 266 "./include/linux/device.h"
struct device_driver {
const char *name;
struct bus_type *bus;
struct module *owner;
const char *mod_name;
bool suppress_bind_attrs;
enum probe_type probe_type;
const struct of_device_id *of_match_table;
const struct acpi_device_id *acpi_match_table;
int (*probe) (struct device *dev);
int (*remove) (struct device *dev);
void (*shutdown) (struct device *dev);
int (*suspend) (struct device *dev, pm_message_t state);
int (*resume) (struct device *dev);
const struct attribute_group **groups;
const struct dev_pm_ops *pm;
struct driver_private *p;
};
extern int __attribute__((warn_unused_result)) driver_register(struct device_driver *drv);
extern void driver_unregister(struct device_driver *drv);
extern struct device_driver *driver_find(const char *name,
struct bus_type *bus);
extern int driver_probe_done(void);
extern void wait_for_device_probe(void);
struct driver_attribute {
struct attribute attr;
ssize_t (*show)(struct device_driver *driver, char *buf);
ssize_t (*store)(struct device_driver *driver, const char *buf,
size_t count);
};
# 317 "./include/linux/device.h"
extern int __attribute__((warn_unused_result)) driver_create_file(struct device_driver *driver,
const struct driver_attribute *attr);
extern void driver_remove_file(struct device_driver *driver,
const struct driver_attribute *attr);
extern int __attribute__((warn_unused_result)) driver_for_each_device(struct device_driver *drv,
struct device *start,
void *data,
int (*fn)(struct device *dev,
void *));
struct device *driver_find_device(struct device_driver *drv,
struct device *start, void *data,
int (*match)(struct device *dev, void *data));
# 344 "./include/linux/device.h"
struct subsys_interface {
const char *name;
struct bus_type *subsys;
struct list_head node;
int (*add_dev)(struct device *dev, struct subsys_interface *sif);
void (*remove_dev)(struct device *dev, struct subsys_interface *sif);
};
int subsys_interface_register(struct subsys_interface *sif);
void subsys_interface_unregister(struct subsys_interface *sif);
int subsys_system_register(struct bus_type *subsys,
const struct attribute_group **groups);
int subsys_virtual_register(struct bus_type *subsys,
const struct attribute_group **groups);
# 389 "./include/linux/device.h"
struct class {
const char *name;
struct module *owner;
const struct attribute_group **class_groups;
const struct attribute_group **dev_groups;
struct kobject *dev_kobj;
int (*dev_uevent)(struct device *dev, struct kobj_uevent_env *env);
char *(*devnode)(struct device *dev, umode_t *mode);
void (*class_release)(struct class *class);
void (*dev_release)(struct device *dev);
int (*suspend)(struct device *dev, pm_message_t state);
int (*resume)(struct device *dev);
int (*shutdown_pre)(struct device *dev);
const struct kobj_ns_type_operations *ns_type;
const void *(*namespace)(struct device *dev);
const struct dev_pm_ops *pm;
struct subsys_private *p;
};
struct class_dev_iter {
struct klist_iter ki;
const struct device_type *type;
};
extern struct kobject *sysfs_dev_block_kobj;
extern struct kobject *sysfs_dev_char_kobj;
extern int __attribute__((warn_unused_result)) __class_register(struct class *class,
struct lock_class_key *key);
extern void class_unregister(struct class *class);
# 434 "./include/linux/device.h"
struct class_compat;
struct class_compat *class_compat_register(const char *name);
void class_compat_unregister(struct class_compat *cls);
int class_compat_create_link(struct class_compat *cls, struct device *dev,
struct device *device_link);
void class_compat_remove_link(struct class_compat *cls, struct device *dev,
struct device *device_link);
extern void class_dev_iter_init(struct class_dev_iter *iter,
struct class *class,
struct device *start,
const struct device_type *type);
extern struct device *class_dev_iter_next(struct class_dev_iter *iter);
extern void class_dev_iter_exit(struct class_dev_iter *iter);
extern int class_for_each_device(struct class *class, struct device *start,
void *data,
int (*fn)(struct device *dev, void *data));
extern struct device *class_find_device(struct class *class,
struct device *start, const void *data,
int (*match)(struct device *, const void *));
struct class_attribute {
struct attribute attr;
ssize_t (*show)(struct class *class, struct class_attribute *attr,
char *buf);
ssize_t (*store)(struct class *class, struct class_attribute *attr,
const char *buf, size_t count);
};
# 474 "./include/linux/device.h"
extern int __attribute__((warn_unused_result)) class_create_file_ns(struct class *class,
const struct class_attribute *attr,
const void *ns);
extern void class_remove_file_ns(struct class *class,
const struct class_attribute *attr,
const void *ns);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) class_create_file(struct class *class,
const struct class_attribute *attr)
{
return class_create_file_ns(class, attr, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void class_remove_file(struct class *class,
const struct class_attribute *attr)
{
return class_remove_file_ns(class, attr, ((void *)0));
}
struct class_attribute_string {
struct class_attribute attr;
char *str;
};
# 506 "./include/linux/device.h"
extern ssize_t show_class_attr_string(struct class *class, struct class_attribute *attr,
char *buf);
struct class_interface {
struct list_head node;
struct class *class;
int (*add_dev) (struct device *, struct class_interface *);
void (*remove_dev) (struct device *, struct class_interface *);
};
extern int __attribute__((warn_unused_result)) class_interface_register(struct class_interface *);
extern void class_interface_unregister(struct class_interface *);
extern struct class * __attribute__((warn_unused_result)) __class_create(struct module *owner,
const char *name,
struct lock_class_key *key);
extern void class_destroy(struct class *cls);
# 542 "./include/linux/device.h"
struct device_type {
const char *name;
const struct attribute_group **groups;
int (*uevent)(struct device *dev, struct kobj_uevent_env *env);
char *(*devnode)(struct device *dev, umode_t *mode,
kuid_t *uid, kgid_t *gid);
void (*release)(struct device *dev);
const struct dev_pm_ops *pm;
};
struct device_attribute {
struct attribute attr;
ssize_t (*show)(struct device *dev, struct device_attribute *attr,
char *buf);
ssize_t (*store)(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count);
};
struct dev_ext_attribute {
struct device_attribute attr;
void *var;
};
ssize_t device_show_ulong(struct device *dev, struct device_attribute *attr,
char *buf);
ssize_t device_store_ulong(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count);
ssize_t device_show_int(struct device *dev, struct device_attribute *attr,
char *buf);
ssize_t device_store_int(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count);
ssize_t device_show_bool(struct device *dev, struct device_attribute *attr,
char *buf);
ssize_t device_store_bool(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count);
# 601 "./include/linux/device.h"
extern int device_create_file(struct device *device,
const struct device_attribute *entry);
extern void device_remove_file(struct device *dev,
const struct device_attribute *attr);
extern bool device_remove_file_self(struct device *dev,
const struct device_attribute *attr);
extern int __attribute__((warn_unused_result)) device_create_bin_file(struct device *dev,
const struct bin_attribute *attr);
extern void device_remove_bin_file(struct device *dev,
const struct bin_attribute *attr);
typedef void (*dr_release_t)(struct device *dev, void *res);
typedef int (*dr_match_t)(struct device *dev, void *res, void *match_data);
# 624 "./include/linux/device.h"
extern void *devres_alloc_node(dr_release_t release, size_t size, gfp_t gfp,
int nid) __attribute__((__malloc__));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *devres_alloc(dr_release_t release, size_t size, gfp_t gfp)
{
return devres_alloc_node(release, size, gfp, (-1));
}
extern void devres_for_each_res(struct device *dev, dr_release_t release,
dr_match_t match, void *match_data,
void (*fn)(struct device *, void *, void *),
void *data);
extern void devres_free(void *res);
extern void devres_add(struct device *dev, void *res);
extern void *devres_find(struct device *dev, dr_release_t release,
dr_match_t match, void *match_data);
extern void *devres_get(struct device *dev, void *new_res,
dr_match_t match, void *match_data);
extern void *devres_remove(struct device *dev, dr_release_t release,
dr_match_t match, void *match_data);
extern int devres_destroy(struct device *dev, dr_release_t release,
dr_match_t match, void *match_data);
extern int devres_release(struct device *dev, dr_release_t release,
dr_match_t match, void *match_data);
extern void * __attribute__((warn_unused_result)) devres_open_group(struct device *dev, void *id,
gfp_t gfp);
extern void devres_close_group(struct device *dev, void *id);
extern void devres_remove_group(struct device *dev, void *id);
extern int devres_release_group(struct device *dev, void *id);
extern void *devm_kmalloc(struct device *dev, size_t size, gfp_t gfp) __attribute__((__malloc__));
extern __attribute__((format(printf, 3, 0)))
char *devm_kvasprintf(struct device *dev, gfp_t gfp, const char *fmt,
va_list ap) __attribute__((__malloc__));
extern __attribute__((format(printf, 3, 4)))
char *devm_kasprintf(struct device *dev, gfp_t gfp, const char *fmt, ...) __attribute__((__malloc__));
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *devm_kzalloc(struct device *dev, size_t size, gfp_t gfp)
{
return devm_kmalloc(dev, size, gfp | (( gfp_t)0x8000u));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *devm_kmalloc_array(struct device *dev,
size_t n, size_t size, gfp_t flags)
{
if (size != 0 && n > (~(size_t)0) / size)
return ((void *)0);
return devm_kmalloc(dev, n * size, flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *devm_kcalloc(struct device *dev,
size_t n, size_t size, gfp_t flags)
{
return devm_kmalloc_array(dev, n, size, flags | (( gfp_t)0x8000u));
}
extern void devm_kfree(struct device *dev, void *p);
extern char *devm_kstrdup(struct device *dev, const char *s, gfp_t gfp) __attribute__((__malloc__));
extern void *devm_kmemdup(struct device *dev, const void *src, size_t len,
gfp_t gfp);
extern unsigned long devm_get_free_pages(struct device *dev,
gfp_t gfp_mask, unsigned int order);
extern void devm_free_pages(struct device *dev, unsigned long addr);
void *devm_ioremap_resource(struct device *dev,
const struct resource *res);
int devm_add_action(struct device *dev, void (*action)(void *), void *data);
void devm_remove_action(struct device *dev, void (*action)(void *), void *data);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int devm_add_action_or_reset(struct device *dev,
void (*action)(void *), void *data)
{
int ret;
ret = devm_add_action(dev, action, data);
if (ret)
action(data);
return ret;
}
# 722 "./include/linux/device.h"
void *__devm_alloc_percpu(struct device *dev, size_t size,
size_t align);
void devm_free_percpu(struct device *dev, void *pdata);
struct device_dma_parameters {
unsigned int max_segment_size;
unsigned long segment_boundary_mask;
};
# 744 "./include/linux/device.h"
enum device_link_state {
DL_STATE_NONE = -1,
DL_STATE_DORMANT = 0,
DL_STATE_AVAILABLE,
DL_STATE_CONSUMER_PROBE,
DL_STATE_ACTIVE,
DL_STATE_SUPPLIER_UNBIND,
};
# 777 "./include/linux/device.h"
struct device_link {
struct device *supplier;
struct list_head s_node;
struct device *consumer;
struct list_head c_node;
enum device_link_state status;
u32 flags;
bool rpm_active;
struct callback_head callback_head;
};
# 797 "./include/linux/device.h"
enum dl_dev_state {
DL_DEV_NO_DRIVER = 0,
DL_DEV_PROBING,
DL_DEV_DRIVER_BOUND,
DL_DEV_UNBINDING,
};
struct dev_links_info {
struct list_head suppliers;
struct list_head consumers;
enum dl_dev_state status;
};
# 892 "./include/linux/device.h"
struct device {
struct device *parent;
struct device_private *p;
struct kobject kobj;
const char *init_name;
const struct device_type *type;
struct mutex mutex;
struct bus_type *bus;
struct device_driver *driver;
void *platform_data;
void *driver_data;
struct dev_links_info links;
struct dev_pm_info power;
struct dev_pm_domain *pm_domain;
# 929 "./include/linux/device.h"
const struct dma_map_ops *dma_ops;
u64 *dma_mask;
u64 coherent_dma_mask;
unsigned long dma_pfn_offset;
struct device_dma_parameters *dma_parms;
struct list_head dma_pools;
struct dma_coherent_mem *dma_mem;
struct cma *cma_area;
struct dev_archdata archdata;
struct device_node *of_node;
struct fwnode_handle *fwnode;
dev_t devt;
u32 id;
spinlock_t devres_lock;
struct list_head devres_head;
struct klist_node knode_class;
struct class *class;
const struct attribute_group **groups;
void (*release)(struct device *dev);
struct iommu_group *iommu_group;
struct iommu_fwspec *iommu_fwspec;
bool offline_disabled:1;
bool offline:1;
bool of_node_reused:1;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct device *kobj_to_dev(struct kobject *kobj)
{
return ({ void *__mptr = (void *)(kobj); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(kobj)), typeof(((struct device *)0)->kobj)) && !__builtin_types_compatible_p(typeof(*(kobj)), typeof(void)))); extern void __compiletime_assert_42(void) ; if (__cond) __compiletime_assert_42(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct device *)(__mptr - __builtin_offsetof(struct device, kobj))); });
}
# 1 "./include/linux/pm_wakeup.h" 1
# 31 "./include/linux/pm_wakeup.h"
struct wake_irq;
# 56 "./include/linux/pm_wakeup.h"
struct wakeup_source {
const char *name;
int id;
struct list_head entry;
spinlock_t lock;
struct wake_irq *wakeirq;
struct timer_list timer;
unsigned long timer_expires;
ktime_t total_time;
ktime_t max_time;
ktime_t last_time;
ktime_t start_prevent_time;
ktime_t prevent_sleep_time;
unsigned long event_count;
unsigned long active_count;
unsigned long relax_count;
unsigned long expire_count;
unsigned long wakeup_count;
struct device *dev;
bool active:1;
bool autosleep_enabled:1;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool device_can_wakeup(struct device *dev)
{
return dev->power.can_wakeup;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool device_may_wakeup(struct device *dev)
{
return dev->power.can_wakeup && !!dev->power.wakeup;
}
extern void wakeup_source_prepare(struct wakeup_source *ws, const char *name);
extern struct wakeup_source *wakeup_source_create(const char *name);
extern void wakeup_source_drop(struct wakeup_source *ws);
extern void wakeup_source_destroy(struct wakeup_source *ws);
extern void wakeup_source_add(struct wakeup_source *ws);
extern void wakeup_source_remove(struct wakeup_source *ws);
extern struct wakeup_source *wakeup_source_register(struct device *dev,
const char *name);
extern void wakeup_source_unregister(struct wakeup_source *ws);
extern int device_wakeup_enable(struct device *dev);
extern int device_wakeup_disable(struct device *dev);
extern void device_set_wakeup_capable(struct device *dev, bool capable);
extern int device_init_wakeup(struct device *dev, bool val);
extern int device_set_wakeup_enable(struct device *dev, bool enable);
extern void __pm_stay_awake(struct wakeup_source *ws);
extern void pm_stay_awake(struct device *dev);
extern void __pm_relax(struct wakeup_source *ws);
extern void pm_relax(struct device *dev);
extern void pm_wakeup_ws_event(struct wakeup_source *ws, unsigned int msec, bool hard);
extern void pm_wakeup_dev_event(struct device *dev, unsigned int msec, bool hard);
# 199 "./include/linux/pm_wakeup.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void wakeup_source_init(struct wakeup_source *ws,
const char *name)
{
wakeup_source_prepare(ws, name);
wakeup_source_add(ws);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void wakeup_source_trash(struct wakeup_source *ws)
{
wakeup_source_remove(ws);
wakeup_source_drop(ws);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __pm_wakeup_event(struct wakeup_source *ws, unsigned int msec)
{
return pm_wakeup_ws_event(ws, msec, false);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pm_wakeup_event(struct device *dev, unsigned int msec)
{
return pm_wakeup_dev_event(dev, msec, false);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pm_wakeup_hard_event(struct device *dev)
{
return pm_wakeup_dev_event(dev, 0, true);
}
# 980 "./include/linux/device.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) const char *dev_name(const struct device *dev)
{
if (dev->init_name)
return dev->init_name;
return kobject_name(&dev->kobj);
}
extern __attribute__((format(printf, 2, 3)))
int dev_set_name(struct device *dev, const char *name, ...);
# 1003 "./include/linux/device.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int dev_to_node(struct device *dev)
{
return -1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_dev_node(struct device *dev, int node)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct irq_domain *dev_get_msi_domain(const struct device *dev)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dev_set_msi_domain(struct device *dev, struct irq_domain *d)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *dev_get_drvdata(const struct device *dev)
{
return dev->driver_data;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dev_set_drvdata(struct device *dev, void *data)
{
dev->driver_data = data;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct pm_subsys_data *dev_to_psd(struct device *dev)
{
return dev ? dev->power.subsys_data : ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int dev_get_uevent_suppress(const struct device *dev)
{
return dev->kobj.uevent_suppress;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dev_set_uevent_suppress(struct device *dev, int val)
{
dev->kobj.uevent_suppress = val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int device_is_registered(struct device *dev)
{
return dev->kobj.state_in_sysfs;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void device_enable_async_suspend(struct device *dev)
{
if (!dev->power.is_prepared)
dev->power.async_suspend = true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void device_disable_async_suspend(struct device *dev)
{
if (!dev->power.is_prepared)
dev->power.async_suspend = false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool device_async_suspend_enabled(struct device *dev)
{
return !!dev->power.async_suspend;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool device_pm_not_required(struct device *dev)
{
return dev->power.no_pm;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void device_set_pm_not_required(struct device *dev)
{
dev->power.no_pm = true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dev_pm_syscore_device(struct device *dev, bool val)
{
dev->power.syscore = val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void device_lock(struct device *dev)
{
mutex_lock(&dev->mutex);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int device_lock_interruptible(struct device *dev)
{
return mutex_lock_interruptible(&dev->mutex);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int device_trylock(struct device *dev)
{
return mutex_trylock(&dev->mutex);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void device_unlock(struct device *dev)
{
mutex_unlock(&dev->mutex);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void device_lock_assert(struct device *dev)
{
do { (void)(&dev->mutex); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct device_node *dev_of_node(struct device *dev)
{
if (!1)
return ((void *)0);
return dev->of_node;
}
void driver_init(void);
extern int __attribute__((warn_unused_result)) device_register(struct device *dev);
extern void device_unregister(struct device *dev);
extern void device_initialize(struct device *dev);
extern int __attribute__((warn_unused_result)) device_add(struct device *dev);
extern void device_del(struct device *dev);
extern int device_for_each_child(struct device *dev, void *data,
int (*fn)(struct device *dev, void *data));
extern int device_for_each_child_reverse(struct device *dev, void *data,
int (*fn)(struct device *dev, void *data));
extern struct device *device_find_child(struct device *dev, void *data,
int (*match)(struct device *dev, void *data));
extern int device_rename(struct device *dev, const char *new_name);
extern int device_move(struct device *dev, struct device *new_parent,
enum dpm_order dpm_order);
extern const char *device_get_devnode(struct device *dev,
umode_t *mode, kuid_t *uid, kgid_t *gid,
const char **tmp);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool device_supports_offline(struct device *dev)
{
return dev->bus && dev->bus->offline && dev->bus->online;
}
extern void lock_device_hotplug(void);
extern void unlock_device_hotplug(void);
extern int lock_device_hotplug_sysfs(void);
extern int device_offline(struct device *dev);
extern int device_online(struct device *dev);
extern void set_primary_fwnode(struct device *dev, struct fwnode_handle *fwnode);
extern void set_secondary_fwnode(struct device *dev, struct fwnode_handle *fwnode);
void device_set_of_node_from_dev(struct device *dev, const struct device *dev2);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int dev_num_vf(struct device *dev)
{
if (dev->bus && dev->bus->num_vf)
return dev->bus->num_vf(dev);
return 0;
}
extern struct device *__root_device_register(const char *name,
struct module *owner);
extern void root_device_unregister(struct device *root);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *dev_get_platdata(const struct device *dev)
{
return dev->platform_data;
}
extern int __attribute__((warn_unused_result)) device_bind_driver(struct device *dev);
extern void device_release_driver(struct device *dev);
extern int __attribute__((warn_unused_result)) device_attach(struct device *dev);
extern int __attribute__((warn_unused_result)) driver_attach(struct device_driver *drv);
extern void device_initial_probe(struct device *dev);
extern int __attribute__((warn_unused_result)) device_reprobe(struct device *dev);
extern bool device_is_bound(struct device *dev);
extern __attribute__((format(printf, 5, 0)))
struct device *device_create_vargs(struct class *cls, struct device *parent,
dev_t devt, void *drvdata,
const char *fmt, va_list vargs);
extern __attribute__((format(printf, 5, 6)))
struct device *device_create(struct class *cls, struct device *parent,
dev_t devt, void *drvdata,
const char *fmt, ...);
extern __attribute__((format(printf, 6, 7)))
struct device *device_create_with_groups(struct class *cls,
struct device *parent, dev_t devt, void *drvdata,
const struct attribute_group **groups,
const char *fmt, ...);
extern void device_destroy(struct class *cls, dev_t devt);
extern int __attribute__((warn_unused_result)) device_add_groups(struct device *dev,
const struct attribute_group **groups);
extern void device_remove_groups(struct device *dev,
const struct attribute_group **groups);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __attribute__((warn_unused_result)) device_add_group(struct device *dev,
const struct attribute_group *grp)
{
const struct attribute_group *groups[] = { grp, ((void *)0) };
return device_add_groups(dev, groups);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void device_remove_group(struct device *dev,
const struct attribute_group *grp)
{
const struct attribute_group *groups[] = { grp, ((void *)0) };
return device_remove_groups(dev, groups);
}
extern int __attribute__((warn_unused_result)) devm_device_add_groups(struct device *dev,
const struct attribute_group **groups);
extern void devm_device_remove_groups(struct device *dev,
const struct attribute_group **groups);
extern int __attribute__((warn_unused_result)) devm_device_add_group(struct device *dev,
const struct attribute_group *grp);
extern void devm_device_remove_group(struct device *dev,
const struct attribute_group *grp);
extern int (*platform_notify)(struct device *dev);
extern int (*platform_notify_remove)(struct device *dev);
extern struct device *get_device(struct device *dev);
extern void put_device(struct device *dev);
extern int devtmpfs_create_node(struct device *dev);
extern int devtmpfs_delete_node(struct device *dev);
extern int devtmpfs_mount(const char *mntdir);
extern void device_shutdown(void);
extern const char *dev_driver_string(const struct device *dev);
struct device_link *device_link_add(struct device *consumer,
struct device *supplier, u32 flags);
void device_link_del(struct device_link *link);
extern __attribute__((format(printf, 3, 0)))
int dev_vprintk_emit(int level, const struct device *dev,
const char *fmt, va_list args);
extern __attribute__((format(printf, 3, 4)))
int dev_printk_emit(int level, const struct device *dev, const char *fmt, ...);
extern __attribute__((format(printf, 3, 4)))
void dev_printk(const char *level, const struct device *dev,
const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
void dev_emerg(const struct device *dev, const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
void dev_alert(const struct device *dev, const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
void dev_crit(const struct device *dev, const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
void dev_err(const struct device *dev, const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
void dev_warn(const struct device *dev, const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
void dev_notice(const struct device *dev, const char *fmt, ...);
extern __attribute__((format(printf, 2, 3)))
void _dev_info(const struct device *dev, const char *fmt, ...);
# 66 "./include/linux/genhd.h" 2
struct partition {
unsigned char boot_ind;
unsigned char head;
unsigned char sector;
unsigned char cyl;
unsigned char sys_ind;
unsigned char end_head;
unsigned char end_sector;
unsigned char end_cyl;
__le32 start_sect;
__le32 nr_sects;
} __attribute__((packed));
struct disk_stats {
unsigned long sectors[2];
unsigned long ios[2];
unsigned long merges[2];
unsigned long ticks[2];
unsigned long io_ticks;
unsigned long time_in_queue;
};
# 100 "./include/linux/genhd.h"
struct partition_meta_info {
char uuid[(36 + 1)];
u8 volname[64];
};
struct hd_struct {
sector_t start_sect;
sector_t nr_sects;
seqcount_t nr_sects_seq;
sector_t alignment_offset;
unsigned int discard_alignment;
struct device __dev;
struct kobject *holder_dir;
int policy, partno;
struct partition_meta_info *info;
unsigned long stamp;
atomic_t in_flight[2];
struct disk_stats *dkstats;
struct percpu_ref ref;
struct callback_head callback_head;
};
# 150 "./include/linux/genhd.h"
enum {
DISK_EVENT_MEDIA_CHANGE = 1 << 0,
DISK_EVENT_EJECT_REQUEST = 1 << 1,
};
struct disk_part_tbl {
struct callback_head callback_head;
int len;
struct hd_struct *last_lookup;
struct hd_struct *part[];
};
struct disk_events;
struct badblocks;
# 177 "./include/linux/genhd.h"
struct gendisk {
int major;
int first_minor;
int minors;
char disk_name[32];
char *(*devnode)(struct gendisk *gd, umode_t *mode);
unsigned int events;
unsigned int async_events;
struct disk_part_tbl *part_tbl;
struct hd_struct part0;
const struct block_device_operations *fops;
struct request_queue *queue;
void *private_data;
int flags;
struct kobject *slave_dir;
struct timer_rand_state *random;
atomic_t sync_io;
struct disk_events *ev;
int node_id;
struct badblocks *bb;
int part_num;
int nparts;
int part_extended;
int part_extended_serial;
char port_structure[32];
int signature;
int physical_sector;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct gendisk *part_to_disk(struct hd_struct *part)
{
if (__builtin_expect(!!(part), 1)) {
if (part->partno)
return ({ void *__mptr = (void *)(((&((part)->__dev))->parent)); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(((&((part)->__dev))->parent))), typeof(((struct gendisk *)0)->part0.__dev)) && !__builtin_types_compatible_p(typeof(*(((&((part)->__dev))->parent))), typeof(void)))); extern void __compiletime_assert_43(void) ; if (__cond) __compiletime_assert_43(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct gendisk *)(__mptr - __builtin_offsetof(struct gendisk, part0.__dev))); });
else
return ({ void *__mptr = (void *)(((&((part)->__dev)))); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(((&((part)->__dev))))), typeof(((struct gendisk *)0)->part0.__dev)) && !__builtin_types_compatible_p(typeof(*(((&((part)->__dev))))), typeof(void)))); extern void __compiletime_assert_44(void) ; if (__cond) __compiletime_assert_44(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct gendisk *)(__mptr - __builtin_offsetof(struct gendisk, part0.__dev))); });
}
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int disk_max_parts(struct gendisk *disk)
{
if (disk->flags & 64)
return 256;
return disk->minors;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool disk_part_scan_enabled(struct gendisk *disk)
{
return disk_max_parts(disk) > 1 &&
!(disk->flags & 512);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) dev_t disk_devt(struct gendisk *disk)
{
return (&(disk)->part0.__dev)->devt;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) dev_t part_devt(struct hd_struct *part)
{
return (&((part)->__dev))->devt;
}
extern struct hd_struct *__disk_get_part(struct gendisk *disk, int partno);
extern struct hd_struct *disk_get_part(struct gendisk *disk, int partno);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void disk_put_part(struct hd_struct *part)
{
if (__builtin_expect(!!(part), 1))
put_device((&((part)->__dev)));
}
# 278 "./include/linux/genhd.h"
struct disk_part_iter {
struct gendisk *disk;
struct hd_struct *part;
int idx;
unsigned int flags;
};
extern void disk_part_iter_init(struct disk_part_iter *piter,
struct gendisk *disk, unsigned int flags);
extern struct hd_struct *disk_part_iter_next(struct disk_part_iter *piter);
extern void disk_part_iter_exit(struct disk_part_iter *piter);
extern struct hd_struct *disk_map_sector_rcu(struct gendisk *disk,
sector_t sector);
# 321 "./include/linux/genhd.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void part_stat_set_all(struct hd_struct *part, int value)
{
int i;
for (((i)) = -1; ((i)) = cpumask_next(((i)), (((const struct cpumask *)&__cpu_possible_mask))), ((i)) < nr_cpu_ids;)
({ void *__p = (({ do { const void *__vpp_verify = (typeof((part->dkstats) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*((part->dkstats))) *)((part->dkstats)))); (typeof((typeof(*((part->dkstats))) *)((part->dkstats)))) (__ptr + (((__per_cpu_offset[(i)])))); }); })); size_t __n = sizeof(struct disk_stats); if ((__n) != 0) { if (__builtin_constant_p((value)) && (value) == 0) __memzero((__p),(__n)); else memset((__p),(value),(__n)); } (__p); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int init_part_stats(struct hd_struct *part)
{
part->dkstats = (typeof(struct disk_stats) *)__alloc_percpu(sizeof(struct disk_stats), __alignof__(struct disk_stats));
if (!part->dkstats)
return 0;
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void free_part_stats(struct hd_struct *part)
{
free_percpu(part->dkstats);
}
# 382 "./include/linux/genhd.h"
void part_in_flight(struct request_queue *q, struct hd_struct *part,
unsigned int inflight[2]);
void part_in_flight_rw(struct request_queue *q, struct hd_struct *part,
unsigned int inflight[2]);
void part_dec_in_flight(struct request_queue *q, struct hd_struct *part,
int rw);
void part_inc_in_flight(struct request_queue *q, struct hd_struct *part,
int rw);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct partition_meta_info *alloc_part_info(struct gendisk *disk)
{
if (disk)
return kzalloc_node(sizeof(struct partition_meta_info),
((( gfp_t)(0x400000u|0x1000000u)) | (( gfp_t)0x40u) | (( gfp_t)0x80u)), disk->node_id);
return kzalloc(sizeof(struct partition_meta_info), ((( gfp_t)(0x400000u|0x1000000u)) | (( gfp_t)0x40u) | (( gfp_t)0x80u)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void free_part_info(struct hd_struct *part)
{
kfree(part->info);
}
extern void part_round_stats(struct request_queue *q, int cpu, struct hd_struct *part);
extern void device_add_disk(struct device *parent, struct gendisk *disk);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void add_disk(struct gendisk *disk)
{
device_add_disk(((void *)0), disk);
}
extern void del_gendisk(struct gendisk *gp);
extern struct gendisk *get_gendisk(dev_t dev, int *partno);
extern struct block_device *bdget_disk(struct gendisk *disk, int partno);
extern void set_device_ro(struct block_device *bdev, int flag);
extern void set_disk_ro(struct gendisk *disk, int flag);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_disk_ro(struct gendisk *disk)
{
return disk->part0.policy;
}
extern void disk_block_events(struct gendisk *disk);
extern void disk_unblock_events(struct gendisk *disk);
extern void disk_flush_events(struct gendisk *disk, unsigned int mask);
extern unsigned int disk_clear_events(struct gendisk *disk, unsigned int mask);
extern void add_disk_randomness(struct gendisk *disk) ;
extern void rand_initialize_disk(struct gendisk *disk);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) sector_t get_start_sect(struct block_device *bdev)
{
return bdev->bd_part->start_sect;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) sector_t get_capacity(struct gendisk *disk)
{
return disk->part0.nr_sects;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_capacity(struct gendisk *disk, sector_t size)
{
disk->part0.nr_sects = size;
}
# 595 "./include/linux/genhd.h"
extern int blk_alloc_devt(struct hd_struct *part, dev_t *devt);
extern void blk_free_devt(dev_t devt);
extern dev_t blk_lookup_devt(const char *name, int partno);
extern char *disk_name (struct gendisk *hd, int partno, char *buf);
extern int disk_expand_part_tbl(struct gendisk *disk, int target);
extern int rescan_partitions(struct gendisk *disk, struct block_device *bdev);
extern int invalidate_partitions(struct gendisk *disk, struct block_device *bdev);
extern struct hd_struct * __attribute__((warn_unused_result)) add_partition(struct gendisk *disk,
int partno, sector_t start,
sector_t len, int flags,
struct partition_meta_info
*info);
extern void __delete_partition(struct percpu_ref *);
extern void delete_partition(struct gendisk *, int);
extern void printk_all_partitions(void);
extern struct gendisk *alloc_disk_node(int minors, int node_id);
extern struct gendisk *alloc_disk(int minors);
extern struct kobject *get_disk(struct gendisk *disk);
extern void put_disk(struct gendisk *disk);
extern void blk_register_region(dev_t devt, unsigned long range,
struct module *module,
struct kobject *(*probe)(dev_t, int *, void *),
int (*lock)(dev_t, void *),
void *data);
extern void blk_unregister_region(dev_t devt, unsigned long range);
extern ssize_t part_size_show(struct device *dev,
struct device_attribute *attr, char *buf);
extern ssize_t part_stat_show(struct device *dev,
struct device_attribute *attr, char *buf);
extern ssize_t part_inflight_show(struct device *dev,
struct device_attribute *attr, char *buf);
# 637 "./include/linux/genhd.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hd_ref_init(struct hd_struct *part)
{
if (percpu_ref_init(&part->ref, __delete_partition, 0,
((( gfp_t)(0x400000u|0x1000000u)) | (( gfp_t)0x40u) | (( gfp_t)0x80u))))
return -12;
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hd_struct_get(struct hd_struct *part)
{
percpu_ref_get(&part->ref);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hd_struct_try_get(struct hd_struct *part)
{
return percpu_ref_tryget_live(&part->ref);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hd_struct_put(struct hd_struct *part)
{
percpu_ref_put(&part->ref);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hd_struct_kill(struct hd_struct *part)
{
percpu_ref_kill(&part->ref);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hd_free_part(struct hd_struct *part)
{
free_part_stats(part);
free_part_info(part);
percpu_ref_exit(&part->ref);
}
# 681 "./include/linux/genhd.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) sector_t part_nr_sects_read(struct hd_struct *part)
{
sector_t nr_sects;
unsigned seq;
do {
seq = read_seqcount_begin(&part->nr_sects_seq);
nr_sects = part->nr_sects;
} while (read_seqcount_retry(&part->nr_sects_seq, seq));
return nr_sects;
# 701 "./include/linux/genhd.h"
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void part_nr_sects_write(struct hd_struct *part, sector_t size)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
write_seqcount_begin(&part->nr_sects_seq);
part->nr_sects = size;
write_seqcount_end(&part->nr_sects_seq);
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_integrity_add(struct gendisk *disk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_integrity_del(struct gendisk *disk) { }
# 12 "./include/linux/blkdev.h" 2
# 1 "./include/linux/pagemap.h" 1
# 1 "./include/linux/mm.h" 1
# 18 "./include/linux/mm.h"
# 1 "./include/linux/range.h" 1
struct range {
u64 start;
u64 end;
};
int add_range(struct range *range, int az, int nr_range,
u64 start, u64 end);
int add_range_with_merge(struct range *range, int az, int nr_range,
u64 start, u64 end);
void subtract_range(struct range *range, int az, u64 start, u64 end);
int clean_sort_range(struct range *range, int az);
void sort_range(struct range *range, int nr_range);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) resource_size_t cap_resource(u64 val)
{
if (val > ((resource_size_t)~0))
return ((resource_size_t)~0);
return val;
}
# 19 "./include/linux/mm.h" 2
# 1 "./include/linux/page_ext.h" 1
# 1 "./include/linux/stacktrace.h" 1
struct task_struct;
struct pt_regs;
struct stack_trace {
unsigned int nr_entries, max_entries;
unsigned long *entries;
int skip;
};
extern void save_stack_trace(struct stack_trace *trace);
extern void save_stack_trace_regs(struct pt_regs *regs,
struct stack_trace *trace);
extern void save_stack_trace_tsk(struct task_struct *tsk,
struct stack_trace *trace);
extern int save_stack_trace_tsk_reliable(struct task_struct *tsk,
struct stack_trace *trace);
extern void print_stack_trace(struct stack_trace *trace, int spaces);
extern int snprint_stack_trace(char *buf, size_t size,
struct stack_trace *trace, int spaces);
# 7 "./include/linux/page_ext.h" 2
# 1 "./include/linux/stackdepot.h" 1
# 24 "./include/linux/stackdepot.h"
typedef u32 depot_stack_handle_t;
struct stack_trace;
depot_stack_handle_t depot_save_stack(struct stack_trace *trace, gfp_t flags);
void depot_fetch_stack(depot_stack_handle_t handle, struct stack_trace *trace);
# 8 "./include/linux/page_ext.h" 2
struct pglist_data;
struct page_ext_operations {
size_t offset;
size_t size;
bool (*need)(void);
void (*init)(void);
};
# 67 "./include/linux/page_ext.h"
struct page_ext;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgdat_page_ext_init(struct pglist_data *pgdat)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page_ext *lookup_page_ext(struct page *page)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_ext_init(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_ext_init_flatmem(void)
{
}
# 25 "./include/linux/mm.h" 2
# 1 "./include/linux/page_ref.h" 1
# 1 "./include/linux/page-flags.h" 1
# 75 "./include/linux/page-flags.h"
enum pageflags {
PG_locked,
PG_referenced,
PG_uptodate,
PG_dirty,
PG_lru,
PG_active,
PG_workingset,
PG_waiters,
PG_error,
PG_slab,
PG_owner_priv_1,
PG_arch_1,
PG_reserved,
PG_private,
PG_private_2,
PG_writeback,
PG_head,
PG_mappedtodisk,
PG_reclaim,
PG_swapbacked,
PG_unevictable,
PG_mlocked,
# 111 "./include/linux/page-flags.h"
PG_rtkcache,
__NR_PAGEFLAGS,
PG_checked = PG_owner_priv_1,
PG_swapcache = PG_owner_priv_1,
PG_fscache = PG_private_2,
PG_pinned = PG_owner_priv_1,
PG_savepinned = PG_dirty,
PG_foreign = PG_owner_priv_1,
PG_slob_free = PG_private,
PG_double_map = PG_private_2,
PG_isolated = PG_reclaim,
};
struct page;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *compound_head(struct page *page)
{
unsigned long head = ({ union { typeof(page->compound_head) __val; char __c[1]; } __u; if (1) __read_once_size(&(page->compound_head), __u.__c, sizeof(page->compound_head)); else __read_once_size_nocheck(&(page->compound_head), __u.__c, sizeof(page->compound_head)); do { } while (0); __u.__val; });
if (__builtin_expect(!!(head & 1), 0))
return (struct page *) (head - 1);
return page;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageTail(struct page *page)
{
return ({ union { typeof(page->compound_head) __val; char __c[1]; } __u; if (1) __read_once_size(&(page->compound_head), __u.__c, sizeof(page->compound_head)); else __read_once_size_nocheck(&(page->compound_head), __u.__c, sizeof(page->compound_head)); do { } while (0); __u.__val; }) & 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageCompound(struct page *page)
{
return test_bit(PG_head, &page->flags) || PageTail(page);
}
# 269 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageLocked(struct page *page) { return test_bit(PG_locked, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageLocked(struct page *page) { __set_bit(PG_locked, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageLocked(struct page *page) { __clear_bit(PG_locked, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageWaiters(struct page *page) { return test_bit(PG_waiters, &({ ((void)(sizeof(( long)(PageTail(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageWaiters(struct page *page) { _set_bit(PG_waiters,&({ ((void)(sizeof(( long)(PageTail(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageWaiters(struct page *page) { _clear_bit(PG_waiters,&({ ((void)(sizeof(( long)(PageTail(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageWaiters(struct page *page) { __clear_bit(PG_waiters, &({ ((void)(sizeof(( long)(PageTail(page))))); page;})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageError(struct page *page) { return test_bit(PG_error, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageError(struct page *page) { _set_bit(PG_error,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageError(struct page *page) { _clear_bit(PG_error,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageError(struct page *page) { return _test_and_clear_bit(PG_error,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageReferenced(struct page *page) { return test_bit(PG_referenced, &compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageReferenced(struct page *page) { _set_bit(PG_referenced,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageReferenced(struct page *page) { _clear_bit(PG_referenced,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageReferenced(struct page *page) { return _test_and_clear_bit(PG_referenced,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageReferenced(struct page *page) { __set_bit(PG_referenced, &compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageDirty(struct page *page) { return test_bit(PG_dirty, &compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageDirty(struct page *page) { _set_bit(PG_dirty,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageDirty(struct page *page) { _clear_bit(PG_dirty,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestSetPageDirty(struct page *page) { return _test_and_set_bit(PG_dirty,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageDirty(struct page *page) { return _test_and_clear_bit(PG_dirty,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageDirty(struct page *page) { __clear_bit(PG_dirty, &compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageLRU(struct page *page) { return test_bit(PG_lru, &compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageLRU(struct page *page) { _set_bit(PG_lru,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageLRU(struct page *page) { _clear_bit(PG_lru,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageLRU(struct page *page) { __clear_bit(PG_lru, &compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageActive(struct page *page) { return test_bit(PG_active, &compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageActive(struct page *page) { _set_bit(PG_active,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageActive(struct page *page) { _clear_bit(PG_active,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageActive(struct page *page) { __clear_bit(PG_active, &compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageActive(struct page *page) { return _test_and_clear_bit(PG_active,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageWorkingset(struct page *page) { return test_bit(PG_workingset, &compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageWorkingset(struct page *page) { _set_bit(PG_workingset,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageWorkingset(struct page *page) { _clear_bit(PG_workingset,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageWorkingset(struct page *page) { return _test_and_clear_bit(PG_workingset,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageSlab(struct page *page) { return test_bit(PG_slab, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageSlab(struct page *page) { __set_bit(PG_slab, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageSlab(struct page *page) { __clear_bit(PG_slab, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageSlobFree(struct page *page) { return test_bit(PG_slob_free, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageSlobFree(struct page *page) { __set_bit(PG_slob_free, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageSlobFree(struct page *page) { __clear_bit(PG_slob_free, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageChecked(struct page *page) { return test_bit(PG_checked, &({ ((void)(sizeof(( long)(0 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageChecked(struct page *page) { _set_bit(PG_checked,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageChecked(struct page *page) { _clear_bit(PG_checked,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PagePinned(struct page *page) { return test_bit(PG_pinned, &({ ((void)(sizeof(( long)(0 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPagePinned(struct page *page) { _set_bit(PG_pinned,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPagePinned(struct page *page) { _clear_bit(PG_pinned,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestSetPagePinned(struct page *page) { return _test_and_set_bit(PG_pinned,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPagePinned(struct page *page) { return _test_and_clear_bit(PG_pinned,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageSavePinned(struct page *page) { return test_bit(PG_savepinned, &({ ((void)(sizeof(( long)(0 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageSavePinned(struct page *page) { _set_bit(PG_savepinned,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageSavePinned(struct page *page) { _clear_bit(PG_savepinned,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); };
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageForeign(struct page *page) { return test_bit(PG_foreign, &({ ((void)(sizeof(( long)(0 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageForeign(struct page *page) { _set_bit(PG_foreign,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageForeign(struct page *page) { _clear_bit(PG_foreign,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); };
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageReserved(struct page *page) { return test_bit(PG_reserved, &({ ((void)(sizeof(( long)(0 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageReserved(struct page *page) { _set_bit(PG_reserved,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageReserved(struct page *page) { _clear_bit(PG_reserved,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageReserved(struct page *page) { __clear_bit(PG_reserved, &({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageSwapBacked(struct page *page) { return test_bit(PG_swapbacked, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageSwapBacked(struct page *page) { _set_bit(PG_swapbacked,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageSwapBacked(struct page *page) { _clear_bit(PG_swapbacked,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageSwapBacked(struct page *page) { __clear_bit(PG_swapbacked, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageSwapBacked(struct page *page) { __set_bit(PG_swapbacked, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PagePrivate(struct page *page) { return test_bit(PG_private, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPagePrivate(struct page *page) { _set_bit(PG_private,&page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPagePrivate(struct page *page) { _clear_bit(PG_private,&page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPagePrivate(struct page *page) { __set_bit(PG_private, &page->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPagePrivate(struct page *page) { __clear_bit(PG_private, &page->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PagePrivate2(struct page *page) { return test_bit(PG_private_2, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPagePrivate2(struct page *page) { _set_bit(PG_private_2,&page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPagePrivate2(struct page *page) { _clear_bit(PG_private_2,&page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestSetPagePrivate2(struct page *page) { return _test_and_set_bit(PG_private_2,&page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPagePrivate2(struct page *page) { return _test_and_clear_bit(PG_private_2,&page->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageOwnerPriv1(struct page *page) { return test_bit(PG_owner_priv_1, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageOwnerPriv1(struct page *page) { _set_bit(PG_owner_priv_1,&page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageOwnerPriv1(struct page *page) { _clear_bit(PG_owner_priv_1,&page->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageOwnerPriv1(struct page *page) { return _test_and_clear_bit(PG_owner_priv_1,&page->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageWriteback(struct page *page) { return test_bit(PG_writeback, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestSetPageWriteback(struct page *page) { return _test_and_set_bit(PG_writeback,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageWriteback(struct page *page) { return _test_and_clear_bit(PG_writeback,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageMappedToDisk(struct page *page) { return test_bit(PG_mappedtodisk, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageMappedToDisk(struct page *page) { _set_bit(PG_mappedtodisk,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageMappedToDisk(struct page *page) { _clear_bit(PG_mappedtodisk,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageReclaim(struct page *page) { return test_bit(PG_reclaim, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageReclaim(struct page *page) { _set_bit(PG_reclaim,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageReclaim(struct page *page) { _clear_bit(PG_reclaim,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageReclaim(struct page *page) { return _test_and_clear_bit(PG_reclaim,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageReadahead(struct page *page) { return test_bit(PG_reclaim, &({ ((void)(sizeof(( long)(0 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageReadahead(struct page *page) { _set_bit(PG_reclaim,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageReadahead(struct page *page) { _clear_bit(PG_reclaim,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageReadahead(struct page *page) { return _test_and_clear_bit(PG_reclaim,&({ ((void)(sizeof(( long)(1 && PageCompound(page))))); page;})->flags); }
# 334 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageSwapCache(struct page *page)
{
return PageSwapBacked(page) && test_bit(PG_swapcache, &page->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageSwapCache(struct page *page) { _set_bit(PG_swapcache,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageSwapCache(struct page *page) { _clear_bit(PG_swapcache,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageUnevictable(struct page *page) { return test_bit(PG_unevictable, &compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageUnevictable(struct page *page) { _set_bit(PG_unevictable,&compound_head(page)->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageUnevictable(struct page *page) { _clear_bit(PG_unevictable,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageUnevictable(struct page *page) { __clear_bit(PG_unevictable, &compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageUnevictable(struct page *page) { return _test_and_clear_bit(PG_unevictable,&compound_head(page)->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageMlocked(struct page *page) { return test_bit(PG_mlocked, &({ ((void)(sizeof(( long)(0 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageMlocked(struct page *page) { _set_bit(PG_mlocked,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageMlocked(struct page *page) { _clear_bit(PG_mlocked,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageMlocked(struct page *page) { __clear_bit(PG_mlocked, &({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestSetPageMlocked(struct page *page) { return _test_and_set_bit(PG_mlocked,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int TestClearPageMlocked(struct page *page) { return _test_and_clear_bit(PG_mlocked,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
# 364 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageUncached(const struct page *page) { return 0; } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void SetPageUncached(struct page *page) { } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ClearPageUncached(struct page *page) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageHWPoison(const struct page *page) { return 0; } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void SetPageHWPoison(struct page *page) { } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ClearPageHWPoison(struct page *page) { }
# 384 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageRTKCache(struct page *page) { return test_bit(PG_rtkcache, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageRTKCache(struct page *page) { _set_bit(PG_rtkcache,&page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageRTKCache(struct page *page) { _clear_bit(PG_rtkcache,&page->flags); }
# 409 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageMappingFlags(struct page *page)
{
return ((unsigned long)page->mapping & (0x1 | 0x2)) != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageAnon(struct page *page)
{
page = compound_head(page);
return ((unsigned long)page->mapping & 0x1) != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int __PageMovable(struct page *page)
{
return ((unsigned long)page->mapping & (0x1 | 0x2)) ==
0x2;
}
# 433 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageKsm(struct page *page)
{
page = compound_head(page);
return ((unsigned long)page->mapping & (0x1 | 0x2)) ==
(0x1 | 0x2);
}
u64 stable_page_flags(struct page *page);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageUptodate(struct page *page)
{
int ret;
page = compound_head(page);
ret = test_bit(PG_uptodate, &(page)->flags);
# 458 "./include/linux/page-flags.h"
if (ret)
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageUptodate(struct page *page)
{
((void)(sizeof(( long)(PageTail(page)))));
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
__set_bit(PG_uptodate, &page->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void SetPageUptodate(struct page *page)
{
((void)(sizeof(( long)(PageTail(page)))));
__asm__ __volatile__ ("dmb " "ishst" : : : "memory");
_set_bit(PG_uptodate,&page->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageUptodate(struct page *page) { _clear_bit(PG_uptodate,&({ ((void)(sizeof(( long)(1 && PageTail(page))))); compound_head(page);})->flags); }
int test_clear_page_writeback(struct page *page);
int __test_set_page_writeback(struct page *page, bool keep_write);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_page_writeback(struct page *page)
{
__test_set_page_writeback(page, false);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_page_writeback_keepwrite(struct page *page)
{
__test_set_page_writeback(page, true);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageHead(struct page *page) { return test_bit(PG_head, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageHead(struct page *page) { __set_bit(PG_head, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageHead(struct page *page) { __clear_bit(PG_head, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void ClearPageHead(struct page *page) { _clear_bit(PG_head,&page->flags); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void set_compound_head(struct page *page, struct page *head)
{
({ union { typeof(page->compound_head) __val; char __c[1]; } __u = { .__val = ( typeof(page->compound_head)) ((unsigned long)head + 1) }; __write_once_size(&(page->compound_head), __u.__c, sizeof(page->compound_head)); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void clear_compound_head(struct page *page)
{
({ union { typeof(page->compound_head) __val; char __c[1]; } __u = { .__val = ( typeof(page->compound_head)) (0) }; __write_once_size(&(page->compound_head), __u.__c, sizeof(page->compound_head)); __u.__val; });
}
# 530 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageHuge(const struct page *page) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageHeadHuge(const struct page *page) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool page_huge_active(struct page *page)
{
return 0;
}
# 654 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageTransHuge(const struct page *page) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageTransCompound(const struct page *page) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageTransCompoundMap(const struct page *page) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageTransTail(const struct page *page) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageDoubleMap(const struct page *page) { return 0; } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void SetPageDoubleMap(struct page *page) { } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ClearPageDoubleMap(struct page *page) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int TestSetPageDoubleMap(struct page *page) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int TestClearPageDoubleMap(struct page *page) { return 0; }
# 692 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageBuddy(struct page *page) { return ({ union { typeof((&page->_mapcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); else __read_once_size_nocheck(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); do { } while (0); __u.__val; }) == (-128); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageBuddy(struct page *page) { ((void)(sizeof(( long)(({ union { typeof((&page->_mapcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); else __read_once_size_nocheck(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); do { } while (0); __u.__val; }) != -1)))); ({ union { typeof(((&page->_mapcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_mapcount)->counter))) (((-128))) }; __write_once_size(&(((&page->_mapcount)->counter)), __u.__c, sizeof(((&page->_mapcount)->counter))); __u.__val; }); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageBuddy(struct page *page) { ((void)(sizeof(( long)(!PageBuddy(page))))); ({ union { typeof(((&page->_mapcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_mapcount)->counter))) ((-1)) }; __write_once_size(&(((&page->_mapcount)->counter)), __u.__c, sizeof(((&page->_mapcount)->counter))); __u.__val; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageBalloon(struct page *page) { return ({ union { typeof((&page->_mapcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); else __read_once_size_nocheck(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); do { } while (0); __u.__val; }) == (-256); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageBalloon(struct page *page) { ((void)(sizeof(( long)(({ union { typeof((&page->_mapcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); else __read_once_size_nocheck(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); do { } while (0); __u.__val; }) != -1)))); ({ union { typeof(((&page->_mapcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_mapcount)->counter))) (((-256))) }; __write_once_size(&(((&page->_mapcount)->counter)), __u.__c, sizeof(((&page->_mapcount)->counter))); __u.__val; }); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageBalloon(struct page *page) { ((void)(sizeof(( long)(!PageBalloon(page))))); ({ union { typeof(((&page->_mapcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_mapcount)->counter))) ((-1)) }; __write_once_size(&(((&page->_mapcount)->counter)), __u.__c, sizeof(((&page->_mapcount)->counter))); __u.__val; }); }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageKmemcg(struct page *page) { return ({ union { typeof((&page->_mapcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); else __read_once_size_nocheck(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); do { } while (0); __u.__val; }) == (-512); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageKmemcg(struct page *page) { ((void)(sizeof(( long)(({ union { typeof((&page->_mapcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); else __read_once_size_nocheck(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); do { } while (0); __u.__val; }) != -1)))); ({ union { typeof(((&page->_mapcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_mapcount)->counter))) (((-512))) }; __write_once_size(&(((&page->_mapcount)->counter)), __u.__c, sizeof(((&page->_mapcount)->counter))); __u.__val; }); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageKmemcg(struct page *page) { ((void)(sizeof(( long)(!PageKmemcg(page))))); ({ union { typeof(((&page->_mapcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_mapcount)->counter))) ((-1)) }; __write_once_size(&(((&page->_mapcount)->counter)), __u.__c, sizeof(((&page->_mapcount)->counter))); __u.__val; }); }
extern bool is_free_buddy_page(struct page *page);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) int PageIsolated(struct page *page) { return test_bit(PG_isolated, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __SetPageIsolated(struct page *page) { __set_bit(PG_isolated, &page->flags); } static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void __ClearPageIsolated(struct page *page) { __clear_bit(PG_isolated, &page->flags); };
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int PageSlabPfmemalloc(struct page *page)
{
((void)(sizeof(( long)(!PageSlab(page)))));
return PageActive(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void SetPageSlabPfmemalloc(struct page *page)
{
((void)(sizeof(( long)(!PageSlab(page)))));
SetPageActive(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __ClearPageSlabPfmemalloc(struct page *page)
{
((void)(sizeof(( long)(!PageSlab(page)))));
__ClearPageActive(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ClearPageSlabPfmemalloc(struct page *page)
{
((void)(sizeof(( long)(!PageSlab(page)))));
ClearPageActive(page);
}
# 777 "./include/linux/page-flags.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_has_private(struct page *page)
{
return !!(page->flags & (1UL << PG_private | 1UL << PG_private_2));
}
# 8 "./include/linux/page_ref.h" 2
# 1 "./include/linux/tracepoint-defs.h" 1
# 12 "./include/linux/tracepoint-defs.h"
# 1 "./include/linux/static_key.h" 1
# 1 "./include/linux/jump_label.h" 1
# 83 "./include/linux/jump_label.h"
extern bool static_key_initialized;
# 114 "./include/linux/jump_label.h"
struct static_key {
atomic_t enabled;
};
# 126 "./include/linux/jump_label.h"
enum jump_label_type {
JUMP_LABEL_NOP = 0,
JUMP_LABEL_JMP,
};
struct module;
# 191 "./include/linux/jump_label.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int static_key_count(struct static_key *key)
{
return ({ union { typeof((&key->enabled)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); else __read_once_size_nocheck(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); do { } while (0); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void jump_label_init(void)
{
static_key_initialized = true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) bool static_key_false(struct static_key *key)
{
if (__builtin_expect(!!(static_key_count(key) > 0), 0))
return true;
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) bool static_key_true(struct static_key *key)
{
if (__builtin_expect(!!(static_key_count(key) > 0), 1))
return true;
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void static_key_slow_inc(struct static_key *key)
{
({ int __ret_warn_on = !!(!static_key_initialized); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_fmt("include/linux/jump_label.h", 217, "%s used before call to jump_label_init", __func__); __builtin_expect(!!(__ret_warn_on), 0); });
atomic_add(1, &key->enabled);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void static_key_slow_dec(struct static_key *key)
{
({ int __ret_warn_on = !!(!static_key_initialized); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_fmt("include/linux/jump_label.h", 223, "%s used before call to jump_label_init", __func__); __builtin_expect(!!(__ret_warn_on), 0); });
atomic_sub(1, &key->enabled);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int jump_label_text_reserved(void *start, void *end)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void jump_label_lock(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void jump_label_unlock(void) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int jump_label_apply_nops(struct module *mod)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void static_key_enable(struct static_key *key)
{
({ int __ret_warn_on = !!(!static_key_initialized); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_fmt("include/linux/jump_label.h", 245, "%s used before call to jump_label_init", __func__); __builtin_expect(!!(__ret_warn_on), 0); });
if (({ union { typeof((&key->enabled)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); else __read_once_size_nocheck(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); do { } while (0); __u.__val; }) != 0) {
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(({ union { typeof((&key->enabled)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); else __read_once_size_nocheck(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); do { } while (0); __u.__val; }) != 1); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/jump_label.h", 248); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
return;
}
({ union { typeof(((&key->enabled)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&key->enabled)->counter))) ((1)) }; __write_once_size(&(((&key->enabled)->counter)), __u.__c, sizeof(((&key->enabled)->counter))); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void static_key_disable(struct static_key *key)
{
({ int __ret_warn_on = !!(!static_key_initialized); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_fmt("include/linux/jump_label.h", 256, "%s used before call to jump_label_init", __func__); __builtin_expect(!!(__ret_warn_on), 0); });
if (({ union { typeof((&key->enabled)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); else __read_once_size_nocheck(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); do { } while (0); __u.__val; }) != 1) {
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(({ union { typeof((&key->enabled)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); else __read_once_size_nocheck(&((&key->enabled)->counter), __u.__c, sizeof((&key->enabled)->counter)); do { } while (0); __u.__val; }) != 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/jump_label.h", 259); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
return;
}
({ union { typeof(((&key->enabled)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&key->enabled)->counter))) ((0)) }; __write_once_size(&(((&key->enabled)->counter)), __u.__c, sizeof(((&key->enabled)->counter))); __u.__val; });
}
# 285 "./include/linux/jump_label.h"
struct static_key_true {
struct static_key key;
};
struct static_key_false {
struct static_key key;
};
# 318 "./include/linux/jump_label.h"
extern bool ____wrong_branch_error(void);
# 2 "./include/linux/static_key.h" 2
# 13 "./include/linux/tracepoint-defs.h" 2
struct trace_print_flags {
unsigned long mask;
const char *name;
};
struct trace_print_flags_u64 {
unsigned long long mask;
const char *name;
};
struct tracepoint_func {
void *func;
void *data;
int prio;
};
struct tracepoint {
const char *name;
struct static_key key;
int (*regfunc)(void);
void (*unregfunc)(void);
struct tracepoint_func *funcs;
};
# 9 "./include/linux/page_ref.h" 2
extern struct tracepoint __tracepoint_page_ref_set;
extern struct tracepoint __tracepoint_page_ref_mod;
extern struct tracepoint __tracepoint_page_ref_mod_and_test;
extern struct tracepoint __tracepoint_page_ref_mod_and_return;
extern struct tracepoint __tracepoint_page_ref_mod_unless;
extern struct tracepoint __tracepoint_page_ref_freeze;
extern struct tracepoint __tracepoint_page_ref_unfreeze;
# 41 "./include/linux/page_ref.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __page_ref_set(struct page *page, int v)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __page_ref_mod(struct page *page, int v)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __page_ref_mod_and_test(struct page *page, int v, int ret)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __page_ref_mod_and_return(struct page *page, int v, int ret)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __page_ref_mod_unless(struct page *page, int v, int u)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __page_ref_freeze(struct page *page, int v, int ret)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __page_ref_unfreeze(struct page *page, int v)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_ref_count(struct page *page)
{
return ({ union { typeof((&page->_refcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_refcount)->counter), __u.__c, sizeof((&page->_refcount)->counter)); else __read_once_size_nocheck(&((&page->_refcount)->counter), __u.__c, sizeof((&page->_refcount)->counter)); do { } while (0); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_count(struct page *page)
{
return ({ union { typeof((&compound_head(page)->_refcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&compound_head(page)->_refcount)->counter), __u.__c, sizeof((&compound_head(page)->_refcount)->counter)); else __read_once_size_nocheck(&((&compound_head(page)->_refcount)->counter), __u.__c, sizeof((&compound_head(page)->_refcount)->counter)); do { } while (0); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_page_count(struct page *page, int v)
{
({ union { typeof(((&page->_refcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_refcount)->counter))) ((v)) }; __write_once_size(&(((&page->_refcount)->counter)), __u.__c, sizeof(((&page->_refcount)->counter))); __u.__val; });
if (false)
__page_ref_set(page, v);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_page_count(struct page *page)
{
set_page_count(page, 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_ref_add(struct page *page, int nr)
{
atomic_add(nr, &page->_refcount);
if (false)
__page_ref_mod(page, nr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_ref_sub(struct page *page, int nr)
{
atomic_sub(nr, &page->_refcount);
if (false)
__page_ref_mod(page, -nr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_ref_inc(struct page *page)
{
atomic_add(1, &page->_refcount);
if (false)
__page_ref_mod(page, 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_ref_dec(struct page *page)
{
atomic_sub(1, &page->_refcount);
if (false)
__page_ref_mod(page, -1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_ref_sub_and_test(struct page *page, int nr)
{
int ret = (({ typeof(atomic_sub_return_relaxed(nr, &page->_refcount)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_sub_return_relaxed(nr, &page->_refcount); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == 0);
if (false)
__page_ref_mod_and_test(page, -nr, ret);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_ref_inc_return(struct page *page)
{
int ret = ({ typeof((atomic_add_return_relaxed(1, &page->_refcount))) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = (atomic_add_return_relaxed(1, &page->_refcount)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
if (false)
__page_ref_mod_and_return(page, 1, ret);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_ref_dec_and_test(struct page *page)
{
int ret = (({ typeof(atomic_sub_return_relaxed(1, &page->_refcount)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_sub_return_relaxed(1, &page->_refcount); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == 0);
if (false)
__page_ref_mod_and_test(page, -1, ret);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_ref_dec_return(struct page *page)
{
int ret = ({ typeof((atomic_sub_return_relaxed(1, &page->_refcount))) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = (atomic_sub_return_relaxed(1, &page->_refcount)); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; });
if (false)
__page_ref_mod_and_return(page, -1, ret);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_ref_add_unless(struct page *page, int nr, int u)
{
int ret = atomic_add_unless(&page->_refcount, nr, u);
if (false)
__page_ref_mod_unless(page, nr, ret);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_ref_freeze(struct page *page, int count)
{
int ret = __builtin_expect(!!(({ typeof(atomic_cmpxchg_relaxed(&page->_refcount, count, 0)) __ret; __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret = atomic_cmpxchg_relaxed(&page->_refcount, count, 0); __asm__ __volatile__ ("dmb " "ish" : : : "memory"); __ret; }) == count), 1);
if (false)
__page_ref_freeze(page, count, ret);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_ref_unfreeze(struct page *page, int count)
{
((void)(sizeof(( long)(page_count(page) != 0))));
((void)(sizeof(( long)(count == 0))));
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
({ union { typeof(((&page->_refcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&page->_refcount)->counter))) ((count)) }; __write_once_size(&(((&page->_refcount)->counter)), __u.__c, sizeof(((&page->_refcount)->counter))); __u.__val; });
if (false)
__page_ref_unfreeze(page, count);
}
# 27 "./include/linux/mm.h" 2
# 1 "./include/linux/memremap.h" 1
# 1 "./include/linux/mm.h" 1
# 5 "./include/linux/memremap.h" 2
# 1 "./arch/arm/include/asm/pgtable.h" 1
# 14 "./arch/arm/include/asm/pgtable.h"
# 1 "./arch/arm/include/asm/proc-fns.h" 1
# 16 "./arch/arm/include/asm/proc-fns.h"
# 1 "./arch/arm/include/asm/glue-proc.h" 1
# 14 "./arch/arm/include/asm/glue-proc.h"
# 1 "./arch/arm/include/asm/glue.h" 1
# 15 "./arch/arm/include/asm/glue-proc.h" 2
# 17 "./arch/arm/include/asm/proc-fns.h" 2
struct mm_struct;
struct processor {
void (*_data_abort)(unsigned long pc);
unsigned long (*_prefetch_abort)(unsigned long lr);
void (*_proc_init)(void);
void (*check_bugs)(void);
void (*_proc_fin)(void);
void (*reset)(unsigned long addr, bool hvc) __attribute__((noreturn));
int (*_do_idle)(void);
void (*dcache_clean_area)(void *addr, int size);
void (*switch_mm)(phys_addr_t pgd_phys, struct mm_struct *mm);
void (*set_pte_ext)(pte_t *ptep, pte_t pte, unsigned int ext);
unsigned int suspend_size;
void (*do_suspend)(void *);
void (*do_resume)(void *);
};
# 106 "./arch/arm/include/asm/proc-fns.h"
extern struct processor processor;
# 131 "./arch/arm/include/asm/proc-fns.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_proc_vtable(const struct processor *p)
{
processor = *p;
}
# 151 "./arch/arm/include/asm/proc-fns.h"
extern void cpu_resume(void);
# 15 "./arch/arm/include/asm/pgtable.h" 2
# 24 "./arch/arm/include/asm/pgtable.h"
# 1 "./include/asm-generic/pgtable-nopud.h" 1
# 1 "./include/asm-generic/pgtable-nop4d-hack.h" 1
# 1 "./include/asm-generic/5level-fixup.h" 1
# 7 "./include/asm-generic/pgtable-nop4d-hack.h" 2
typedef struct { pgd_t pgd; } pud_t;
# 27 "./include/asm-generic/pgtable-nop4d-hack.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pgd_none(pgd_t pgd) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pgd_bad(pgd_t pgd) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pgd_present(pgd_t pgd) { return 1; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgd_clear(pgd_t *pgd) { }
# 40 "./include/asm-generic/pgtable-nop4d-hack.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pud_t *pud_offset(pgd_t *pgd, unsigned long address)
{
return (pud_t *)pgd;
}
# 9 "./include/asm-generic/pgtable-nopud.h" 2
# 25 "./arch/arm/include/asm/pgtable.h" 2
# 1 "./arch/arm/include/asm/pgtable-hwdef.h" 1
# 16 "./arch/arm/include/asm/pgtable-hwdef.h"
# 1 "./arch/arm/include/asm/pgtable-2level-hwdef.h" 1
# 17 "./arch/arm/include/asm/pgtable-hwdef.h" 2
# 27 "./arch/arm/include/asm/pgtable.h" 2
# 1 "./arch/arm/include/asm/tlbflush.h" 1
# 19 "./arch/arm/include/asm/tlbflush.h"
# 1 "./arch/arm/include/asm/glue.h" 1
# 20 "./arch/arm/include/asm/tlbflush.h" 2
# 210 "./arch/arm/include/asm/tlbflush.h"
struct cpu_tlb_fns {
void (*flush_user_range)(unsigned long, unsigned long, struct vm_area_struct *);
void (*flush_kern_range)(unsigned long, unsigned long);
unsigned long tlb_flags;
};
# 234 "./arch/arm/include/asm/tlbflush.h"
extern struct cpu_tlb_fns cpu_tlb;
# 326 "./arch/arm/include/asm/tlbflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __local_flush_tlb_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 9) | (1 << 12))) asm("mcr " "p15, 0, %0, " "c8, c7, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 9) | (1 << 12))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 9) | (1 << 12)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 10) | (1 << 13))) asm("mcr " "p15, 0, %0, " "c8, c6, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 10) | (1 << 13))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c6, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 10) | (1 << 13)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11) | (1 << 14))) asm("mcr " "p15, 0, %0, " "c8, c5, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11) | (1 << 14))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c5, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 11) | (1 << 14)) : "cc"); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void local_flush_tlb_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "nshst" : : : "memory");
__local_flush_tlb_all();
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 21))) asm("mcr " "p15, 0, %0, " "c8, c7, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 21))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 21)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))))) {
__asm__ __volatile__ ("dsb " "nsh" : : : "memory");
__asm__ __volatile__ ("isb " "" : : : "memory");
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __flush_tlb_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
__local_flush_tlb_all();
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 21))) asm("mcr " "p15, 0, %0, " "c8, c3, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 21))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c3, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 21)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))))) {
__asm__ __volatile__ ("dsb " "ish" : : : "memory");
__asm__ __volatile__ ("isb " "" : : : "memory");
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __local_flush_tlb_mm(struct mm_struct *mm)
{
const int zero = 0;
const int asid = ((unsigned int)((mm)->context.id.counter & ~((~0ULL) << 8)));
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 9)|(1 << 10)|(1 << 11))) {
if (cpumask_test_cpu((current_thread_info()->cpu), mm_cpumask(mm))) {
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 9))) asm("mcr " "p15, 0, %0, " "c8, c7, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 9))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 9)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 10))) asm("mcr " "p15, 0, %0, " "c8, c6, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 10))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c6, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 10)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11))) asm("mcr " "p15, 0, %0, " "c8, c5, 0" : : "r" (zero) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c5, 0" : : "r" (zero), "r" (__tlb_flag), "Ir" ((1 << 11)) : "cc"); } while (0);
}
}
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 16))) asm("mcr " "p15, 0, %0, " "c8, c7, 2" : : "r" (asid) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 16))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 2" : : "r" (asid), "r" (__tlb_flag), "Ir" ((1 << 16)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 17))) asm("mcr " "p15, 0, %0, " "c8, c6, 2" : : "r" (asid) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 17))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c6, 2" : : "r" (asid), "r" (__tlb_flag), "Ir" ((1 << 17)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 18))) asm("mcr " "p15, 0, %0, " "c8, c5, 2" : : "r" (asid) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 18))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c5, 2" : : "r" (asid), "r" (__tlb_flag), "Ir" ((1 << 18)) : "cc"); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void local_flush_tlb_mm(struct mm_struct *mm)
{
const int asid = ((unsigned int)((mm)->context.id.counter & ~((~0ULL) << 8)));
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "nshst" : : : "memory");
__local_flush_tlb_mm(mm);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 22))) asm("mcr " "p15, 0, %0, " "c8, c7, 2" : : "r" (asid) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 22))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 2" : : "r" (asid), "r" (__tlb_flag), "Ir" ((1 << 22)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28)))))
__asm__ __volatile__ ("dsb " "nsh" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __flush_tlb_mm(struct mm_struct *mm)
{
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
__local_flush_tlb_mm(mm);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 22))) asm("mcr " "p15, 0, %0, " "c8, c3, 0" : : "r" (0) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 22))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c3, 0" : : "r" (0), "r" (__tlb_flag), "Ir" ((1 << 22)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28)))))
__asm__ __volatile__ ("dsb " "ish" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
uaddr = (uaddr & (~((1 << 12) - 1))) | ((unsigned int)((vma->vm_mm)->context.id.counter & ~((~0ULL) << 8)));
if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 1)|(1 << 2)|(1 << 3)|(1 << 11)) &&
cpumask_test_cpu((current_thread_info()->cpu), mm_cpumask(vma->vm_mm))) {
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 1))) asm("mcr " "p15, 0, %0, " "c8, c7, 1" : : "r" (uaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 1))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 1" : : "r" (uaddr), "r" (__tlb_flag), "Ir" ((1 << 1)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 2))) asm("mcr " "p15, 0, %0, " "c8, c6, 1" : : "r" (uaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 2))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c6, 1" : : "r" (uaddr), "r" (__tlb_flag), "Ir" ((1 << 2)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3))) asm("mcr " "p15, 0, %0, " "c8, c5, 1" : : "r" (uaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c5, 1" : : "r" (uaddr), "r" (__tlb_flag), "Ir" ((1 << 3)) : "cc"); } while (0);
if (!((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3)))) && ((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11)))))
asm("mcr p15, 0, %0, c8, c5, 0" : : "r" (zero) : "cc");
}
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 4))) asm("mcr " "p15, 0, %0, " "c8, c7, 1" : : "r" (uaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 4))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 1" : : "r" (uaddr), "r" (__tlb_flag), "Ir" ((1 << 4)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 5))) asm("mcr " "p15, 0, %0, " "c8, c6, 1" : : "r" (uaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 5))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c6, 1" : : "r" (uaddr), "r" (__tlb_flag), "Ir" ((1 << 5)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 6))) asm("mcr " "p15, 0, %0, " "c8, c5, 1" : : "r" (uaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 6))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c5, 1" : : "r" (uaddr), "r" (__tlb_flag), "Ir" ((1 << 6)) : "cc"); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
local_flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
{
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
uaddr = (uaddr & (~((1 << 12) - 1))) | ((unsigned int)((vma->vm_mm)->context.id.counter & ~((~0ULL) << 8)));
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "nshst" : : : "memory");
__local_flush_tlb_page(vma, uaddr);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("mcr " "p15, 0, %0, " "c8, c7, 1" : : "r" (uaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 1" : : "r" (uaddr), "r" (__tlb_flag), "Ir" ((1 << 20)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28)))))
__asm__ __volatile__ ("dsb " "nsh" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
__flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
{
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
uaddr = (uaddr & (~((1 << 12) - 1))) | ((unsigned int)((vma->vm_mm)->context.id.counter & ~((~0ULL) << 8)));
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
__local_flush_tlb_page(vma, uaddr);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("mcr " "p15, 0, %0, " "c8, c3, 3" : : "r" (uaddr & (~((1 << 12) - 1))) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c3, 3" : : "r" (uaddr & (~((1 << 12) - 1))), "r" (__tlb_flag), "Ir" ((1 << 20)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28)))))
__asm__ __volatile__ ("dsb " "ish" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __local_flush_tlb_kernel_page(unsigned long kaddr)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 1))) asm("mcr " "p15, 0, %0, " "c8, c7, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 1))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 1)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 2))) asm("mcr " "p15, 0, %0, " "c8, c6, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 2))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c6, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 2)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3))) asm("mcr " "p15, 0, %0, " "c8, c5, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c5, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 3)) : "cc"); } while (0);
if (!((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 3)))) && ((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 11)))))
asm("mcr p15, 0, %0, c8, c5, 0" : : "r" (zero) : "cc");
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 4))) asm("mcr " "p15, 0, %0, " "c8, c7, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 4))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 4)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 5))) asm("mcr " "p15, 0, %0, " "c8, c6, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 5))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c6, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 5)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 6))) asm("mcr " "p15, 0, %0, " "c8, c5, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 6))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c5, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 6)) : "cc"); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void local_flush_tlb_kernel_page(unsigned long kaddr)
{
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
kaddr &= (~((1 << 12) - 1));
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "nshst" : : : "memory");
__local_flush_tlb_kernel_page(kaddr);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("mcr " "p15, 0, %0, " "c8, c7, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c7, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 20)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))))) {
__asm__ __volatile__ ("dsb " "nsh" : : : "memory");
__asm__ __volatile__ ("isb " "" : : : "memory");
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __flush_tlb_kernel_page(unsigned long kaddr)
{
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
kaddr &= (~((1 << 12) - 1));
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
__local_flush_tlb_kernel_page(kaddr);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("mcr " "p15, 0, %0, " "c8, c3, 1" : : "r" (kaddr) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 20))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c8, c3, 1" : : "r" (kaddr), "r" (__tlb_flag), "Ir" ((1 << 20)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 28))))) {
__asm__ __volatile__ ("dsb " "ish" : : : "memory");
__asm__ __volatile__ ("isb " "" : : : "memory");
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __local_flush_bp_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 19))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 19)))))
asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void local_flush_bp_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
__local_flush_bp_all();
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 23))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 23)))))
asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __flush_bp_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
__local_flush_bp_all();
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 23))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 23)))))
asm("mcr p15, 0, %0, c7, c1, 6" : : "r" (zero));
}
# 580 "./arch/arm/include/asm/tlbflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void flush_pmd_entry(void *pmd)
{
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 30))) asm("mcr " "p15, 0, %0, " "c7, c10, 1 @ flush_pmd" : : "r" (pmd) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 30))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c7, c10, 1 @ flush_pmd" : : "r" (pmd), "r" (__tlb_flag), "Ir" ((1 << 30)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 29))) asm("mcr " "p15, 1, %0, " "c15, c9, 1 @ L2 flush_pmd" : : "r" (pmd) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 29))) asm("tst %1, %2\n\t" "mcrne " "p15, 1, %0, " "c15, c9, 1 @ L2 flush_pmd" : : "r" (pmd), "r" (__tlb_flag), "Ir" ((1 << 29)) : "cc"); } while (0);
if (((((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31))) || (__tlb_flag & (0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 31)))))
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clean_pmd_entry(void *pmd)
{
const unsigned int __tlb_flag = cpu_tlb.tlb_flags;
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 30))) asm("mcr " "p15, 0, %0, " "c7, c10, 1 @ flush_pmd" : : "r" (pmd) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 30))) asm("tst %1, %2\n\t" "mcrne " "p15, 0, %0, " "c7, c10, 1 @ flush_pmd" : : "r" (pmd), "r" (__tlb_flag), "Ir" ((1 << 30)) : "cc"); } while (0);
do { if (((-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (-1UL) & (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) & ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 29))) asm("mcr " "p15, 1, %0, " "c15, c9, 1 @ L2 flush_pmd" : : "r" (pmd) : "cc"); else if ((0 | 0 | 0 | 0 | 0 | 0 | (((1 << 31) | (1 << 28) | (1 << 21) | (1 << 20) | (1 << 22) | (1 << 23)) | ((1 << 31) | (1 << 30) | (1 << 28) | (1 << 12) | (1 << 4) | (1 << 16) | (1 << 19)))) & ((1 << 29))) asm("tst %1, %2\n\t" "mcrne " "p15, 1, %0, " "c15, c9, 1 @ L2 flush_pmd" : : "r" (pmd), "r" (__tlb_flag), "Ir" ((1 << 29)) : "cc"); } while (0);
}
# 619 "./arch/arm/include/asm/tlbflush.h"
extern void flush_tlb_all(void);
extern void flush_tlb_mm(struct mm_struct *mm);
extern void flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr);
extern void flush_tlb_kernel_page(unsigned long kaddr);
extern void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end);
extern void flush_tlb_kernel_range(unsigned long start, unsigned long end);
extern void flush_bp_all(void);
# 638 "./arch/arm/include/asm/tlbflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void update_mmu_cache(struct vm_area_struct *vma,
unsigned long addr, pte_t *ptep)
{
}
# 674 "./arch/arm/include/asm/tlbflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void erratum_a15_798181_init(void) {}
extern bool (*erratum_a15_798181_handler)(void);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool erratum_a15_798181(void)
{
if (__builtin_expect(!!(0 && erratum_a15_798181_handler), 0))
return erratum_a15_798181_handler();
return false;
}
# 30 "./arch/arm/include/asm/pgtable.h" 2
# 1 "./arch/arm/include/asm/pgtable-2level.h" 1
# 189 "./arch/arm/include/asm/pgtable-2level.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t *pmd_offset(pud_t *pud, unsigned long addr)
{
return (pmd_t *)pud;
}
# 217 "./arch/arm/include/asm/pgtable-2level.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mkspecial(pte_t pte) { return pte; }
# 35 "./arch/arm/include/asm/pgtable.h" 2
# 52 "./arch/arm/include/asm/pgtable.h"
extern void __pte_error(const char *file, int line, pte_t);
extern void __pmd_error(const char *file, int line, pmd_t);
extern void __pgd_error(const char *file, int line, pgd_t);
# 84 "./arch/arm/include/asm/pgtable.h"
extern pgprot_t pgprot_user;
extern pgprot_t pgprot_kernel;
extern pgprot_t pgprot_hyp_device;
extern pgprot_t pgprot_s2;
extern pgprot_t pgprot_s2_device;
# 135 "./arch/arm/include/asm/pgtable.h"
struct file;
extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size, pgprot_t vma_prot);
# 176 "./arch/arm/include/asm/pgtable.h"
extern struct page *empty_zero_page;
extern pgd_t swapper_pg_dir[2048];
# 192 "./arch/arm/include/asm/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t *pmd_page_vaddr(pmd_t pmd)
{
return ((void *)__phys_to_virt((phys_addr_t)((pmd) & (~0UL) & (s32)(~((1 << 12) - 1)))));
}
# 243 "./arch/arm/include/asm/pgtable.h"
extern void __sync_icache_dcache(pte_t pteval);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_pte_at(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pteval)
{
unsigned long ext = 0;
if (addr < ((0xC0000000UL) - (0x01000000UL)) && ((((u32)((((pteval_t)(1)) << 0)) == ((((pteval_t)(1)) << 0)) ? ((pteval)) & ((((pteval_t)(1)) << 0)) : !!(((pteval)) & ((((pteval_t)(1)) << 0))))) && ((u32)((((pteval_t)(1)) << 8)) == ((((pteval_t)(1)) << 8)) ? ((pteval)) & ((((pteval_t)(1)) << 8)) : !!(((pteval)) & ((((pteval_t)(1)) << 8)))) && (((u32)((((pteval_t)(1)) << 1)) == ((((pteval_t)(1)) << 1)) ? ((pteval)) & ((((pteval_t)(1)) << 1)) : !!(((pteval)) & ((((pteval_t)(1)) << 1))))))) {
if (!(0))
__sync_icache_dcache(pteval);
ext |= (((pteval_t)(1)) << 11);
}
processor.set_pte_ext(ptep,pteval,ext);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t clear_pte_bit(pte_t pte, pgprot_t prot)
{
(pte) &= ~(prot);
return pte;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t set_pte_bit(pte_t pte, pgprot_t prot)
{
(pte) |= (prot);
return pte;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_wrprotect(pte_t pte)
{
return set_pte_bit(pte, ((((pteval_t)(1)) << 7)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mkwrite(pte_t pte)
{
return clear_pte_bit(pte, ((((pteval_t)(1)) << 7)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mkclean(pte_t pte)
{
return clear_pte_bit(pte, ((((pteval_t)(1)) << 6)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mkdirty(pte_t pte)
{
return set_pte_bit(pte, ((((pteval_t)(1)) << 6)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mkold(pte_t pte)
{
return clear_pte_bit(pte, ((((pteval_t)(1)) << 1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mkyoung(pte_t pte)
{
return set_pte_bit(pte, ((((pteval_t)(1)) << 1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mkexec(pte_t pte)
{
return clear_pte_bit(pte, ((((pteval_t)(1)) << 9)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mknexec(pte_t pte)
{
return set_pte_bit(pte, ((((pteval_t)(1)) << 9)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
const pteval_t mask = (((pteval_t)(1)) << 9) | (((pteval_t)(1)) << 7) | (((pteval_t)(1)) << 8) |
(((pteval_t)(1)) << 11) | (((pteval_t)(1)) << 0);
(pte) = ((pte) & ~mask) | ((newprot) & mask);
return pte;
}
# 354 "./arch/arm/include/asm/pgtable.h"
# 1 "./include/asm-generic/pgtable.h" 1
# 30 "./include/asm-generic/pgtable.h"
extern int ptep_set_access_flags(struct vm_area_struct *vma,
unsigned long address, pte_t *ptep,
pte_t entry, int dirty);
# 44 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmdp_set_access_flags(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp,
pmd_t entry, int dirty)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_45(void) ; if (__cond) __compiletime_assert_45(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pudp_set_access_flags(struct vm_area_struct *vma,
unsigned long address, pud_t *pudp,
pud_t entry, int dirty)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_46(void) ; if (__cond) __compiletime_assert_46(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int ptep_test_and_clear_young(struct vm_area_struct *vma,
unsigned long address,
pte_t *ptep)
{
pte_t pte = *ptep;
int r = 1;
if (!(((u32)((((pteval_t)(1)) << 1)) == ((((pteval_t)(1)) << 1)) ? ((pte)) & ((((pteval_t)(1)) << 1)) : !!(((pte)) & ((((pteval_t)(1)) << 1))))))
r = 0;
else
set_pte_at(vma->vm_mm, address, ptep, pte_mkold(pte));
return r;
}
# 91 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmdp_test_and_clear_young(struct vm_area_struct *vma,
unsigned long address,
pmd_t *pmdp)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_47(void) ; if (__cond) __compiletime_assert_47(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return 0;
}
int ptep_clear_flush_young(struct vm_area_struct *vma,
unsigned long address, pte_t *ptep);
# 115 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmdp_clear_flush_young(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_48(void) ; if (__cond) __compiletime_assert_48(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t ptep_get_and_clear(struct mm_struct *mm,
unsigned long address,
pte_t *ptep)
{
pte_t pte = *ptep;
processor.set_pte_ext(ptep,(0),0);
return pte;
}
# 180 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t ptep_get_and_clear_full(struct mm_struct *mm,
unsigned long address, pte_t *ptep,
int full)
{
pte_t pte;
pte = ptep_get_and_clear(mm, address, ptep);
return pte;
}
# 196 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pte_clear_not_present_full(struct mm_struct *mm,
unsigned long address,
pte_t *ptep,
int full)
{
processor.set_pte_ext(ptep,(0),0);
}
extern pte_t ptep_clear_flush(struct vm_area_struct *vma,
unsigned long address,
pte_t *ptep);
extern pmd_t pmdp_huge_clear_flush(struct vm_area_struct *vma,
unsigned long address,
pmd_t *pmdp);
extern pud_t pudp_huge_clear_flush(struct vm_area_struct *vma,
unsigned long address,
pud_t *pudp);
struct mm_struct;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep)
{
pte_t old_pte = *ptep;
set_pte_at(mm, address, ptep, pte_wrprotect(old_pte));
}
# 262 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pmdp_set_wrprotect(struct mm_struct *mm,
unsigned long address, pmd_t *pmdp)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_49(void) ; if (__cond) __compiletime_assert_49(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
}
# 279 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pudp_set_wrprotect(struct mm_struct *mm,
unsigned long address, pud_t *pudp)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_50(void) ; if (__cond) __compiletime_assert_50(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
}
# 292 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t pmdp_collapse_flush(struct vm_area_struct *vma,
unsigned long address,
pmd_t *pmdp)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_51(void) ; if (__cond) __compiletime_assert_51(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return *pmdp;
}
extern void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
pgtable_t pgtable);
extern pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
# 328 "./include/asm-generic/pgtable.h"
extern void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pmdp_huge_split_prepare(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pte_same(pte_t pte_a, pte_t pte_b)
{
return (pte_a) == (pte_b);
}
# 354 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pte_unused(pte_t pte)
{
return 0;
}
# 397 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_same(pmd_t pmd_a, pmd_t pmd_b)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_52(void) ; if (__cond) __compiletime_assert_52(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_same(pud_t pud_a, pud_t pud_b)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_53(void) ; if (__cond) __compiletime_assert_53(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return 0;
}
# 445 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)
{
if ((oldprot) == ((((oldprot) & ~((((pteval_t)(0x0f)) << 2))) | ((((pteval_t)(0x00)) << 2)))))
newprot = (((newprot) & ~((((pteval_t)(0x0f)) << 2))) | ((((pteval_t)(0x00)) << 2)));
if ((oldprot) == ((((oldprot) & ~((((pteval_t)(0x0f)) << 2))) | ((((pteval_t)(0x01)) << 2)))))
newprot = (((newprot) & ~((((pteval_t)(0x0f)) << 2))) | ((((pteval_t)(0x01)) << 2)));
if ((oldprot) == ((((oldprot) & ~((((pteval_t)(0x0f)) << 2))) | ((((pteval_t)(0x00)) << 2)))))
newprot = (((newprot) & ~((((pteval_t)(0x0f)) << 2))) | ((((pteval_t)(0x00)) << 2)));
return newprot;
}
# 494 "./include/asm-generic/pgtable.h"
void pgd_clear_bad(pgd_t *);
void p4d_clear_bad(pgd_t *);
void pud_clear_bad(pud_t *);
void pmd_clear_bad(pmd_t *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pgd_none_or_clear_bad(pgd_t *pgd)
{
if (pgd_none(*pgd))
return 1;
if (__builtin_expect(!!(pgd_bad(*pgd)), 0)) {
pgd_clear_bad(pgd);
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int p4d_none_or_clear_bad(pgd_t *p4d)
{
if (0)
return 1;
if (__builtin_expect(!!(0), 0)) {
p4d_clear_bad(p4d);
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_none_or_clear_bad(pud_t *pud)
{
if ((0))
return 1;
if (__builtin_expect(!!((0)), 0)) {
pud_clear_bad(pud);
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_none_or_clear_bad(pmd_t *pmd)
{
if ((!(*pmd)))
return 1;
if (__builtin_expect(!!(((*pmd) & 2)), 0)) {
pmd_clear_bad(pmd);
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t __ptep_modify_prot_start(struct mm_struct *mm,
unsigned long addr,
pte_t *ptep)
{
return ptep_get_and_clear(mm, addr, ptep);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __ptep_modify_prot_commit(struct mm_struct *mm,
unsigned long addr,
pte_t *ptep, pte_t pte)
{
set_pte_at(mm, addr, ptep, pte);
}
# 581 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t ptep_modify_prot_start(struct mm_struct *mm,
unsigned long addr,
pte_t *ptep)
{
return __ptep_modify_prot_start(mm, addr, ptep);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ptep_modify_prot_commit(struct mm_struct *mm,
unsigned long addr,
pte_t *ptep, pte_t pte)
{
__ptep_modify_prot_commit(mm, addr, ptep, pte);
}
# 667 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pte_soft_dirty(pte_t pte)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_soft_dirty(pmd_t pmd)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_mksoft_dirty(pte_t pte)
{
return pte;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t pmd_mksoft_dirty(pmd_t pmd)
{
return pmd;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_clear_soft_dirty(pte_t pte)
{
return pte;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t pmd_clear_soft_dirty(pmd_t pmd)
{
return pmd;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_swp_mksoft_dirty(pte_t pte)
{
return pte;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pte_swp_soft_dirty(pte_t pte)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t pte_swp_clear_soft_dirty(pte_t pte)
{
return pte;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t pmd_swp_mksoft_dirty(pmd_t pmd)
{
return pmd;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_swp_soft_dirty(pmd_t pmd)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t pmd_swp_clear_soft_dirty(pmd_t pmd)
{
return pmd;
}
# 739 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int track_pfn_remap(struct vm_area_struct *vma, pgprot_t *prot,
unsigned long pfn, unsigned long addr,
unsigned long size)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void track_pfn_insert(struct vm_area_struct *vma, pgprot_t *prot,
pfn_t pfn)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int track_pfn_copy(struct vm_area_struct *vma)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void untrack_pfn(struct vm_area_struct *vma,
unsigned long pfn, unsigned long size)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void untrack_pfn_moved(struct vm_area_struct *vma)
{
}
# 803 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_zero_pfn(unsigned long pfn)
{
extern unsigned long zero_pfn;
return pfn == zero_pfn;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long my_zero_pfn(unsigned long addr)
{
extern unsigned long zero_pfn;
return zero_pfn;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_trans_huge(pmd_t pmd)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_write(pmd_t pmd)
{
do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/asm-generic/pgtable.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "826" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_write(pud_t pud)
{
do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/asm-generic/pgtable.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "835" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_trans_huge(pud_t pud)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t pmd_read_atomic(pmd_t *pmdp)
{
return *pmdp;
}
# 885 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_none_or_trans_huge_or_clear_bad(pmd_t *pmd)
{
pmd_t pmdval = pmd_read_atomic(pmd);
# 920 "./include/asm-generic/pgtable.h"
if ((!(pmdval)) || pmd_trans_huge(pmdval) ||
(0 && !((pmdval))))
return 1;
if (__builtin_expect(!!(((pmdval) & 2)), 0)) {
pmd_clear_bad(pmd);
return 1;
}
return 0;
}
# 943 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_trans_unstable(pmd_t *pmd)
{
return 0;
}
# 961 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pte_protnone(pte_t pte)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_protnone(pmd_t pmd)
{
return 0;
}
# 997 "./include/asm-generic/pgtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int p4d_set_huge(pgd_t *p4d, phys_addr_t addr, pgprot_t prot)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int p4d_clear_huge(pgd_t *p4d)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_clear_huge(pud_t *pud)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_clear_huge(pmd_t *pmd)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_free_pmd_page(pud_t *pud, unsigned long addr)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_free_pte_page(pmd_t *pmd, unsigned long addr)
{
return 0;
}
# 1050 "./include/asm-generic/pgtable.h"
struct file;
int phys_mem_access_prot_allowed(struct file *file, unsigned long pfn,
unsigned long size, pgprot_t *vma_prot);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void init_espfix_bsp(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool pfn_modify_allowed(unsigned long pfn, pgprot_t prot)
{
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool arch_has_pfn_modify_check(void)
{
return false;
}
# 355 "./arch/arm/include/asm/pgtable.h" 2
# 9 "./include/linux/memremap.h" 2
struct resource;
struct device;
# 21 "./include/linux/memremap.h"
struct vmem_altmap {
const unsigned long base_pfn;
const unsigned long reserve;
unsigned long free;
unsigned long align;
unsigned long alloc;
};
unsigned long vmem_altmap_offset(struct vmem_altmap *altmap);
void vmem_altmap_free(struct vmem_altmap *altmap, unsigned long nr_pfns);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct vmem_altmap *to_vmem_altmap(unsigned long memmap_start)
{
return ((void *)0);
}
# 69 "./include/linux/memremap.h"
enum memory_type {
MEMORY_DEVICE_HOST = 0,
MEMORY_DEVICE_PRIVATE,
MEMORY_DEVICE_PUBLIC,
};
# 107 "./include/linux/memremap.h"
typedef int (*dev_page_fault_t)(struct vm_area_struct *vma,
unsigned long addr,
const struct page *page,
unsigned int flags,
pmd_t *pmdp);
typedef void (*dev_page_free_t)(struct page *page, void *data);
# 125 "./include/linux/memremap.h"
struct dev_pagemap {
dev_page_fault_t page_fault;
dev_page_free_t page_free;
struct vmem_altmap *altmap;
const struct resource *res;
struct percpu_ref *ref;
struct device *dev;
void *data;
enum memory_type type;
};
# 143 "./include/linux/memremap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *devm_memremap_pages(struct device *dev,
struct resource *res, struct percpu_ref *ref,
struct vmem_altmap *altmap)
{
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(1); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/memremap.h", 152); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
return ERR_PTR(-6);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dev_pagemap *find_dev_pagemap(resource_size_t phys)
{
return ((void *)0);
}
# 184 "./include/linux/memremap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct dev_pagemap *get_dev_pagemap(unsigned long pfn,
struct dev_pagemap *pgmap)
{
const struct resource *res = pgmap ? pgmap->res : ((void *)0);
resource_size_t phys = ((phys_addr_t)(pfn) << 12);
if (res && phys >= res->start && phys <= res->end) {
percpu_ref_get(pgmap->ref);
return pgmap;
}
rcu_read_lock();
pgmap = find_dev_pagemap(phys);
if (pgmap && !percpu_ref_tryget_live(pgmap->ref))
pgmap = ((void *)0);
rcu_read_unlock();
return pgmap;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void put_dev_pagemap(struct dev_pagemap *pgmap)
{
if (pgmap)
percpu_ref_put(pgmap->ref);
}
# 28 "./include/linux/mm.h" 2
struct mempolicy;
struct anon_vma;
struct anon_vma_chain;
struct file_ra_state;
struct user_struct;
struct writeback_control;
struct bdi_writeback;
void init_mm_internals(void);
extern unsigned long max_mapnr;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_max_mapnr(unsigned long limit)
{
max_mapnr = limit;
}
extern unsigned long DRAM_size;
extern unsigned long totalram_pages;
extern void * high_memory;
extern int page_cluster;
extern int sysctl_legacy_va_layout;
extern const int mmap_rnd_bits_min;
extern const int mmap_rnd_bits_max;
extern int mmap_rnd_bits __attribute__((__section__(".data..read_mostly")));
# 131 "./include/linux/mm.h"
extern int sysctl_max_map_count;
extern unsigned long sysctl_user_reserve_kbytes;
extern unsigned long sysctl_admin_reserve_kbytes;
extern int sysctl_overcommit_memory;
extern int sysctl_overcommit_ratio;
extern unsigned long sysctl_overcommit_kbytes;
extern int overcommit_ratio_handler(struct ctl_table *, int, void *,
size_t *, loff_t *);
extern int overcommit_kbytes_handler(struct ctl_table *, int, void *,
size_t *, loff_t *);
# 162 "./include/linux/mm.h"
extern struct kmem_cache *vm_area_cachep;
# 299 "./include/linux/mm.h"
extern pgprot_t protection_map[16];
# 332 "./include/linux/mm.h"
struct vm_fault {
struct vm_area_struct *vma;
unsigned int flags;
gfp_t gfp_mask;
unsigned long pgoff;
unsigned long address;
pmd_t *pmd;
pud_t *pud;
pte_t orig_pte;
struct page *cow_page;
struct mem_cgroup *memcg;
struct page *page;
pte_t *pte;
spinlock_t *ptl;
pgtable_t prealloc_pte;
};
enum page_entry_size {
PE_SIZE_PTE = 0,
PE_SIZE_PMD,
PE_SIZE_PUD,
};
struct vm_operations_struct {
void (*open)(struct vm_area_struct * area);
void (*close)(struct vm_area_struct * area);
int (*split)(struct vm_area_struct * area, unsigned long addr);
int (*mremap)(struct vm_area_struct * area);
int (*fault)(struct vm_fault *vmf);
int (*huge_fault)(struct vm_fault *vmf, enum page_entry_size pe_size);
void (*map_pages)(struct vm_fault *vmf,
unsigned long start_pgoff, unsigned long end_pgoff);
int (*page_mkwrite)(struct vm_fault *vmf);
int (*pfn_mkwrite)(struct vm_fault *vmf);
int (*access)(struct vm_area_struct *vma, unsigned long addr,
void *buf, int len, int write);
const char *(*name)(struct vm_area_struct *vma);
# 438 "./include/linux/mm.h"
struct page *(*find_special_page)(struct vm_area_struct *vma,
unsigned long addr);
};
struct mmu_gather;
struct inode;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pmd_devmap(pmd_t pmd)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pud_devmap(pud_t pud)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pgd_devmap(pgd_t pgd)
{
return 0;
}
# 1 "./include/linux/huge_mm.h" 1
# 1 "./include/linux/sched/coredump.h" 1
# 17 "./include/linux/sched/coredump.h"
extern void set_dumpable(struct mm_struct *mm, int value);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __get_dumpable(unsigned long mm_flags)
{
return mm_flags & ((1 << 2) - 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_dumpable(struct mm_struct *mm)
{
return __get_dumpable(mm->flags);
}
# 6 "./include/linux/huge_mm.h" 2
extern int do_huge_pmd_anonymous_page(struct vm_fault *vmf);
extern int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr,
struct vm_area_struct *vma);
extern void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd);
extern int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm,
pud_t *dst_pud, pud_t *src_pud, unsigned long addr,
struct vm_area_struct *vma);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud)
{
}
extern int do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd);
extern struct page *follow_trans_huge_pmd(struct vm_area_struct *vma,
unsigned long addr,
pmd_t *pmd,
unsigned int flags);
extern bool madvise_free_huge_pmd(struct mmu_gather *tlb,
struct vm_area_struct *vma,
pmd_t *pmd, unsigned long addr, unsigned long next);
extern int zap_huge_pmd(struct mmu_gather *tlb,
struct vm_area_struct *vma,
pmd_t *pmd, unsigned long addr);
extern int zap_huge_pud(struct mmu_gather *tlb,
struct vm_area_struct *vma,
pud_t *pud, unsigned long addr);
extern int mincore_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, unsigned long end,
unsigned char *vec);
extern bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
unsigned long new_addr, unsigned long old_end,
pmd_t *old_pmd, pmd_t *new_pmd);
extern int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, pgprot_t newprot,
int prot_numa);
int vmf_insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr,
pmd_t *pmd, pfn_t pfn, bool write);
int vmf_insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr,
pud_t *pud, pfn_t pfn, bool write);
enum transparent_hugepage_flag {
TRANSPARENT_HUGEPAGE_FLAG,
TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG,
TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG,
TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_FLAG,
TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_OR_MADV_FLAG,
TRANSPARENT_HUGEPAGE_DEFRAG_REQ_MADV_FLAG,
TRANSPARENT_HUGEPAGE_DEFRAG_KHUGEPAGED_FLAG,
TRANSPARENT_HUGEPAGE_USE_ZERO_PAGE_FLAG,
};
struct kobject;
struct kobj_attribute;
extern ssize_t single_hugepage_flag_store(struct kobject *kobj,
struct kobj_attribute *attr,
const char *buf, size_t count,
enum transparent_hugepage_flag flag);
extern ssize_t single_hugepage_flag_show(struct kobject *kobj,
struct kobj_attribute *attr, char *buf,
enum transparent_hugepage_flag flag);
extern struct kobj_attribute shmem_enabled_attr;
# 259 "./include/linux/huge_mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool transparent_hugepage_enabled(struct vm_area_struct *vma)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void prep_transhuge_page(struct page *page) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool
can_split_huge_page(struct page *page, int *pextra_pins)
{
do { bool __cond = !(!(1)); extern void __compiletime_assert_54(void) ; if (__cond) __compiletime_assert_54(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0);
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int
split_huge_page_to_list(struct page *page, struct list_head *list)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int split_huge_page(struct page *page)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void deferred_split_huge_page(struct page *page) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long address, bool freeze, struct page *page) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void split_huge_pmd_address(struct vm_area_struct *vma,
unsigned long address, bool freeze, struct page *page) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int hugepage_madvise(struct vm_area_struct *vma,
unsigned long *vm_flags, int advice)
{
do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/huge_mm.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "300" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vma_adjust_trans_huge(struct vm_area_struct *vma,
unsigned long start,
unsigned long end,
long adjust_next)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int is_swap_pmd(pmd_t pmd)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *pmd_trans_huge_lock(pmd_t *pmd,
struct vm_area_struct *vma)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *pud_trans_huge_lock(pud_t *pud,
struct vm_area_struct *vma)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t orig_pmd)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_huge_zero_page(struct page *page)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_huge_zero_pud(pud_t pud)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mm_put_huge_zero_page(struct mm_struct *mm)
{
return;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *follow_devmap_pmd(struct vm_area_struct *vma,
unsigned long addr, pmd_t *pmd, int flags)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *follow_devmap_pud(struct vm_area_struct *vma,
unsigned long addr, pud_t *pud, int flags)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool thp_migration_supported(void)
{
return false;
}
# 469 "./include/linux/mm.h" 2
# 486 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int put_page_testzero(struct page *page)
{
((void)(sizeof(( long)(page_ref_count(page) == 0))));
return page_ref_dec_and_test(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int get_page_unless_zero(struct page *page)
{
return page_ref_add_unless(page, 1, 0);
}
extern int page_is_ram(unsigned long pfn);
enum {
REGION_INTERSECTS,
REGION_DISJOINT,
REGION_MIXED,
};
int region_intersects(resource_size_t offset, size_t size, unsigned long flags,
unsigned long desc);
struct page *vmalloc_to_page(const void *addr);
unsigned long vmalloc_to_pfn(const void *addr);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_vmalloc_addr(const void *x)
{
unsigned long addr = (unsigned long)x;
return addr >= (((unsigned long)high_memory + (8*1024*1024)) & ~((8*1024*1024)-1)) && addr < 0xff800000UL;
}
extern int is_vmalloc_or_module_addr(const void *x);
extern void *kvmalloc_node(size_t size, gfp_t flags, int node);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kvmalloc(size_t size, gfp_t flags)
{
return kvmalloc_node(size, flags, (-1));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kvzalloc_node(size_t size, gfp_t flags, int node)
{
return kvmalloc_node(size, flags | (( gfp_t)0x8000u), node);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kvzalloc(size_t size, gfp_t flags)
{
return kvmalloc(size, flags | (( gfp_t)0x8000u));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *kvmalloc_array(size_t n, size_t size, gfp_t flags)
{
if (size != 0 && n > (~(size_t)0) / size)
return ((void *)0);
return kvmalloc(n * size, flags);
}
extern void kvfree(const void *addr);
extern void kvfree_sensitive(const void *addr, size_t len);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int compound_mapcount(struct page *page)
{
((void)(sizeof(( long)(!PageCompound(page)))));
page = compound_head(page);
return ({ union { typeof((compound_mapcount_ptr(page))->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((compound_mapcount_ptr(page))->counter), __u.__c, sizeof((compound_mapcount_ptr(page))->counter)); else __read_once_size_nocheck(&((compound_mapcount_ptr(page))->counter), __u.__c, sizeof((compound_mapcount_ptr(page))->counter)); do { } while (0); __u.__val; }) + 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_mapcount_reset(struct page *page)
{
({ union { typeof(((&(page)->_mapcount)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&(page)->_mapcount)->counter))) ((-1)) }; __write_once_size(&(((&(page)->_mapcount)->counter)), __u.__c, sizeof(((&(page)->_mapcount)->counter))); __u.__val; });
}
int __page_mapcount(struct page *page);
# 600 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_mapcount(struct page *page)
{
if (__builtin_expect(!!(PageCompound(page)), 0))
return __page_mapcount(page);
return ({ union { typeof((&page->_mapcount)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); else __read_once_size_nocheck(&((&page->_mapcount)->counter), __u.__c, sizeof((&page->_mapcount)->counter)); do { } while (0); __u.__val; }) + 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int total_mapcount(struct page *page)
{
return page_mapcount(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_trans_huge_mapcount(struct page *page,
int *total_mapcount)
{
int mapcount = page_mapcount(page);
if (total_mapcount)
*total_mapcount = mapcount;
return mapcount;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *virt_to_head_page(const void *x)
{
struct page *page = (mem_map + ((((((unsigned long)(x) - (0xC0000000UL)) >> 12) + ((unsigned long)((0x00000000UL) >> 12)))) - ((unsigned long)((0x00000000UL) >> 12))));
return compound_head(page);
}
void __put_page(struct page *page);
void put_pages_list(struct list_head *pages);
void split_page(struct page *page, unsigned int order);
typedef void compound_page_dtor(struct page *);
enum compound_dtor_id {
NULL_COMPOUND_DTOR,
COMPOUND_PAGE_DTOR,
NR_COMPOUND_DTORS,
};
extern compound_page_dtor * const compound_page_dtors[];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_compound_page_dtor(struct page *page,
enum compound_dtor_id compound_dtor)
{
((void)(sizeof(( long)(compound_dtor >= NR_COMPOUND_DTORS))));
page[1].compound_dtor = compound_dtor;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) compound_page_dtor *get_compound_page_dtor(struct page *page)
{
((void)(sizeof(( long)(page[1].compound_dtor >= NR_COMPOUND_DTORS))));
return compound_page_dtors[page[1].compound_dtor];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int compound_order(struct page *page)
{
if (!PageHead(page))
return 0;
return page[1].compound_order;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_compound_order(struct page *page, unsigned int order)
{
page[1].compound_order = order;
}
void free_compound_page(struct page *page);
# 693 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t maybe_mkwrite(pte_t pte, struct vm_area_struct *vma)
{
if (__builtin_expect(!!(vma->vm_flags & 0x00000002), 1))
pte = pte_mkwrite(pte);
return pte;
}
int alloc_set_pte(struct vm_fault *vmf, struct mem_cgroup *memcg,
struct page *page);
int finish_fault(struct vm_fault *vmf);
int finish_mkwrite_fault(struct vm_fault *vmf);
# 813 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) enum zone_type page_zonenum(const struct page *page)
{
return (page->flags >> (((((sizeof(unsigned long)*8) - 0) - 0) - 2) * (2 != 0))) & ((1UL << 2) - 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_zone_device_page(const struct page *page)
{
return false;
}
# 837 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void put_zone_device_private_or_public_page(struct page *page)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_device_private_page(const struct page *page)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_device_public_page(const struct page *page)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void get_page(struct page *page)
{
page = compound_head(page);
((void)(sizeof(( long)(((unsigned int) page_ref_count(page) + 127u <= 127u)))));
page_ref_inc(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((warn_unused_result)) bool try_get_page(struct page *page)
{
page = compound_head(page);
if (({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(page_ref_count(page) <= 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/mm.h", 870); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); }))
return false;
page_ref_inc(page);
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void put_page(struct page *page)
{
page = compound_head(page);
if (0 && __builtin_expect(!!(is_device_private_page(page) || __builtin_expect(!!(is_device_public_page(page)), 0)), 0)) {
put_zone_device_private_or_public_page(page);
return;
}
if (put_page_testzero(page))
__put_page(page);
}
# 908 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_zone_id(struct page *page)
{
return (page->flags >> ((((((sizeof(unsigned long)*8) - 0) - 0) < ((((sizeof(unsigned long)*8) - 0) - 0) - 2))? (((sizeof(unsigned long)*8) - 0) - 0) : ((((sizeof(unsigned long)*8) - 0) - 0) - 2)) * ((0 + 2) != 0))) & ((1UL << (0 + 2)) - 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int zone_to_nid(struct zone *zone)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_to_nid(const struct page *page)
{
return (page->flags >> ((((sizeof(unsigned long)*8) - 0) - 0) * (0 != 0))) & ((1UL << 0) - 1);
}
# 996 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_cpupid_xchg_last(struct page *page, int cpupid)
{
return page_to_nid(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_cpupid_last(struct page *page)
{
return page_to_nid(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpupid_to_nid(int cpupid)
{
return -1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpupid_to_pid(int cpupid)
{
return -1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpupid_to_cpu(int cpupid)
{
return -1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int cpu_pid_to_cpupid(int nid, int pid)
{
return -1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cpupid_pid_unset(int cpupid)
{
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_cpupid_reset_last(struct page *page)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool cpupid_match_pid(struct task_struct *task, int cpupid)
{
return false;
}
# 1058 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u8 page_kasan_tag(const struct page *page)
{
return 0xff;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_kasan_tag_set(struct page *page, u8 tag) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void page_kasan_tag_reset(struct page *page) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct zone *page_zone(const struct page *page)
{
return &(&contig_page_data)->node_zones[page_zonenum(page)];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pg_data_t *page_pgdat(const struct page *page)
{
return (&contig_page_data);
}
# 1090 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_page_zone(struct page *page, enum zone_type zone)
{
page->flags &= ~(((1UL << 2) - 1) << (((((sizeof(unsigned long)*8) - 0) - 0) - 2) * (2 != 0)));
page->flags |= (zone & ((1UL << 2) - 1)) << (((((sizeof(unsigned long)*8) - 0) - 0) - 2) * (2 != 0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_page_node(struct page *page, unsigned long node)
{
page->flags &= ~(((1UL << 0) - 1) << ((((sizeof(unsigned long)*8) - 0) - 0) * (0 != 0)));
page->flags |= (node & ((1UL << 0) - 1)) << ((((sizeof(unsigned long)*8) - 0) - 0) * (0 != 0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_page_links(struct page *page, enum zone_type zone,
unsigned long node, unsigned long pfn)
{
set_page_zone(page, zone);
set_page_node(page, node);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct mem_cgroup *page_memcg(struct page *page)
{
return page->mem_cgroup;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct mem_cgroup *page_memcg_rcu(struct page *page)
{
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(!rcu_read_lock_held()); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/mm.h", 1119); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
return ({ union { typeof(page->mem_cgroup) __val; char __c[1]; } __u; if (1) __read_once_size(&(page->mem_cgroup), __u.__c, sizeof(page->mem_cgroup)); else __read_once_size_nocheck(&(page->mem_cgroup), __u.__c, sizeof(page->mem_cgroup)); do { } while (0); __u.__val; });
}
# 1137 "./include/linux/mm.h"
# 1 "./include/linux/vmstat.h" 1
# 1 "./include/linux/vm_event_item.h" 1
# 39 "./include/linux/vm_event_item.h"
enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
PGALLOC_NORMAL, PGALLOC_HIGH, PGALLOC_MOVABLE,
ALLOCSTALL_NORMAL, ALLOCSTALL_HIGH, ALLOCSTALL_MOVABLE,
PGSCAN_SKIP_NORMAL, PGSCAN_SKIP_HIGH, PGSCAN_SKIP_MOVABLE,
PGFREE, PGACTIVATE, PGDEACTIVATE, PGLAZYFREE,
PGFAULT, PGMAJFAULT,
PGLAZYFREED,
PGREFILL,
PGSTEAL_KSWAPD,
PGSTEAL_DIRECT,
PGSCAN_KSWAPD,
PGSCAN_DIRECT,
PGSCAN_DIRECT_THROTTLE,
PGINODESTEAL, SLABS_SCANNED, KSWAPD_INODESTEAL,
KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
PAGEOUTRUN, PGROTATED,
DROP_PAGECACHE, DROP_SLAB,
OOM_KILL,
# 68 "./include/linux/vm_event_item.h"
PGMIGRATE_SUCCESS, PGMIGRATE_FAIL,
COMPACTMIGRATE_SCANNED, COMPACTFREE_SCANNED,
COMPACTISOLATED,
COMPACTSTALL, COMPACTFAIL, COMPACTSUCCESS,
KCOMPACTD_WAKE,
KCOMPACTD_MIGRATE_SCANNED, KCOMPACTD_FREE_SCANNED,
UNEVICTABLE_PGCULLED,
UNEVICTABLE_PGSCANNED,
UNEVICTABLE_PGRESCUED,
UNEVICTABLE_PGMLOCKED,
UNEVICTABLE_PGMUNLOCKED,
UNEVICTABLE_PGCLEARED,
UNEVICTABLE_PGSTRANDED,
# 124 "./include/linux/vm_event_item.h"
SWAP_RA,
SWAP_RA_HIT,
NR_VM_EVENT_ITEMS
};
# 9 "./include/linux/vmstat.h" 2
extern int sysctl_stat_interval;
# 24 "./include/linux/vmstat.h"
struct vm_event_state {
unsigned long event[NR_VM_EVENT_ITEMS];
};
extern __attribute__((section(".data..percpu" ""))) __typeof__(struct vm_event_state) vm_event_states;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __count_vm_event(enum vm_event_item item)
{
do { do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(vm_event_states.event[item])) { case 1: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 2: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 4: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; case 8: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void count_vm_event(enum vm_event_item item)
{
do { do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(vm_event_states.event[item])) { case 1: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 2: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 4: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 8: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += 1; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __count_vm_events(enum vm_event_item item, long delta)
{
do { do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(vm_event_states.event[item])) { case 1: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0);break; case 2: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0);break; case 4: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0);break; case 8: do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void count_vm_events(enum vm_event_item item, long delta)
{
do { do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(vm_event_states.event[item])) { case 1: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 2: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 4: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; case 8: do { unsigned long __flags; do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); __flags = arch_local_irq_save(); } while (0); do { *({ do { const void *__vpp_verify = (typeof((&(vm_event_states.event[item])) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))); (typeof((typeof(*(&(vm_event_states.event[item]))) *)(&(vm_event_states.event[item])))) (__ptr + ((__my_cpu_offset()))); }); }) += delta; } while (0); do { ({ unsigned long __dummy; typeof(__flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(__flags); } while (0); } while (0);break; default: __bad_size_call_parameter();break; } } while (0);
}
extern void all_vm_events(unsigned long *);
extern void vm_events_fold_cpu(int cpu);
# 110 "./include/linux/vmstat.h"
extern atomic_long_t vm_zone_stat[NR_VM_ZONE_STAT_ITEMS];
extern atomic_long_t vm_numa_stat[0];
extern atomic_long_t vm_node_stat[NR_VM_NODE_STAT_ITEMS];
# 142 "./include/linux/vmstat.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void zone_page_state_add(long x, struct zone *zone,
enum zone_stat_item item)
{
atomic_long_add(x, &zone->vm_stat[item]);
atomic_long_add(x, &vm_zone_stat[item]);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void node_page_state_add(long x, struct pglist_data *pgdat,
enum node_stat_item item)
{
atomic_long_add(x, &pgdat->vm_stat[item]);
atomic_long_add(x, &vm_node_stat[item]);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long global_zone_page_state(enum zone_stat_item item)
{
long x = atomic_long_read(&vm_zone_stat[item]);
if (x < 0)
x = 0;
return x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long global_node_page_state(enum node_stat_item item)
{
long x = atomic_long_read(&vm_node_stat[item]);
if (x < 0)
x = 0;
return x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long zone_page_state(struct zone *zone,
enum zone_stat_item item)
{
long x = atomic_long_read(&zone->vm_stat[item]);
if (x < 0)
x = 0;
return x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long zone_page_state_snapshot(struct zone *zone,
enum zone_stat_item item)
{
long x = atomic_long_read(&zone->vm_stat[item]);
int cpu;
for (((cpu)) = -1; ((cpu)) = cpumask_next(((cpu)), (((const struct cpumask *)&__cpu_online_mask))), ((cpu)) < nr_cpu_ids;)
x += ({ do { const void *__vpp_verify = (typeof((zone->pageset) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*((zone->pageset))) *)((zone->pageset)))); (typeof((typeof(*((zone->pageset))) *)((zone->pageset)))) (__ptr + (((__per_cpu_offset[(cpu)])))); }); })->vm_stat_diff[item];
if (x < 0)
x = 0;
return x;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long node_page_state_snapshot(pg_data_t *pgdat,
enum node_stat_item item)
{
long x = atomic_long_read(&pgdat->vm_stat[item]);
int cpu;
for (((cpu)) = -1; ((cpu)) = cpumask_next(((cpu)), (((const struct cpumask *)&__cpu_online_mask))), ((cpu)) < nr_cpu_ids;)
x += ({ do { const void *__vpp_verify = (typeof((pgdat->per_cpu_nodestats) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*((pgdat->per_cpu_nodestats))) *)((pgdat->per_cpu_nodestats)))); (typeof((typeof(*((pgdat->per_cpu_nodestats))) *)((pgdat->per_cpu_nodestats)))) (__ptr + (((__per_cpu_offset[(cpu)])))); }); })->vm_node_stat_diff[item];
if (x < 0)
x = 0;
return x;
}
# 244 "./include/linux/vmstat.h"
void __mod_zone_page_state(struct zone *, enum zone_stat_item item, long);
void __inc_zone_page_state(struct page *, enum zone_stat_item);
void __dec_zone_page_state(struct page *, enum zone_stat_item);
void __mod_node_page_state(struct pglist_data *, enum node_stat_item item, long);
void __inc_node_page_state(struct page *, enum node_stat_item);
void __dec_node_page_state(struct page *, enum node_stat_item);
void mod_zone_page_state(struct zone *, enum zone_stat_item, long);
void inc_zone_page_state(struct page *, enum zone_stat_item);
void dec_zone_page_state(struct page *, enum zone_stat_item);
void mod_node_page_state(struct pglist_data *, enum node_stat_item, long);
void inc_node_page_state(struct page *, enum node_stat_item);
void dec_node_page_state(struct page *, enum node_stat_item);
extern void inc_node_state(struct pglist_data *, enum node_stat_item);
extern void __inc_zone_state(struct zone *, enum zone_stat_item);
extern void __inc_node_state(struct pglist_data *, enum node_stat_item);
extern void dec_zone_state(struct zone *, enum zone_stat_item);
extern void __dec_zone_state(struct zone *, enum zone_stat_item);
extern void __dec_node_state(struct pglist_data *, enum node_stat_item);
void quiet_vmstat(void);
void cpu_vm_stats_fold(int cpu);
void refresh_zone_stat_thresholds(void);
struct ctl_table;
int vmstat_refresh(struct ctl_table *, int write,
void *buffer, size_t *lenp, loff_t *ppos);
void drain_zonestat(struct zone *zone, struct per_cpu_pageset *);
int calculate_pressure_threshold(struct zone *zone);
int calculate_normal_threshold(struct zone *zone);
void set_pgdat_percpu_threshold(pg_data_t *pgdat,
int (*calculate_pressure)(struct zone *));
# 375 "./include/linux/vmstat.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __mod_zone_freepage_state(struct zone *zone, int nr_pages,
int migratetype)
{
__mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
if (__builtin_expect(!!((migratetype) == MIGRATE_CMA), 0))
__mod_zone_page_state(zone, NR_FREE_CMA_PAGES, nr_pages);
}
extern const char * const vmstat_text[];
# 1138 "./include/linux/mm.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void *lowmem_page_address(const struct page *page)
{
return ((void *)__phys_to_virt((phys_addr_t)(((phys_addr_t)(((unsigned long)((page) - mem_map) + ((unsigned long)((0x00000000UL) >> 12)))) << 12))));
}
# 1161 "./include/linux/mm.h"
void *page_address(const struct page *page);
void set_page_address(struct page *page, void *virtual);
void page_address_init(void);
# 1172 "./include/linux/mm.h"
extern void *page_rmapping(struct page *page);
extern struct anon_vma *page_anon_vma(struct page *page);
extern struct address_space *page_mapping(struct page *page);
extern struct address_space *__page_file_mapping(struct page *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
struct address_space *page_file_mapping(struct page *page)
{
if (__builtin_expect(!!(PageSwapCache(page)), 0))
return __page_file_mapping(page);
return page->mapping;
}
extern unsigned long __page_file_index(struct page *page);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long page_index(struct page *page)
{
if (__builtin_expect(!!(PageSwapCache(page)), 0))
return __page_file_index(page);
return page->index;
}
bool page_mapped(struct page *page);
struct address_space *page_mapping(struct page *page);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool page_is_pfmemalloc(struct page *page)
{
return page->index == -1UL;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_page_pfmemalloc(struct page *page)
{
page->index = -1UL;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_page_pfmemalloc(struct page *page)
{
page->index = 0;
}
# 1278 "./include/linux/mm.h"
extern void pagefault_out_of_memory(void);
# 1288 "./include/linux/mm.h"
extern void show_free_areas(unsigned int flags, nodemask_t *nodemask);
void shmem_set_file(struct vm_area_struct *vma, struct file *file);
extern bool can_do_mlock(void);
extern int user_shm_lock(size_t, struct user_struct *);
extern void user_shm_unlock(size_t, struct user_struct *);
struct zap_details {
struct address_space *check_mapping;
unsigned long first_index;
unsigned long last_index;
};
struct page *_vm_normal_page(struct vm_area_struct *vma, unsigned long addr,
pte_t pte, bool with_public_device);
struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr,
pmd_t pmd);
int zap_vma_ptes(struct vm_area_struct *vma, unsigned long address,
unsigned long size);
void zap_page_range(struct vm_area_struct *vma, unsigned long address,
unsigned long size);
void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma,
unsigned long start, unsigned long end);
# 1343 "./include/linux/mm.h"
struct mm_walk {
int (*pud_entry)(pud_t *pud, unsigned long addr,
unsigned long next, struct mm_walk *walk);
int (*pmd_entry)(pmd_t *pmd, unsigned long addr,
unsigned long next, struct mm_walk *walk);
int (*pte_entry)(pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk);
int (*pte_hole)(unsigned long addr, unsigned long next,
struct mm_walk *walk);
int (*hugetlb_entry)(pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long next,
struct mm_walk *walk);
int (*test_walk)(unsigned long addr, unsigned long next,
struct mm_walk *walk);
struct mm_struct *mm;
struct vm_area_struct *vma;
void *private;
};
int walk_page_range(unsigned long addr, unsigned long end,
struct mm_walk *walk);
int walk_page_vma(struct vm_area_struct *vma, struct mm_walk *walk);
void free_pgd_range(struct mmu_gather *tlb, unsigned long addr,
unsigned long end, unsigned long floor, unsigned long ceiling);
int copy_page_range(struct mm_struct *dst, struct mm_struct *src,
struct vm_area_struct *vma);
void unmap_mapping_range(struct address_space *mapping,
loff_t const holebegin, loff_t const holelen, int even_cows);
int follow_pte_pmd(struct mm_struct *mm, unsigned long address,
unsigned long *start, unsigned long *end,
pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp);
int follow_pfn(struct vm_area_struct *vma, unsigned long address,
unsigned long *pfn);
int follow_phys(struct vm_area_struct *vma, unsigned long address,
unsigned int flags, unsigned long *prot, resource_size_t *phys);
int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
void *buf, int len, int write);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void unmap_shared_mapping_range(struct address_space *mapping,
loff_t const holebegin, loff_t const holelen)
{
unmap_mapping_range(mapping, holebegin, holelen, 0);
}
extern void truncate_pagecache(struct inode *inode, loff_t new);
extern void truncate_setsize(struct inode *inode, loff_t newsize);
void pagecache_isize_extended(struct inode *inode, loff_t from, loff_t to);
void truncate_pagecache_range(struct inode *inode, loff_t offset, loff_t end);
int truncate_inode_page(struct address_space *mapping, struct page *page);
int generic_error_remove_page(struct address_space *mapping, struct page *page);
int invalidate_inode_page(struct page *page);
extern int handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
unsigned int flags);
extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
unsigned long address, unsigned int fault_flags,
bool *unlocked);
# 1419 "./include/linux/mm.h"
extern int access_process_vm(struct task_struct *tsk, unsigned long addr, void *buf, int len,
unsigned int gup_flags);
extern int access_remote_vm(struct mm_struct *mm, unsigned long addr,
void *buf, int len, unsigned int gup_flags);
extern int __access_remote_vm(struct task_struct *tsk, struct mm_struct *mm,
unsigned long addr, void *buf, int len, unsigned int gup_flags);
long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
struct vm_area_struct **vmas, int *locked);
long get_user_pages(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
struct vm_area_struct **vmas);
long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages, int *locked);
long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
struct page **pages, unsigned int gup_flags);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) long get_user_pages_longterm(unsigned long start,
unsigned long nr_pages, unsigned int gup_flags,
struct page **pages, struct vm_area_struct **vmas)
{
return get_user_pages(start, nr_pages, gup_flags, pages, vmas);
}
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages);
struct frame_vector {
unsigned int nr_allocated;
unsigned int nr_frames;
bool got_ref;
bool is_pfns;
void *ptrs[0];
};
struct frame_vector *frame_vector_create(unsigned int nr_frames);
void frame_vector_destroy(struct frame_vector *vec);
int get_vaddr_frames(unsigned long start, unsigned int nr_pfns,
unsigned int gup_flags, struct frame_vector *vec);
void put_vaddr_frames(struct frame_vector *vec);
int frame_vector_to_pages(struct frame_vector *vec);
void frame_vector_to_pfns(struct frame_vector *vec);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int frame_vector_count(struct frame_vector *vec)
{
return vec->nr_frames;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page **frame_vector_pages(struct frame_vector *vec)
{
if (vec->is_pfns) {
int err = frame_vector_to_pages(vec);
if (err)
return ERR_PTR(err);
}
return (struct page **)(vec->ptrs);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long *frame_vector_pfns(struct frame_vector *vec)
{
if (!vec->is_pfns)
frame_vector_to_pfns(vec);
return (unsigned long *)(vec->ptrs);
}
struct kvec;
int get_kernel_pages(const struct kvec *iov, int nr_pages, int write,
struct page **pages);
int get_kernel_page(unsigned long start, int write, struct page **pages);
struct page *get_dump_page(unsigned long addr);
extern int try_to_release_page(struct page * page, gfp_t gfp_mask);
extern void do_invalidatepage(struct page *page, unsigned int offset,
unsigned int length);
int __set_page_dirty_nobuffers(struct page *page);
int __set_page_dirty_no_writeback(struct page *page);
int redirty_page_for_writepage(struct writeback_control *wbc,
struct page *page);
void account_page_dirtied(struct page *page, struct address_space *mapping);
void account_page_cleaned(struct page *page, struct address_space *mapping,
struct bdi_writeback *wb);
int set_page_dirty(struct page *page);
int set_page_dirty_lock(struct page *page);
void cancel_dirty_page(struct page *page);
int clear_page_dirty_for_io(struct page *page);
int get_cmdline(struct task_struct *task, char *buffer, int buflen);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool vma_is_anonymous(struct vm_area_struct *vma)
{
return !vma->vm_ops;
}
bool vma_is_shmem(struct vm_area_struct *vma);
int vma_is_stack_for_current(struct vm_area_struct *vma);
extern unsigned long move_page_tables(struct vm_area_struct *vma,
unsigned long old_addr, struct vm_area_struct *new_vma,
unsigned long new_addr, unsigned long len,
bool need_rmap_locks);
extern unsigned long change_protection(struct vm_area_struct *vma, unsigned long start,
unsigned long end, pgprot_t newprot,
int dirty_accountable, int prot_numa);
extern int mprotect_fixup(struct vm_area_struct *vma,
struct vm_area_struct **pprev, unsigned long start,
unsigned long end, unsigned long newflags);
int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long get_mm_counter(struct mm_struct *mm, int member)
{
long val = atomic_long_read(&mm->rss_stat.count[member]);
if (val < 0)
val = 0;
return (unsigned long)val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void add_mm_counter(struct mm_struct *mm, int member, long value)
{
atomic_long_add(value, &mm->rss_stat.count[member]);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void inc_mm_counter(struct mm_struct *mm, int member)
{
atomic_long_inc(&mm->rss_stat.count[member]);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dec_mm_counter(struct mm_struct *mm, int member)
{
atomic_long_dec(&mm->rss_stat.count[member]);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mm_counter_file(struct page *page)
{
if (PageSwapBacked(page))
return MM_SHMEMPAGES;
return MM_FILEPAGES;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mm_counter(struct page *page)
{
if (PageAnon(page))
return MM_ANONPAGES;
return mm_counter_file(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long get_mm_rss(struct mm_struct *mm)
{
return get_mm_counter(mm, MM_FILEPAGES) +
get_mm_counter(mm, MM_ANONPAGES) +
get_mm_counter(mm, MM_SHMEMPAGES);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long get_mm_hiwater_rss(struct mm_struct *mm)
{
return ({ typeof(mm->hiwater_rss) __UNIQUE_ID_max1_55 = (mm->hiwater_rss); typeof(get_mm_rss(mm)) __UNIQUE_ID_max2_56 = (get_mm_rss(mm)); (void) (&__UNIQUE_ID_max1_55 == &__UNIQUE_ID_max2_56); __UNIQUE_ID_max1_55 > __UNIQUE_ID_max2_56 ? __UNIQUE_ID_max1_55 : __UNIQUE_ID_max2_56; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long get_mm_hiwater_vm(struct mm_struct *mm)
{
return ({ typeof(mm->hiwater_vm) __UNIQUE_ID_max1_57 = (mm->hiwater_vm); typeof(mm->total_vm) __UNIQUE_ID_max2_58 = (mm->total_vm); (void) (&__UNIQUE_ID_max1_57 == &__UNIQUE_ID_max2_58); __UNIQUE_ID_max1_57 > __UNIQUE_ID_max2_58 ? __UNIQUE_ID_max1_57 : __UNIQUE_ID_max2_58; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void update_hiwater_rss(struct mm_struct *mm)
{
unsigned long _rss = get_mm_rss(mm);
if ((mm)->hiwater_rss < _rss)
(mm)->hiwater_rss = _rss;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void update_hiwater_vm(struct mm_struct *mm)
{
if (mm->hiwater_vm < mm->total_vm)
mm->hiwater_vm = mm->total_vm;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void reset_mm_hiwater_rss(struct mm_struct *mm)
{
mm->hiwater_rss = get_mm_rss(mm);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void setmax_mm_hiwater_rss(unsigned long *maxrss,
struct mm_struct *mm)
{
unsigned long hiwater_rss = get_mm_hiwater_rss(mm);
if (*maxrss < hiwater_rss)
*maxrss = hiwater_rss;
}
void sync_mm_rss(struct mm_struct *mm);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int pte_devmap(pte_t pte)
{
return 0;
}
int vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot);
extern pte_t *__get_locked_pte(struct mm_struct *mm, unsigned long addr,
spinlock_t **ptl);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
spinlock_t **ptl)
{
pte_t *ptep;
(ptep = __get_locked_pte(mm, addr, ptl));
return ptep;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __p4d_alloc(struct mm_struct *mm, pgd_t *pgd,
unsigned long address)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __pud_alloc(struct mm_struct *mm, pgd_t *p4d,
unsigned long address)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __pmd_alloc(struct mm_struct *mm, pud_t *pud,
unsigned long address)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mm_nr_pmds_init(struct mm_struct *mm) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long mm_nr_pmds(struct mm_struct *mm)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mm_inc_nr_pmds(struct mm_struct *mm) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mm_dec_nr_pmds(struct mm_struct *mm) {}
# 1733 "./include/linux/mm.h"
int __pte_alloc(struct mm_struct *mm, pmd_t *pmd, unsigned long address);
int __pte_alloc_kernel(pmd_t *pmd, unsigned long address);
# 1758 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
{
return (__builtin_expect(!!((0)), 0) && __pmd_alloc(mm, pud, address))?
((void *)0): pmd_offset(pud, address);
}
# 1776 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ptlock_cache_init(void)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool ptlock_alloc(struct page *page)
{
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ptlock_free(struct page *page)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *ptlock_ptr(struct page *page)
{
return &page->ptl;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)
{
return ptlock_ptr((mem_map + ((((unsigned long)(((*pmd) & (~0UL)) >> 12))) - ((unsigned long)((0x00000000UL) >> 12)))));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool ptlock_init(struct page *page)
{
((void)(sizeof(( long)(*(unsigned long *)&page->ptl))));
if (!ptlock_alloc(page))
return false;
do { spinlock_check(ptlock_ptr(page)); do { *(&(ptlock_ptr(page))->rlock) = (raw_spinlock_t) { .raw_lock = { { 0 } }, }; } while (0); } while (0);
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pte_lock_deinit(struct page *page)
{
page->mapping = ((void *)0);
ptlock_free(page);
}
# 1836 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgtable_init(void)
{
ptlock_cache_init();
do { } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool pgtable_page_ctor(struct page *page)
{
if (!ptlock_init(page))
return false;
inc_zone_page_state(page, NR_PAGETABLE);
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgtable_page_dtor(struct page *page)
{
pte_lock_deinit(page);
dec_zone_page_state(page, NR_PAGETABLE);
}
# 1917 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *pmd_lockptr(struct mm_struct *mm, pmd_t *pmd)
{
return &mm->page_table_lock;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool pgtable_pmd_page_ctor(struct page *page) { return true; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pgtable_pmd_page_dtor(struct page *page) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd)
{
spinlock_t *ptl = pmd_lockptr(mm, pmd);
spin_lock(ptl);
return ptl;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *pud_lockptr(struct mm_struct *mm, pud_t *pud)
{
return &mm->page_table_lock;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud)
{
spinlock_t *ptl = pud_lockptr(mm, pud);
spin_lock(ptl);
return ptl;
}
extern void __attribute__ ((__section__(".init.text"))) pagecache_init(void);
extern void free_area_init(unsigned long * zones_size);
extern void free_area_init_node(int nid, unsigned long * zones_size,
unsigned long zone_start_pfn, unsigned long *zholes_size);
extern void free_initmem(void);
extern unsigned long free_reserved_area(void *start, void *end,
int poison, char *s);
extern void free_highmem_page(struct page *page);
extern void adjust_managed_page_count(struct page *page, long count);
extern void mem_init_print_info(const char *str);
extern void reserve_bootmem_region(phys_addr_t start, phys_addr_t end);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __free_reserved_page(struct page *page)
{
ClearPageReserved(page);
init_page_count(page);
__free_pages((page), 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void free_reserved_page(struct page *page)
{
__free_reserved_page(page);
adjust_managed_page_count(page, 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mark_page_reserved(struct page *page)
{
SetPageReserved(page);
adjust_managed_page_count(page, -1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long free_initmem_default(int poison)
{
extern char __init_begin[], __init_end[];
return free_reserved_area(&__init_begin, &__init_end,
poison, "unused kernel");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long get_num_physpages(void)
{
int nid;
unsigned long phys_pages = 0;
for ( (nid) = 0; (nid) == 0; (nid) = 1)
phys_pages += ((&contig_page_data)->node_present_pages);
return phys_pages;
}
# 2076 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int __early_pfn_to_nid(unsigned long pfn,
struct mminit_pfnnid_cache *state)
{
return 0;
}
# 2089 "./include/linux/mm.h"
extern void set_dma_reserve(unsigned long new_dma_reserve);
extern void memmap_init_zone(unsigned long, int, unsigned long,
unsigned long, enum memmap_context);
extern void setup_per_zone_wmarks(void);
extern int __attribute__ ((__section__(".meminit.text"))) __attribute__((no_instrument_function)) init_per_zone_wmark_min(void);
extern void mem_init(void);
extern void __attribute__ ((__section__(".init.text"))) mmap_init(void);
extern void show_mem(unsigned int flags, nodemask_t *nodemask);
extern long si_mem_available(void);
extern void si_meminfo(struct sysinfo * val);
extern void si_meminfo_node(struct sysinfo *val, int nid);
extern __attribute__((format(printf, 3, 4)))
void warn_alloc(gfp_t gfp_mask, nodemask_t *nodemask, const char *fmt, ...);
extern void setup_per_cpu_pageset(void);
extern void zone_pcp_update(struct zone *zone);
extern void zone_pcp_reset(struct zone *zone);
extern int min_free_kbytes;
extern int watermark_scale_factor;
extern atomic_long_t mmap_pages_allocated;
extern int nommu_shrink_inode_mappings(struct inode *, size_t, size_t);
void vma_interval_tree_insert(struct vm_area_struct *node,
struct rb_root_cached *root);
void vma_interval_tree_insert_after(struct vm_area_struct *node,
struct vm_area_struct *prev,
struct rb_root_cached *root);
void vma_interval_tree_remove(struct vm_area_struct *node,
struct rb_root_cached *root);
struct vm_area_struct *vma_interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start, unsigned long last);
struct vm_area_struct *vma_interval_tree_iter_next(struct vm_area_struct *node,
unsigned long start, unsigned long last);
void anon_vma_interval_tree_insert(struct anon_vma_chain *node,
struct rb_root_cached *root);
void anon_vma_interval_tree_remove(struct anon_vma_chain *node,
struct rb_root_cached *root);
struct anon_vma_chain *
anon_vma_interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start, unsigned long last);
struct anon_vma_chain *anon_vma_interval_tree_iter_next(
struct anon_vma_chain *node, unsigned long start, unsigned long last);
# 2155 "./include/linux/mm.h"
extern int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin);
extern int __vma_adjust(struct vm_area_struct *vma, unsigned long start,
unsigned long end, unsigned long pgoff, struct vm_area_struct *insert,
struct vm_area_struct *expand);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int vma_adjust(struct vm_area_struct *vma, unsigned long start,
unsigned long end, unsigned long pgoff, struct vm_area_struct *insert)
{
return __vma_adjust(vma, start, end, pgoff, insert, ((void *)0));
}
extern struct vm_area_struct *vma_merge(struct mm_struct *,
struct vm_area_struct *prev, unsigned long addr, unsigned long end,
unsigned long vm_flags, struct anon_vma *, struct file *, unsigned long,
struct mempolicy *, struct vm_userfaultfd_ctx, const char *);
extern struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *);
extern int __split_vma(struct mm_struct *, struct vm_area_struct *,
unsigned long addr, int new_below);
extern int split_vma(struct mm_struct *, struct vm_area_struct *,
unsigned long addr, int new_below);
extern int insert_vm_struct(struct mm_struct *, struct vm_area_struct *);
extern void __vma_link_rb(struct mm_struct *, struct vm_area_struct *,
struct rb_node **, struct rb_node *);
extern void unlink_file_vma(struct vm_area_struct *);
extern struct vm_area_struct *copy_vma(struct vm_area_struct **,
unsigned long addr, unsigned long len, unsigned long pgoff,
bool *need_rmap_locks);
extern void exit_mmap(struct mm_struct *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int check_data_rlimit(unsigned long rlim,
unsigned long new,
unsigned long start,
unsigned long end_data,
unsigned long start_data)
{
if (rlim < (~0UL)) {
if (((new - start) + (end_data - start_data)) > rlim)
return -28;
}
return 0;
}
extern int mm_take_all_locks(struct mm_struct *mm);
extern void mm_drop_all_locks(struct mm_struct *mm);
extern void set_mm_exe_file(struct mm_struct *mm, struct file *new_exe_file);
extern struct file *get_mm_exe_file(struct mm_struct *mm);
extern struct file *get_task_exe_file(struct task_struct *task);
extern bool may_expand_vm(struct mm_struct *, vm_flags_t, unsigned long npages);
extern void vm_stat_account(struct mm_struct *, vm_flags_t, long npages);
extern bool vma_is_special_mapping(const struct vm_area_struct *vma,
const struct vm_special_mapping *sm);
extern struct vm_area_struct *_install_special_mapping(struct mm_struct *mm,
unsigned long addr, unsigned long len,
unsigned long flags,
const struct vm_special_mapping *spec);
extern int install_special_mapping(struct mm_struct *mm,
unsigned long addr, unsigned long len,
unsigned long flags, struct page **pages);
extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
struct list_head *uf);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
struct list_head *uf);
extern int do_munmap(struct mm_struct *, unsigned long, size_t,
struct list_head *uf);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
do_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
unsigned long pgoff, unsigned long *populate,
struct list_head *uf)
{
return do_mmap(file, addr, len, prot, flags, 0, pgoff, populate, uf);
}
extern int __mm_populate(unsigned long addr, unsigned long len,
int ignore_errors);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mm_populate(unsigned long addr, unsigned long len)
{
(void) __mm_populate(addr, len, 1);
}
extern int __attribute__((warn_unused_result)) vm_brk(unsigned long, unsigned long);
extern int __attribute__((warn_unused_result)) vm_brk_flags(unsigned long, unsigned long, unsigned long);
extern int vm_munmap(unsigned long, size_t);
extern unsigned long __attribute__((warn_unused_result)) vm_mmap(struct file *, unsigned long,
unsigned long, unsigned long,
unsigned long, unsigned long);
struct vm_unmapped_area_info {
unsigned long flags;
unsigned long length;
unsigned long low_limit;
unsigned long high_limit;
unsigned long align_mask;
unsigned long align_offset;
};
extern unsigned long unmapped_area(struct vm_unmapped_area_info *info);
extern unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info);
# 2280 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
vm_unmapped_area(struct vm_unmapped_area_info *info)
{
if (info->flags & 1)
return unmapped_area_topdown(info);
else
return unmapped_area(info);
}
extern void truncate_inode_pages(struct address_space *, loff_t);
extern void truncate_inode_pages_range(struct address_space *,
loff_t lstart, loff_t lend);
extern void truncate_inode_pages_final(struct address_space *);
extern int filemap_fault(struct vm_fault *vmf);
extern void filemap_map_pages(struct vm_fault *vmf,
unsigned long start_pgoff, unsigned long end_pgoff);
extern int filemap_page_mkwrite(struct vm_fault *vmf);
int __attribute__((warn_unused_result)) write_one_page(struct page *page);
void task_dirty_inc(struct task_struct *tsk);
int force_page_cache_readahead(struct address_space *mapping, struct file *filp,
unsigned long offset, unsigned long nr_to_read);
void page_cache_sync_readahead(struct address_space *mapping,
struct file_ra_state *ra,
struct file *filp,
unsigned long offset,
unsigned long size);
void page_cache_async_readahead(struct address_space *mapping,
struct file_ra_state *ra,
struct file *filp,
struct page *pg,
unsigned long offset,
unsigned long size);
extern unsigned long stack_guard_gap;
extern int expand_stack(struct vm_area_struct *vma, unsigned long address);
extern int expand_downwards(struct vm_area_struct *vma,
unsigned long address);
extern struct vm_area_struct * find_vma(struct mm_struct * mm, unsigned long addr);
extern struct vm_area_struct * find_vma_prev(struct mm_struct * mm, unsigned long addr,
struct vm_area_struct **pprev);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct vm_area_struct * find_vma_intersection(struct mm_struct * mm, unsigned long start_addr, unsigned long end_addr)
{
struct vm_area_struct * vma = find_vma(mm,start_addr);
if (vma && end_addr <= vma->vm_start)
vma = ((void *)0);
return vma;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long vm_start_gap(struct vm_area_struct *vma)
{
unsigned long vm_start = vma->vm_start;
if (vma->vm_flags & 0x00000100) {
vm_start -= stack_guard_gap;
if (vm_start > vma->vm_start)
vm_start = 0;
}
return vm_start;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long vm_end_gap(struct vm_area_struct *vma)
{
unsigned long vm_end = vma->vm_end;
if (vma->vm_flags & 0x00000000) {
vm_end += stack_guard_gap;
if (vm_end < vma->vm_end)
vm_end = -((1UL) << 12);
}
return vm_end;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long vma_pages(struct vm_area_struct *vma)
{
return (vma->vm_end - vma->vm_start) >> 12;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct vm_area_struct *find_exact_vma(struct mm_struct *mm,
unsigned long vm_start, unsigned long vm_end)
{
struct vm_area_struct *vma = find_vma(mm, vm_start);
if (vma && (vma->vm_start != vm_start || vma->vm_end != vm_end))
vma = ((void *)0);
return vma;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool range_in_vma(struct vm_area_struct *vma,
unsigned long start, unsigned long end)
{
return (vma && vma->vm_start <= start && end <= vma->vm_end);
}
pgprot_t vm_get_page_prot(unsigned long vm_flags);
void vma_set_page_prot(struct vm_area_struct *vma);
# 2420 "./include/linux/mm.h"
struct vm_area_struct *find_extend_vma(struct mm_struct *, unsigned long addr);
int remap_pfn_range(struct vm_area_struct *, unsigned long addr,
unsigned long pfn, unsigned long size, pgprot_t);
int vm_insert_page(struct vm_area_struct *, unsigned long addr, struct page *);
int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
unsigned long pfn);
int vm_insert_pfn_prot(struct vm_area_struct *vma, unsigned long addr,
unsigned long pfn, pgprot_t pgprot);
int vm_insert_mixed(struct vm_area_struct *vma, unsigned long addr,
pfn_t pfn);
int vm_insert_mixed_mkwrite(struct vm_area_struct *vma, unsigned long addr,
pfn_t pfn);
int vm_iomap_memory(struct vm_area_struct *vma, phys_addr_t start, unsigned long len);
struct page *follow_page_mask(struct vm_area_struct *vma,
unsigned long address, unsigned int foll_flags,
unsigned int *page_mask);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *follow_page(struct vm_area_struct *vma,
unsigned long address, unsigned int foll_flags)
{
unsigned int unused_page_mask;
return follow_page_mask(vma, address, foll_flags, &unused_page_mask);
}
# 2464 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int vm_fault_to_errno(int vm_fault, int foll_flags)
{
if (vm_fault & 0x0001)
return -12;
if (vm_fault & (0x0010 | 0x0020))
return (foll_flags & 0x100) ? -133 : -14;
if (vm_fault & (0x0002 | 0x0040))
return -14;
return 0;
}
typedef int (*pte_fn_t)(pte_t *pte, pgtable_t token, unsigned long addr,
void *data);
extern int apply_to_page_range(struct mm_struct *mm, unsigned long address,
unsigned long size, pte_fn_t fn, void *data);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool page_poisoning_enabled(void) { return false; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kernel_poison_pages(struct page *page, int numpages,
int enable) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool page_is_poisoned(struct page *page) { return false; }
extern struct static_key_false init_on_alloc;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool want_init_on_alloc(gfp_t flags)
{
if (__builtin_expect(!!(({ if (!__builtin_types_compatible_p(typeof(*&(&init_on_alloc)->key), struct static_key) && !__builtin_types_compatible_p(typeof(*&(&init_on_alloc)->key), struct static_key_true) && !__builtin_types_compatible_p(typeof(*&(&init_on_alloc)->key), struct static_key_false)) ____wrong_branch_error(); static_key_count((struct static_key *)&(&init_on_alloc)->key) > 0; })), 0) &&
!page_poisoning_enabled())
return true;
return flags & (( gfp_t)0x8000u);
}
extern struct static_key_false init_on_free;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool want_init_on_free(void)
{
return __builtin_expect(!!(({ if (!__builtin_types_compatible_p(typeof(*&(&init_on_free)->key), struct static_key) && !__builtin_types_compatible_p(typeof(*&(&init_on_free)->key), struct static_key_true) && !__builtin_types_compatible_p(typeof(*&(&init_on_free)->key), struct static_key_false)) ____wrong_branch_error(); static_key_count((struct static_key *)&(&init_on_free)->key) > 0; })), 0) &&
!page_poisoning_enabled();
}
# 2537 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
kernel_map_pages(struct page *page, int numpages, int enable) {}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool debug_pagealloc_enabled(void)
{
return false;
}
extern struct vm_area_struct *get_gate_vma(struct mm_struct *mm);
extern int in_gate_area_no_mm(unsigned long addr);
extern int in_gate_area(struct mm_struct *mm, unsigned long addr);
# 2564 "./include/linux/mm.h"
extern bool process_shares_mm(struct task_struct *p, struct mm_struct *mm);
extern int sysctl_drop_caches;
int drop_caches_sysctl_handler(struct ctl_table *, int,
void *, size_t *, loff_t *);
void drop_slab(void);
void drop_slab_node(int nid);
extern int randomize_va_space;
const char * arch_vma_name(struct vm_area_struct *vma);
void print_vma_addr(char *prefix, unsigned long rip);
void sparse_mem_maps_populate_node(struct page **map_map,
unsigned long pnum_begin,
unsigned long pnum_end,
unsigned long map_count,
int nodeid);
struct page *sparse_mem_map_populate(unsigned long pnum, int nid);
pgd_t *vmemmap_pgd_populate(unsigned long addr, int node);
pgd_t *vmemmap_p4d_populate(pgd_t *pgd, unsigned long addr, int node);
pud_t *vmemmap_pud_populate(pgd_t *p4d, unsigned long addr, int node);
pmd_t *vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node);
pte_t *vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node);
void *vmemmap_alloc_block(unsigned long size, int node);
struct vmem_altmap;
void *__vmemmap_alloc_block_buf(unsigned long size, int node,
struct vmem_altmap *altmap);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *vmemmap_alloc_block_buf(unsigned long size, int node)
{
return __vmemmap_alloc_block_buf(size, node, ((void *)0));
}
void vmemmap_verify(pte_t *, int, unsigned long, unsigned long);
int vmemmap_populate_basepages(unsigned long start, unsigned long end,
int node);
int vmemmap_populate(unsigned long start, unsigned long end, int node);
void vmemmap_populate_print_last(void);
void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
unsigned long nr_pages);
enum mf_flags {
MF_COUNT_INCREASED = 1 << 0,
MF_ACTION_REQUIRED = 1 << 1,
MF_MUST_KILL = 1 << 2,
MF_SOFT_OFFLINE = 1 << 3,
};
extern int memory_failure(unsigned long pfn, int trapno, int flags);
extern void memory_failure_queue(unsigned long pfn, int trapno, int flags);
extern int unpoison_memory(unsigned long pfn);
extern int get_hwpoison_page(struct page *page);
extern int sysctl_memory_failure_early_kill;
extern int sysctl_memory_failure_recovery;
extern void shake_page(struct page *p, int access);
extern atomic_long_t num_poisoned_pages;
extern int soft_offline_page(struct page *page, int flags);
enum mf_result {
MF_IGNORED,
MF_FAILED,
MF_DELAYED,
MF_RECOVERED,
};
enum mf_action_page_type {
MF_MSG_KERNEL,
MF_MSG_KERNEL_HIGH_ORDER,
MF_MSG_SLAB,
MF_MSG_DIFFERENT_COMPOUND,
MF_MSG_POISONED_HUGE,
MF_MSG_HUGE,
MF_MSG_FREE_HUGE,
MF_MSG_NON_PMD_HUGE,
MF_MSG_UNMAP_FAILED,
MF_MSG_DIRTY_SWAPCACHE,
MF_MSG_CLEAN_SWAPCACHE,
MF_MSG_DIRTY_MLOCKED_LRU,
MF_MSG_CLEAN_MLOCKED_LRU,
MF_MSG_DIRTY_UNEVICTABLE_LRU,
MF_MSG_CLEAN_UNEVICTABLE_LRU,
MF_MSG_DIRTY_LRU,
MF_MSG_CLEAN_LRU,
MF_MSG_TRUNCATED_LRU,
MF_MSG_BUDDY,
MF_MSG_BUDDY_2ND,
MF_MSG_UNKNOWN,
};
# 2681 "./include/linux/mm.h"
extern struct page_ext_operations debug_guardpage_ops;
# 2711 "./include/linux/mm.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int debug_guardpage_minorder(void) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool debug_guardpage_enabled(void) { return false; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool page_is_guard(struct page *page) { return false; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void setup_nr_node_ids(void) {}
# 9 "./include/linux/pagemap.h" 2
# 1 "./include/linux/highmem.h" 1
# 1 "./include/linux/uaccess.h" 1
# 14 "./include/linux/uaccess.h"
# 1 "./arch/arm/include/asm/uaccess.h" 1
# 16 "./arch/arm/include/asm/uaccess.h"
# 1 "./arch/arm/include/asm/domain.h" 1
# 88 "./arch/arm/include/asm/domain.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int get_domain(void)
{
unsigned int domain;
asm(
"mrc p15, 0, %0, c3, c0 @ get domain"
: "=r" (domain)
: "m" (current_thread_info()->cpu_domain));
return domain;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_domain(unsigned val)
{
asm volatile(
"mcr p15, 0, %0, c3, c0 @ set domain"
: : "r" (val) : "memory");
__asm__ __volatile__ ("isb " "" : : : "memory");
}
# 128 "./arch/arm/include/asm/domain.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void modify_domain(unsigned dom, unsigned type) { }
# 17 "./arch/arm/include/asm/uaccess.h" 2
# 1 "./arch/arm/include/generated/asm/extable.h" 1
# 1 "./include/asm-generic/extable.h" 1
# 18 "./include/asm-generic/extable.h"
struct exception_table_entry
{
unsigned long insn, fixup;
};
struct pt_regs;
extern int fixup_exception(struct pt_regs *regs);
# 2 "./arch/arm/include/generated/asm/extable.h" 2
# 21 "./arch/arm/include/asm/uaccess.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int uaccess_save_and_enable(void)
{
unsigned int old_domain = get_domain();
set_domain((old_domain & ~((3) << (2 * (1)))) |
((1) << (2 * (1))));
return old_domain;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void uaccess_restore(unsigned int flags)
{
set_domain(flags);
}
extern int __get_user_bad(void);
extern int __put_user_bad(void);
# 69 "./arch/arm/include/asm/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_fs(mm_segment_t fs)
{
current_thread_info()->addr_limit = fs;
__asm__ __volatile__ ("dsb " "nsh" : : : "memory");
__asm__ __volatile__ ("isb " "" : : : "memory");
modify_domain(2, fs ? 1 : 1);
}
# 108 "./arch/arm/include/asm/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *__uaccess_mask_range_ptr(const void *ptr,
size_t size)
{
void *safe_ptr = (void *)ptr;
unsigned long tmp;
asm volatile(
" sub %1, %3, #1\n"
" subs %1, %1, %0\n"
" addhs %1, %1, #1\n"
" subhss %1, %1, %2\n"
" movlo %0, #0\n"
: "+r" (safe_ptr), "=&r" (tmp)
: "r" (size), "r" (current_thread_info()->addr_limit)
: "cc");
__asm__ __volatile__(".inst 0xe320f014" : : : "memory");
return safe_ptr;
}
# 139 "./arch/arm/include/asm/uaccess.h"
extern int __get_user_1(void *);
extern int __get_user_2(void *);
extern int __get_user_4(void *);
extern int __get_user_32t_8(void *);
extern int __get_user_8(void *);
extern int __get_user_64t_1(void *);
extern int __get_user_64t_2(void *);
extern int __get_user_64t_4(void *);
# 239 "./arch/arm/include/asm/uaccess.h"
extern int __put_user_1(void *, unsigned int);
extern int __put_user_2(void *, unsigned int);
extern int __put_user_4(void *, unsigned int);
extern int __put_user_8(void *, unsigned long long);
# 514 "./arch/arm/include/asm/uaccess.h"
extern unsigned long __attribute__((warn_unused_result))
arm_copy_from_user(void *to, const void *from, unsigned long n);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __attribute__((warn_unused_result))
raw_copy_from_user(void *to, const void *from, unsigned long n)
{
unsigned int __ua_flags;
__ua_flags = uaccess_save_and_enable();
n = arm_copy_from_user(to, from, n);
uaccess_restore(__ua_flags);
return n;
}
extern unsigned long __attribute__((warn_unused_result))
arm_copy_to_user(void *to, const void *from, unsigned long n);
extern unsigned long __attribute__((warn_unused_result))
__copy_to_user_std(void *to, const void *from, unsigned long n);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __attribute__((warn_unused_result))
raw_copy_to_user(void *to, const void *from, unsigned long n)
{
unsigned int __ua_flags;
__ua_flags = uaccess_save_and_enable();
n = arm_copy_to_user(to, from, n);
uaccess_restore(__ua_flags);
return n;
}
extern unsigned long __attribute__((warn_unused_result))
arm_clear_user(void *addr, unsigned long n);
extern unsigned long __attribute__((warn_unused_result))
__clear_user_std(void *addr, unsigned long n);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __attribute__((warn_unused_result))
__clear_user(void *addr, unsigned long n)
{
unsigned int __ua_flags = uaccess_save_and_enable();
n = arm_clear_user(addr, n);
uaccess_restore(__ua_flags);
return n;
}
# 579 "./arch/arm/include/asm/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __attribute__((warn_unused_result)) clear_user(void *to, unsigned long n)
{
if ((({ unsigned long flag, roksum; (void)0; __asm__("adds %1, %2, %3; sbcccs %1, %1, %0; movcc %0, #0" : "=&r" (flag), "=&r" (roksum) : "r" (to), "Ir" (n), "0" (current_thread_info()->addr_limit) : "cc"); flag; }) == 0))
n = __clear_user(to, n);
return n;
}
extern long strncpy_from_user(char *dest, const char *src, long count);
extern __attribute__((warn_unused_result)) long strnlen_user(const char *str, long n);
# 15 "./include/linux/uaccess.h" 2
# 61 "./include/linux/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long
__copy_from_user_inatomic(void *to, const void *from, unsigned long n)
{
kasan_check_write(to, n);
check_object_size(to, n, false);
return raw_copy_from_user(to, from, n);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long
__copy_from_user(void *to, const void *from, unsigned long n)
{
might_fault();
kasan_check_write(to, n);
check_object_size(to, n, false);
return raw_copy_from_user(to, from, n);
}
# 91 "./include/linux/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long
__copy_to_user_inatomic(void *to, const void *from, unsigned long n)
{
kasan_check_read(from, n);
check_object_size(from, n, true);
return raw_copy_to_user(to, from, n);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long
__copy_to_user(void *to, const void *from, unsigned long n)
{
might_fault();
kasan_check_read(from, n);
check_object_size(from, n, true);
return raw_copy_to_user(to, from, n);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
_copy_from_user(void *to, const void *from, unsigned long n)
{
unsigned long res = n;
might_fault();
if (__builtin_expect(!!((({ unsigned long flag, roksum; (void)0; __asm__("adds %1, %2, %3; sbcccs %1, %1, %0; movcc %0, #0" : "=&r" (flag), "=&r" (roksum) : "r" (from), "Ir" (n), "0" (current_thread_info()->addr_limit) : "cc"); flag; }) == 0)), 1)) {
kasan_check_write(to, n);
res = raw_copy_from_user(to, from, n);
}
if (__builtin_expect(!!(res), 0))
({ void *__p = (to + (n - res)); size_t __n = res; if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
return res;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long
_copy_to_user(void *to, const void *from, unsigned long n)
{
might_fault();
if ((({ unsigned long flag, roksum; (void)0; __asm__("adds %1, %2, %3; sbcccs %1, %1, %0; movcc %0, #0" : "=&r" (flag), "=&r" (roksum) : "r" (to), "Ir" (n), "0" (current_thread_info()->addr_limit) : "cc"); flag; }) == 0)) {
kasan_check_read(from, n);
n = raw_copy_to_user(to, from, n);
}
return n;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long __attribute__((warn_unused_result))
copy_from_user(void *to, const void *from, unsigned long n)
{
if (__builtin_expect(!!(check_copy_size(to, n, false)), 1))
n = _copy_from_user(to, from, n);
return n;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) unsigned long __attribute__((warn_unused_result))
copy_to_user(void *to, const void *from, unsigned long n)
{
if (__builtin_expect(!!(check_copy_size(from, n, true)), 1))
n = _copy_to_user(to, from, n);
return n;
}
# 169 "./include/linux/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void pagefault_disabled_inc(void)
{
(current_thread_info()->task)->pagefault_disabled++;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) __attribute__((always_inline)) void pagefault_disabled_dec(void)
{
(current_thread_info()->task)->pagefault_disabled--;
}
# 186 "./include/linux/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pagefault_disable(void)
{
pagefault_disabled_inc();
__asm__ __volatile__("": : :"memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pagefault_enable(void)
{
__asm__ __volatile__("": : :"memory");
pagefault_disabled_dec();
}
# 225 "./include/linux/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long __copy_from_user_inatomic_nocache(void *to,
const void *from, unsigned long n)
{
return __copy_from_user_inatomic(to, from, n);
}
# 242 "./include/linux/uaccess.h"
extern long probe_kernel_read(void *dst, const void *src, size_t size);
extern long __probe_kernel_read(void *dst, const void *src, size_t size);
# 254 "./include/linux/uaccess.h"
extern long __attribute__((no_instrument_function)) probe_kernel_write(void *dst, const void *src, size_t size);
extern long __attribute__((no_instrument_function)) __probe_kernel_write(void *dst, const void *src, size_t size);
extern long strncpy_from_unsafe(char *dst, const void *unsafe_addr, long count);
# 274 "./include/linux/uaccess.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long user_access_save(void) { return 0UL; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void user_access_restore(unsigned long flags) { }
# 10 "./include/linux/highmem.h" 2
# 1 "./include/linux/hardirq.h" 1
# 1 "./include/linux/ftrace_irq.h" 1
# 10 "./include/linux/ftrace_irq.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_ftrace_nmi_enter(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void arch_ftrace_nmi_exit(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ftrace_nmi_enter(void)
{
arch_ftrace_nmi_enter();
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ftrace_nmi_exit(void)
{
arch_ftrace_nmi_exit();
}
# 8 "./include/linux/hardirq.h" 2
# 1 "./include/linux/vtime.h" 1
# 1 "./include/linux/context_tracking_state.h" 1
# 1 "./include/linux/static_key.h" 1
# 7 "./include/linux/context_tracking_state.h" 2
struct context_tracking {
bool active;
int recursion;
enum ctx_state {
CONTEXT_DISABLED = -1,
CONTEXT_KERNEL = 0,
CONTEXT_USER,
CONTEXT_GUEST,
} state;
};
# 44 "./include/linux/context_tracking_state.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool context_tracking_in_user(void) { return false; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool context_tracking_active(void) { return false; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool context_tracking_is_enabled(void) { return false; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool context_tracking_cpu_is_enabled(void) { return false; }
# 6 "./include/linux/vtime.h" 2
struct task_struct;
# 40 "./include/linux/vtime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool vtime_accounting_cpu_enabled(void) { return false; }
# 65 "./include/linux/vtime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_task_switch(struct task_struct *prev) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_account_system(struct task_struct *tsk) { }
# 77 "./include/linux/vtime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_user_enter(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_user_exit(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_guest_enter(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_guest_exit(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_init_idle(struct task_struct *tsk, int cpu) { }
# 93 "./include/linux/vtime.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_account_irq_enter(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_account_irq_exit(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vtime_flush(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void irqtime_account_irq(struct task_struct *tsk) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void account_irq_enter_time(struct task_struct *tsk)
{
vtime_account_irq_enter(tsk);
irqtime_account_irq(tsk);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void account_irq_exit_time(struct task_struct *tsk)
{
vtime_account_irq_exit(tsk);
irqtime_account_irq(tsk);
}
# 9 "./include/linux/hardirq.h" 2
# 1 "./arch/arm/include/asm/hardirq.h" 1
# 1 "./arch/arm/include/asm/irq.h" 1
# 1 "arch/arm/mach-rtd2851a/include/mach/irqs.h" 1
# 9 "./arch/arm/include/asm/irq.h" 2
# 26 "./arch/arm/include/asm/irq.h"
struct irqaction;
struct pt_regs;
extern void asm_do_IRQ(unsigned int, struct pt_regs *);
void handle_IRQ(unsigned int, struct pt_regs *);
void init_IRQ(void);
extern void (*handle_arch_irq)(struct pt_regs *);
extern void set_handle_irq(void (*handle_irq)(struct pt_regs *));
extern void arch_trigger_cpumask_backtrace(const cpumask_t *mask,
bool exclude_self);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int nr_legacy_irqs(void)
{
return 16;
}
# 8 "./arch/arm/include/asm/hardirq.h" 2
typedef struct {
unsigned int __softirq_pending;
unsigned int ipi_irqs[10];
} __attribute__((__aligned__((1 << 6)))) irq_cpustat_t;
# 1 "./include/linux/irq_cpustat.h" 1
# 21 "./include/linux/irq_cpustat.h"
extern irq_cpustat_t irq_stat[];
# 20 "./arch/arm/include/asm/hardirq.h" 2
u64 smp_irq_stat_cpu(unsigned int cpu);
# 10 "./include/linux/hardirq.h" 2
extern void synchronize_irq(unsigned int irq);
extern bool synchronize_hardirq(unsigned int irq);
# 26 "./include/linux/hardirq.h"
extern void rcu_nmi_enter(void);
extern void rcu_nmi_exit(void);
# 46 "./include/linux/hardirq.h"
extern void irq_enter(void);
# 61 "./include/linux/hardirq.h"
extern void irq_exit(void);
# 11 "./include/linux/highmem.h" 2
# 1 "./arch/arm/include/asm/cacheflush.h" 1
# 15 "./arch/arm/include/asm/cacheflush.h"
# 1 "./arch/arm/include/asm/glue-cache.h" 1
# 13 "./arch/arm/include/asm/glue-cache.h"
# 1 "./arch/arm/include/asm/glue.h" 1
# 14 "./arch/arm/include/asm/glue-cache.h" 2
# 129 "./arch/arm/include/asm/glue-cache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_flush_icache_all(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_flush_kern_cache_all(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_flush_kern_cache_louis(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_flush_user_cache_all(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_flush_user_cache_range(unsigned long a,
unsigned long b, unsigned int c) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_coherent_kern_range(unsigned long a, unsigned long b) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int nop_coherent_user_range(unsigned long a,
unsigned long b) { return 0; }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_flush_kern_dcache_area(void *a, size_t s) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_dma_flush_range(const void *a, const void *b) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_dma_map_area(const void *s, size_t l, int f) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void nop_dma_unmap_area(const void *s, size_t l, int f) { }
# 16 "./arch/arm/include/asm/cacheflush.h" 2
# 1 "./arch/arm/include/asm/cachetype.h" 1
# 13 "./arch/arm/include/asm/cachetype.h"
extern unsigned int cacheid;
# 54 "./arch/arm/include/asm/cachetype.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int __attribute__((pure)) cacheid_is(unsigned int mask)
{
return ((0) & mask) |
(~((1 << 0)) & ((1 << 1) | (1 << 3) | (1 << 4) | (1 << 5)) & mask & cacheid);
}
# 72 "./arch/arm/include/asm/cachetype.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_csselr(unsigned int cache_selector)
{
asm volatile("mcr p15, 2, %0, c0, c0, 0" : : "r" (cache_selector));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int read_ccsidr(void)
{
unsigned int val;
asm volatile("mrc p15, 1, %0, c0, c0, 0" : "=r" (val));
return val;
}
# 18 "./arch/arm/include/asm/cacheflush.h" 2
# 1 "./arch/arm/include/asm/outercache.h" 1
# 26 "./arch/arm/include/asm/outercache.h"
struct l2x0_regs;
struct outer_cache_fns {
void (*inv_range)(unsigned long, unsigned long);
void (*clean_range)(unsigned long, unsigned long);
void (*flush_range)(unsigned long, unsigned long);
void (*flush_all)(void);
void (*disable)(void);
void (*resume)(void);
void (*write_sec)(unsigned long, unsigned);
void (*configure)(const struct l2x0_regs *);
};
extern struct outer_cache_fns outer_cache;
# 120 "./arch/arm/include/asm/outercache.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outer_inv_range(phys_addr_t start, phys_addr_t end)
{ }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outer_clean_range(phys_addr_t start, phys_addr_t end)
{ }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outer_flush_range(phys_addr_t start, phys_addr_t end)
{ }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outer_flush_all(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outer_disable(void) { }
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outer_resume(void) { }
# 19 "./arch/arm/include/asm/cacheflush.h" 2
# 104 "./arch/arm/include/asm/cacheflush.h"
struct cpu_cache_fns {
void (*flush_icache_all)(void);
void (*flush_kern_all)(void);
void (*flush_kern_louis)(void);
void (*flush_user_all)(void);
void (*flush_user_range)(unsigned long, unsigned long, unsigned int);
void (*coherent_kern_range)(unsigned long, unsigned long);
int (*coherent_user_range)(unsigned long, unsigned long);
void (*flush_kern_dcache_area)(void *, size_t);
void (*dma_map_area)(const void *, size_t, int);
void (*dma_unmap_area)(const void *, size_t, int);
void (*dma_flush_range)(const void *, const void *);
} ;
# 147 "./arch/arm/include/asm/cacheflush.h"
extern void v7_flush_icache_all(void);
extern void v7_flush_kern_cache_all(void);
extern void v7_flush_kern_cache_louis(void);
extern void v7_flush_user_cache_all(void);
extern void v7_flush_user_cache_range(unsigned long, unsigned long, unsigned int);
extern void v7_coherent_kern_range(unsigned long, unsigned long);
extern int v7_coherent_user_range(unsigned long, unsigned long);
extern void v7_flush_kern_dcache_area(void *, size_t);
extern void v7_dma_flush_range(const void *, const void *);
extern void v7_dma_inv_range(const void *, const void *);
# 174 "./arch/arm/include/asm/cacheflush.h"
extern void copy_to_user_page(struct vm_area_struct *, struct page *,
unsigned long, void *, const void *, unsigned long);
# 211 "./arch/arm/include/asm/cacheflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __flush_icache_all(void)
{
v7_flush_icache_all();
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
}
# 224 "./arch/arm/include/asm/cacheflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void vivt_flush_cache_mm(struct mm_struct *mm)
{
if (cpumask_test_cpu((current_thread_info()->cpu), mm_cpumask(mm)))
v7_flush_user_cache_all();
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
vivt_flush_cache_range(struct vm_area_struct *vma, unsigned long start, unsigned long end)
{
struct mm_struct *mm = vma->vm_mm;
if (!mm || cpumask_test_cpu((current_thread_info()->cpu), mm_cpumask(mm)))
v7_flush_user_cache_range(start & (~((1 << 12) - 1)), ((((end)) + ((typeof((end)))((((1UL) << 12))) - 1)) & ~((typeof((end)))((((1UL) << 12))) - 1)),
vma->vm_flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
vivt_flush_cache_page(struct vm_area_struct *vma, unsigned long user_addr, unsigned long pfn)
{
struct mm_struct *mm = vma->vm_mm;
if (!mm || cpumask_test_cpu((current_thread_info()->cpu), mm_cpumask(mm))) {
unsigned long addr = user_addr & (~((1 << 12) - 1));
v7_flush_user_cache_range(addr, addr + ((1UL) << 12), vma->vm_flags);
}
}
# 259 "./arch/arm/include/asm/cacheflush.h"
extern void flush_cache_mm(struct mm_struct *mm);
extern void flush_cache_range(struct vm_area_struct *vma, unsigned long start, unsigned long end);
extern void flush_cache_page(struct vm_area_struct *vma, unsigned long user_addr, unsigned long pfn);
# 298 "./arch/arm/include/asm/cacheflush.h"
extern void flush_dcache_page(struct page *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void flush_kernel_vmap_range(void *addr, int size)
{
if ((cacheid_is((1 << 0)) || cacheid_is((1 << 2))))
v7_flush_kern_dcache_area(addr, (size_t)size);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void invalidate_kernel_vmap_range(void *addr, int size)
{
if ((cacheid_is((1 << 0)) || cacheid_is((1 << 2))))
v7_flush_kern_dcache_area(addr, (size_t)size);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void flush_anon_page(struct vm_area_struct *vma,
struct page *page, unsigned long vmaddr)
{
extern void __flush_anon_page(struct vm_area_struct *vma,
struct page *, unsigned long);
if (PageAnon(page))
__flush_anon_page(vma, page, vmaddr);
}
extern void flush_kernel_dcache_page(struct page *);
# 345 "./arch/arm/include/asm/cacheflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void flush_cache_vmap(unsigned long start, unsigned long end)
{
if (!cacheid_is((1 << 1)))
v7_flush_kern_cache_all();
else
__asm__ __volatile__ ("dsb " "ishst" : : : "memory");
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void flush_cache_vunmap(unsigned long start, unsigned long end)
{
if (!cacheid_is((1 << 1)))
v7_flush_kern_cache_all();
}
# 400 "./arch/arm/include/asm/cacheflush.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __sync_cache_range_w(volatile void *p, size_t size)
{
char *_p = (char *)p;
v7_flush_kern_dcache_area(_p, size);
outer_clean_range(__virt_to_phys_nodebug((unsigned long)(_p)), __virt_to_phys_nodebug((unsigned long)(_p + size)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __sync_cache_range_r(volatile void *p, size_t size)
{
char *_p = (char *)p;
# 432 "./arch/arm/include/asm/cacheflush.h"
v7_flush_kern_dcache_area(_p, size);
}
# 484 "./arch/arm/include/asm/cacheflush.h"
void flush_uprobe_xol_access(struct page *page, unsigned long uaddr,
void *kaddr, unsigned long len);
# 13 "./include/linux/highmem.h" 2
# 32 "./include/linux/highmem.h"
# 1 "./arch/arm/include/asm/kmap_types.h" 1
# 33 "./include/linux/highmem.h" 2
# 1 "./arch/arm/include/asm/highmem.h" 1
# 21 "./arch/arm/include/asm/highmem.h"
extern pte_t *pkmap_page_table;
extern pte_t *fixmap_page_table;
extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
# 54 "./arch/arm/include/asm/highmem.h"
extern void *kmap_high_get(struct page *page);
# 67 "./arch/arm/include/asm/highmem.h"
extern void *kmap(struct page *page);
extern void kunmap(struct page *page);
extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
# 36 "./include/linux/highmem.h" 2
unsigned int nr_free_highpages(void);
extern unsigned long totalhigh_pages;
void kmap_flush_unused(void);
struct page *kmap_to_page(void *addr);
# 90 "./include/linux/highmem.h"
extern __attribute__((section(".data..percpu" ""))) __typeof__(int) __kmap_atomic_idx;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kmap_atomic_idx_push(void)
{
int idx = ({ __this_cpu_preempt_check("add_return"); ({ typeof(__kmap_atomic_idx) pscr2_ret__; do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(__kmap_atomic_idx)) { case 1: pscr2_ret__ = ({ typeof(&(__kmap_atomic_idx)) __p = ({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); *__p += 1; *__p; }); break; case 2: pscr2_ret__ = ({ typeof(&(__kmap_atomic_idx)) __p = ({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); *__p += 1; *__p; }); break; case 4: pscr2_ret__ = ({ typeof(&(__kmap_atomic_idx)) __p = ({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); *__p += 1; *__p; }); break; case 8: pscr2_ret__ = ({ typeof(&(__kmap_atomic_idx)) __p = ({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); *__p += 1; *__p; }); break; default: __bad_size_call_parameter(); break; } pscr2_ret__; }); }) - 1;
return idx;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int kmap_atomic_idx(void)
{
return ({ __this_cpu_preempt_check("read"); ({ typeof(__kmap_atomic_idx) pscr_ret__; do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(__kmap_atomic_idx)) { case 1: pscr_ret__ = ({ *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); }); break; case 2: pscr_ret__ = ({ *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); }); break; case 4: pscr_ret__ = ({ *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); }); break; case 8: pscr_ret__ = ({ *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }); }); break; default: __bad_size_call_parameter(); break; } pscr_ret__; }); }) - 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void kmap_atomic_idx_pop(void)
{
({ __this_cpu_preempt_check("add"); do { do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); switch(sizeof(__kmap_atomic_idx)) { case 1: do { *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(__kmap_atomic_idx))(1); } while (0);break; case 2: do { *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(__kmap_atomic_idx))(1); } while (0);break; case 4: do { *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(__kmap_atomic_idx))(1); } while (0);break; case 8: do { *({ do { const void *__vpp_verify = (typeof((&(__kmap_atomic_idx)) + 0))((void *)0); (void)__vpp_verify; } while (0); ({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))); (typeof((typeof(*(&(__kmap_atomic_idx))) *)(&(__kmap_atomic_idx)))) (__ptr + ((__my_cpu_offset()))); }); }) += -(typeof(__kmap_atomic_idx))(1); } while (0);break; default: __bad_size_call_parameter();break; } } while (0); });
}
# 157 "./include/linux/highmem.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *
__alloc_zeroed_user_highpage(gfp_t movableflags,
struct vm_area_struct *vma,
unsigned long vaddr)
{
struct page *page = alloc_pages_node(numa_node_id(), (((( gfp_t)(0x400000u|0x1000000u)) | (( gfp_t)0x40u) | (( gfp_t)0x80u) | (( gfp_t)0x20000u)) | (( gfp_t)0x02u)) | movableflags, 0);
if (page)
cpu_user.cpu_clear_user_highpage(page, vaddr);
return page;
}
# 180 "./include/linux/highmem.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *
alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma,
unsigned long vaddr)
{
return __alloc_zeroed_user_highpage((( gfp_t)0x08u), vma, vaddr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_highpage(struct page *page)
{
void *kaddr = kmap_atomic(page);
({ void *__p = ((void *)(kaddr)); size_t __n = ((1UL) << 12); if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
do { do { bool __cond = !(!(__builtin_types_compatible_p(typeof((kaddr)), typeof(struct page *)))); extern void __compiletime_assert_59(void) ; if (__cond) __compiletime_assert_59(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __kunmap_atomic(kaddr); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void zero_user_segments(struct page *page,
unsigned start1, unsigned end1,
unsigned start2, unsigned end2)
{
void *kaddr = kmap_atomic(page);
do { if (__builtin_expect(!!(end1 > ((1UL) << 12) || end2 > ((1UL) << 12)), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/highmem.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "200" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
if (end1 > start1)
({ void *__p = (kaddr + start1); size_t __n = end1 - start1; if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
if (end2 > start2)
({ void *__p = (kaddr + start2); size_t __n = end2 - start2; if ((__n) != 0) { if (__builtin_constant_p((0)) && (0) == 0) __memzero((__p),(__n)); else memset((__p),(0),(__n)); } (__p); });
do { do { bool __cond = !(!(__builtin_types_compatible_p(typeof((kaddr)), typeof(struct page *)))); extern void __compiletime_assert_60(void) ; if (__cond) __compiletime_assert_60(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __kunmap_atomic(kaddr); } while (0);
flush_dcache_page(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void zero_user_segment(struct page *page,
unsigned start, unsigned end)
{
zero_user_segments(page, start, end, 0, 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void zero_user(struct page *page,
unsigned start, unsigned size)
{
zero_user_segments(page, start, start + size, 0, 0);
}
# 240 "./include/linux/highmem.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void copy_highpage(struct page *to, struct page *from)
{
char *vfrom, *vto;
vfrom = kmap_atomic(from);
vto = kmap_atomic(to);
copy_page(vto, vfrom);
do { do { bool __cond = !(!(__builtin_types_compatible_p(typeof((vto)), typeof(struct page *)))); extern void __compiletime_assert_61(void) ; if (__cond) __compiletime_assert_61(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __kunmap_atomic(vto); } while (0);
do { do { bool __cond = !(!(__builtin_types_compatible_p(typeof((vfrom)), typeof(struct page *)))); extern void __compiletime_assert_62(void) ; if (__cond) __compiletime_assert_62(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __kunmap_atomic(vfrom); } while (0);
}
# 12 "./include/linux/pagemap.h" 2
# 1 "./include/linux/hugetlb_inline.h" 1
# 16 "./include/linux/hugetlb_inline.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool is_vm_hugetlb_page(struct vm_area_struct *vma)
{
return false;
}
# 18 "./include/linux/pagemap.h" 2
enum mapping_flags {
AS_EIO = 0,
AS_ENOSPC = 1,
AS_MM_ALL_LOCKS = 2,
AS_UNEVICTABLE = 3,
AS_EXITING = 4,
AS_NO_WRITEBACK_TAGS = 5,
};
# 46 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_set_error(struct address_space *mapping, int error)
{
if (__builtin_expect(!!(!error), 1))
return;
filemap_set_wb_err(mapping, error);
if (error == -28)
_set_bit(AS_ENOSPC,&mapping->flags);
else
_set_bit(AS_EIO,&mapping->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_set_unevictable(struct address_space *mapping)
{
_set_bit(AS_UNEVICTABLE,&mapping->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_clear_unevictable(struct address_space *mapping)
{
_clear_bit(AS_UNEVICTABLE,&mapping->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mapping_unevictable(struct address_space *mapping)
{
if (mapping)
return test_bit(AS_UNEVICTABLE, &mapping->flags);
return !!mapping;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_set_exiting(struct address_space *mapping)
{
_set_bit(AS_EXITING,&mapping->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mapping_exiting(struct address_space *mapping)
{
return test_bit(AS_EXITING, &mapping->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_set_no_writeback_tags(struct address_space *mapping)
{
_set_bit(AS_NO_WRITEBACK_TAGS,&mapping->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int mapping_use_writeback_tags(struct address_space *mapping)
{
return !test_bit(AS_NO_WRITEBACK_TAGS, &mapping->flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) gfp_t mapping_gfp_mask(struct address_space * mapping)
{
return mapping->gfp_mask;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) gfp_t mapping_gfp_constraint(struct address_space *mapping,
gfp_t gfp_mask)
{
return mapping_gfp_mask(mapping) & gfp_mask;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void mapping_set_gfp_mask(struct address_space *m, gfp_t mask)
{
m->gfp_mask = mask;
}
void release_pages(struct page **pages, int nr, bool cold);
# 165 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_cache_get_speculative(struct page *page)
{
# 184 "./include/linux/pagemap.h"
if (__builtin_expect(!!(!get_page_unless_zero(page)), 0)) {
return 0;
}
((void)(sizeof(( long)(PageTail(page)))));
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int page_cache_add_speculative(struct page *page, int count)
{
((void)(sizeof(( long)(((preempt_count() & ((((1UL << (4))-1) << ((0 + 8) + 8)) | (((1UL << (8))-1) << (0 + 8)) | (((1UL << (1))-1) << (((0 + 8) + 8) + 4)))))))));
# 213 "./include/linux/pagemap.h"
if (__builtin_expect(!!(!page_ref_add_unless(page, count, 0)), 0))
return 0;
((void)(sizeof(( long)(PageCompound(page) && page != compound_head(page)))));
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *__page_cache_alloc(gfp_t gfp)
{
return alloc_pages_node(numa_node_id(), gfp, 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *page_cache_alloc(struct address_space *x)
{
return __page_cache_alloc(mapping_gfp_mask(x));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *page_cache_alloc_cold(struct address_space *x)
{
return __page_cache_alloc(mapping_gfp_mask(x)|(( gfp_t)0x100u));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) gfp_t readahead_gfp_mask(struct address_space *x)
{
return mapping_gfp_mask(x) |
(( gfp_t)0x100u) | (( gfp_t)0x1000u) | (( gfp_t)0x200u);
}
typedef int filler_t(struct file *, struct page *);
unsigned long page_cache_next_hole(struct address_space *mapping,
unsigned long index, unsigned long max_scan);
unsigned long page_cache_prev_hole(struct address_space *mapping,
unsigned long index, unsigned long max_scan);
# 261 "./include/linux/pagemap.h"
struct page *pagecache_get_page(struct address_space *mapping, unsigned long offset,
int fgp_flags, gfp_t cache_gfp_mask);
# 274 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *find_get_page(struct address_space *mapping,
unsigned long offset)
{
return pagecache_get_page(mapping, offset, 0, 0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *find_get_page_flags(struct address_space *mapping,
unsigned long offset, int fgp_flags)
{
return pagecache_get_page(mapping, offset, fgp_flags, 0);
}
# 299 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *find_lock_page(struct address_space *mapping,
unsigned long offset)
{
return pagecache_get_page(mapping, offset, 0x00000002, 0);
}
# 324 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *find_or_create_page(struct address_space *mapping,
unsigned long offset, gfp_t gfp_mask)
{
return pagecache_get_page(mapping, offset,
0x00000002|0x00000001|0x00000004,
gfp_mask);
}
# 345 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *grab_cache_page_nowait(struct address_space *mapping,
unsigned long index)
{
return pagecache_get_page(mapping, index,
0x00000002|0x00000004|0x00000010|0x00000020,
mapping_gfp_mask(mapping));
}
struct page *find_get_entry(struct address_space *mapping, unsigned long offset);
struct page *find_lock_entry(struct address_space *mapping, unsigned long offset);
unsigned find_get_entries(struct address_space *mapping, unsigned long start,
unsigned int nr_entries, struct page **entries,
unsigned long *indices);
unsigned find_get_pages_range(struct address_space *mapping, unsigned long *start,
unsigned long end, unsigned int nr_pages,
struct page **pages);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned find_get_pages(struct address_space *mapping,
unsigned long *start, unsigned int nr_pages,
struct page **pages)
{
return find_get_pages_range(mapping, start, (unsigned long)-1, nr_pages,
pages);
}
unsigned find_get_pages_contig(struct address_space *mapping, unsigned long start,
unsigned int nr_pages, struct page **pages);
unsigned find_get_pages_range_tag(struct address_space *mapping, unsigned long *index,
unsigned long end, int tag, unsigned int nr_pages,
struct page **pages);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned find_get_pages_tag(struct address_space *mapping,
unsigned long *index, int tag, unsigned int nr_pages,
struct page **pages)
{
return find_get_pages_range_tag(mapping, index, (unsigned long)-1, tag,
nr_pages, pages);
}
unsigned find_get_entries_tag(struct address_space *mapping, unsigned long start,
int tag, unsigned int nr_entries,
struct page **entries, unsigned long *indices);
struct page *grab_cache_page_write_begin(struct address_space *mapping,
unsigned long index, unsigned flags);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *grab_cache_page(struct address_space *mapping,
unsigned long index)
{
return find_or_create_page(mapping, index, mapping_gfp_mask(mapping));
}
extern struct page * read_cache_page(struct address_space *mapping,
unsigned long index, filler_t *filler, void *data);
extern struct page * read_cache_page_gfp(struct address_space *mapping,
unsigned long index, gfp_t gfp_mask);
extern int read_cache_pages(struct address_space *mapping,
struct list_head *pages, filler_t *filler, void *data);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *read_mapping_page(struct address_space *mapping,
unsigned long index, void *data)
{
filler_t *filler = mapping->a_ops->readpage;
return read_cache_page(mapping, index, filler, data);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long page_to_index(struct page *page)
{
unsigned long pgoff;
if (__builtin_expect(!!(!PageTransTail(page)), 1))
return page->index;
pgoff = compound_head(page)->index;
pgoff += page - compound_head(page);
return pgoff;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long page_to_pgoff(struct page *page)
{
if (__builtin_expect(!!(PageHeadHuge(page)), 0))
return page->index << compound_order(page);
return page_to_index(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) loff_t page_offset(struct page *page)
{
return ((loff_t)page->index) << 12;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) loff_t page_file_offset(struct page *page)
{
return ((loff_t)page_index(page)) << 12;
}
extern unsigned long linear_hugepage_index(struct vm_area_struct *vma,
unsigned long address);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long linear_page_index(struct vm_area_struct *vma,
unsigned long address)
{
unsigned long pgoff;
if (__builtin_expect(!!(is_vm_hugetlb_page(vma)), 0))
return linear_hugepage_index(vma, address);
pgoff = (address - vma->vm_start) >> 12;
pgoff += vma->vm_pgoff;
return pgoff;
}
extern void __lock_page(struct page *page);
extern int __lock_page_killable(struct page *page);
extern int __lock_page_or_retry(struct page *page, struct mm_struct *mm,
unsigned int flags);
extern void unlock_page(struct page *page);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int trylock_page(struct page *page)
{
page = compound_head(page);
return (__builtin_expect(!!(!_test_and_set_bit(PG_locked,&page->flags)), 1));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void lock_page(struct page *page)
{
do { do { } while (0); } while (0);
if (!trylock_page(page))
__lock_page(page);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int lock_page_killable(struct page *page)
{
do { do { } while (0); } while (0);
if (!trylock_page(page))
return __lock_page_killable(page);
return 0;
}
# 511 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int lock_page_or_retry(struct page *page, struct mm_struct *mm,
unsigned int flags)
{
do { do { } while (0); } while (0);
return trylock_page(page) || __lock_page_or_retry(page, mm, flags);
}
extern void wait_on_page_bit(struct page *page, int bit_nr);
extern int wait_on_page_bit_killable(struct page *page, int bit_nr);
# 532 "./include/linux/pagemap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void wait_on_page_locked(struct page *page)
{
if (PageLocked(page))
wait_on_page_bit(compound_head(page), PG_locked);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int wait_on_page_locked_killable(struct page *page)
{
if (!PageLocked(page))
return 0;
return wait_on_page_bit_killable(compound_head(page), PG_locked);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void wait_on_page_writeback(struct page *page)
{
if (PageWriteback(page))
wait_on_page_bit(page, PG_writeback);
}
extern void end_page_writeback(struct page *page);
void wait_for_stable_page(struct page *page);
void page_endio(struct page *page, bool is_write, int err);
extern void add_page_wait_queue(struct page *page, wait_queue_entry_t *waiter);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int fault_in_pages_writeable(char *uaddr, int size)
{
char *end = uaddr + size - 1;
if (__builtin_expect(!!(size == 0), 0))
return 0;
if (__builtin_expect(!!(uaddr > end), 0))
return -14;
do {
if (__builtin_expect(!!(({ int __pu_err = 0; do { const __typeof__(*((uaddr))) *__pu_ptr = ((uaddr)); __typeof__(*((uaddr))) __pu_val = ((0)); unsigned int __ua_flags; might_fault(); __ua_flags = uaccess_save_and_enable(); switch (sizeof(*((uaddr)))) { case 1: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "1" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; case 2: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "2" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; case 4: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "4" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; case 8: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "8" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; default: __pu_err = __put_user_bad(); break; } uaccess_restore(__ua_flags); } while (0); __pu_err; }) != 0), 0))
return -14;
uaddr += ((1UL) << 12);
} while (uaddr <= end);
if (((unsigned long)uaddr & (~((1 << 12) - 1))) ==
((unsigned long)end & (~((1 << 12) - 1))))
return ({ int __pu_err = 0; do { const __typeof__(*((end))) *__pu_ptr = ((end)); __typeof__(*((end))) __pu_val = ((0)); unsigned int __ua_flags; might_fault(); __ua_flags = uaccess_save_and_enable(); switch (sizeof(*((end)))) { case 1: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "1" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; case 2: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "2" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; case 4: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "4" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; case 8: ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register typeof(__pu_val) __r2 asm("r2") = __pu_val; register const void *__p asm("r0") = __pu_ptr; register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r2" "; " ".ifnc " "%2" "r2" ",fpr11; " ".ifnc " "%2" "r2" ",r11fp; " ".ifnc " "%2" "r2" ",ipr12; " ".ifnc " "%2" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __put_user_" "8" : "=&r" (__e) : "0" (__p), "r" (__r2), "r" (__l) : "ip", "lr", "cc"); __pu_err = __e; }); break; default: __pu_err = __put_user_bad(); break; } uaccess_restore(__ua_flags); } while (0); __pu_err; });
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int fault_in_pages_readable(const char *uaddr, int size)
{
volatile char c;
const char *end = uaddr + size - 1;
if (__builtin_expect(!!(size == 0), 0))
return 0;
if (__builtin_expect(!!(uaddr > end), 0))
return -14;
do {
if (__builtin_expect(!!(({ might_fault(); ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register const typeof(*(uaddr)) *__p asm("r0") = (uaddr); register __typeof__(__builtin_choose_expr(sizeof(c) > sizeof(0UL), 0ULL, 0UL)) __r2 asm("r2"); register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); unsigned int __ua_flags = uaccess_save_and_enable(); switch (sizeof(*(__p))) { case 1: if (sizeof((c)) >= 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "1" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "1" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; case 2: if (sizeof((c)) >= 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "2" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "2" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; case 4: if (sizeof((c)) >= 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "4" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "4" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; case 8: if (sizeof((c)) < 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "4" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "8" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; default: __e = __get_user_bad(); break; } uaccess_restore(__ua_flags); c = (typeof(*(uaddr))) __r2; __e; }); }) != 0), 0))
return -14;
uaddr += ((1UL) << 12);
} while (uaddr <= end);
if (((unsigned long)uaddr & (~((1 << 12) - 1))) ==
((unsigned long)end & (~((1 << 12) - 1)))) {
return ({ might_fault(); ({ unsigned long __limit = current_thread_info()->addr_limit - 1; register const typeof(*(end)) *__p asm("r0") = (end); register __typeof__(__builtin_choose_expr(sizeof(c) > sizeof(0UL), 0ULL, 0UL)) __r2 asm("r2"); register unsigned long __l asm("r1") = __limit; register int __e asm("r0"); unsigned int __ua_flags = uaccess_save_and_enable(); switch (sizeof(*(__p))) { case 1: if (sizeof((c)) >= 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "1" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "1" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; case 2: if (sizeof((c)) >= 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "2" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "2" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; case 4: if (sizeof((c)) >= 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "4" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "4" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; case 8: if (sizeof((c)) < 8) __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "4" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); else __asm__ __volatile__ ( ".ifnc " "%0" "," "r0" "; " ".ifnc " "%0" "r0" ",fpr11; " ".ifnc " "%0" "r0" ",r11fp; " ".ifnc " "%0" "r0" ",ipr12; " ".ifnc " "%0" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r1" "; " ".ifnc " "%3" "r1" ",fpr11; " ".ifnc " "%3" "r1" ",r11fp; " ".ifnc " "%3" "r1" ",ipr12; " ".ifnc " "%3" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __get_user_" "8" : "=&r" (__e), "=r" (__r2) : "0" (__p), "r" (__l) : "lr", "cc"); break; default: __e = __get_user_bad(); break; } uaccess_restore(__ua_flags); c = (typeof(*(end))) __r2; __e; }); });
}
(void)c;
return 0;
}
int add_to_page_cache_locked(struct page *page, struct address_space *mapping,
unsigned long index, gfp_t gfp_mask);
int add_to_page_cache_lru(struct page *page, struct address_space *mapping,
unsigned long index, gfp_t gfp_mask);
extern void delete_from_page_cache(struct page *page);
extern void __delete_from_page_cache(struct page *page, void *shadow);
int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int add_to_page_cache(struct page *page,
struct address_space *mapping, unsigned long offset, gfp_t gfp_mask)
{
int error;
__SetPageLocked(page);
error = add_to_page_cache_locked(page, mapping, offset, gfp_mask);
if (__builtin_expect(!!(error), 0))
__ClearPageLocked(page);
return error;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long dir_pages(struct inode *inode)
{
return (unsigned long)(inode->i_size + ((1UL) << 12) - 1) >>
12;
}
# 17 "./include/linux/blkdev.h" 2
# 1 "./include/linux/backing-dev-defs.h" 1
# 11 "./include/linux/backing-dev-defs.h"
# 1 "./include/linux/flex_proportions.h" 1
# 28 "./include/linux/flex_proportions.h"
struct fprop_global {
struct percpu_counter events;
unsigned int period;
seqcount_t sequence;
};
int fprop_global_init(struct fprop_global *p, gfp_t gfp);
void fprop_global_destroy(struct fprop_global *p);
bool fprop_new_period(struct fprop_global *p, int periods);
struct fprop_local_single {
unsigned long events;
unsigned int period;
raw_spinlock_t lock;
};
int fprop_local_init_single(struct fprop_local_single *pl);
void fprop_local_destroy_single(struct fprop_local_single *pl);
void __fprop_inc_single(struct fprop_global *p, struct fprop_local_single *pl);
void fprop_fraction_single(struct fprop_global *p,
struct fprop_local_single *pl, unsigned long *numerator,
unsigned long *denominator);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
void fprop_inc_single(struct fprop_global *p, struct fprop_local_single *pl)
{
unsigned long flags;
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0); } while (0);
__fprop_inc_single(p, pl);
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0); } while (0);
}
struct fprop_local_percpu {
struct percpu_counter events;
unsigned int period;
raw_spinlock_t lock;
};
int fprop_local_init_percpu(struct fprop_local_percpu *pl, gfp_t gfp);
void fprop_local_destroy_percpu(struct fprop_local_percpu *pl);
void __fprop_inc_percpu(struct fprop_global *p, struct fprop_local_percpu *pl);
void __fprop_inc_percpu_max(struct fprop_global *p, struct fprop_local_percpu *pl,
int max_frac);
void fprop_fraction_percpu(struct fprop_global *p,
struct fprop_local_percpu *pl, unsigned long *numerator,
unsigned long *denominator);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function))
void fprop_inc_percpu(struct fprop_global *p, struct fprop_local_percpu *pl)
{
unsigned long flags;
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); flags = arch_local_irq_save(); } while (0); } while (0);
__fprop_inc_percpu(p, pl);
do { do { ({ unsigned long __dummy; typeof(flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(flags); } while (0); } while (0);
}
# 12 "./include/linux/backing-dev-defs.h" 2
struct page;
struct device;
struct dentry;
enum wb_state {
WB_registered,
WB_writeback_running,
WB_has_dirty_io,
};
enum wb_congested_state {
WB_async_congested,
WB_sync_congested,
};
typedef int (congested_fn)(void *, int);
enum wb_stat_item {
WB_RECLAIMABLE,
WB_WRITEBACK,
WB_DIRTIED,
WB_WRITTEN,
NR_WB_STAT_ITEMS
};
# 53 "./include/linux/backing-dev-defs.h"
struct bdi_writeback_congested {
unsigned long state;
atomic_t refcnt;
struct backing_dev_info *__bdi;
int blkcg_id;
struct rb_node rb_node;
};
# 85 "./include/linux/backing-dev-defs.h"
struct bdi_writeback {
struct backing_dev_info *bdi;
unsigned long state;
unsigned long last_old_flush;
struct list_head b_dirty;
struct list_head b_io;
struct list_head b_more_io;
struct list_head b_dirty_time;
spinlock_t list_lock;
struct percpu_counter stat[NR_WB_STAT_ITEMS];
struct bdi_writeback_congested *congested;
unsigned long bw_time_stamp;
unsigned long dirtied_stamp;
unsigned long written_stamp;
unsigned long write_bandwidth;
unsigned long avg_write_bandwidth;
unsigned long dirty_ratelimit;
unsigned long balanced_dirty_ratelimit;
struct fprop_local_percpu completions;
int dirty_exceeded;
spinlock_t work_lock;
struct list_head work_list;
struct delayed_work dwork;
unsigned long dirty_sleep;
struct list_head bdi_node;
struct percpu_ref refcnt;
struct fprop_local_percpu memcg_completions;
struct cgroup_subsys_state *memcg_css;
struct cgroup_subsys_state *blkcg_css;
struct list_head memcg_node;
struct list_head blkcg_node;
union {
struct work_struct release_work;
struct callback_head rcu;
};
};
struct backing_dev_info {
struct list_head bdi_list;
unsigned long ra_pages;
unsigned long io_pages;
congested_fn *congested_fn;
void *congested_data;
const char *name;
struct kref refcnt;
unsigned int capabilities;
unsigned int min_ratio;
unsigned int max_ratio, max_prop_frac;
atomic_long_t tot_write_bandwidth;
struct bdi_writeback wb;
struct list_head wb_list;
struct radix_tree_root cgwb_tree;
struct rb_root cgwb_congested_tree;
struct mutex cgwb_release_mutex;
struct rw_semaphore wb_switch_rwsem;
wait_queue_head_t wb_waitq;
struct device *dev;
struct device *owner;
struct timer_list laptop_mode_wb_timer;
};
enum {
BLK_RW_ASYNC = 0,
BLK_RW_SYNC = 1,
};
void clear_wb_congested(struct bdi_writeback_congested *congested, int sync);
void set_wb_congested(struct bdi_writeback_congested *congested, int sync);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void clear_bdi_congested(struct backing_dev_info *bdi, int sync)
{
clear_wb_congested(bdi->wb.congested, sync);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_bdi_congested(struct backing_dev_info *bdi, int sync)
{
set_wb_congested(bdi->wb.congested, sync);
}
struct wb_lock_cookie {
bool locked;
unsigned long flags;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool wb_tryget(struct bdi_writeback *wb)
{
if (wb != &wb->bdi->wb)
return percpu_ref_tryget(&wb->refcnt);
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void wb_get(struct bdi_writeback *wb)
{
if (wb != &wb->bdi->wb)
percpu_ref_get(&wb->refcnt);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void wb_put(struct bdi_writeback *wb)
{
if (({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(!wb->bdi); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/backing-dev-defs.h", 237); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); })) {
return;
}
if (wb != &wb->bdi->wb)
percpu_ref_put(&wb->refcnt);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool wb_dying(struct bdi_writeback *wb)
{
return percpu_ref_is_dying(&wb->refcnt);
}
# 18 "./include/linux/blkdev.h" 2
# 1 "./include/linux/mempool.h" 1
# 11 "./include/linux/mempool.h"
struct kmem_cache;
typedef void * (mempool_alloc_t)(gfp_t gfp_mask, void *pool_data);
typedef void (mempool_free_t)(void *element, void *pool_data);
typedef struct mempool_s {
spinlock_t lock;
int min_nr;
int curr_nr;
void **elements;
void *pool_data;
mempool_alloc_t *alloc;
mempool_free_t *free;
wait_queue_head_t wait;
} mempool_t;
extern mempool_t *mempool_create(int min_nr, mempool_alloc_t *alloc_fn,
mempool_free_t *free_fn, void *pool_data);
extern mempool_t *mempool_create_node(int min_nr, mempool_alloc_t *alloc_fn,
mempool_free_t *free_fn, void *pool_data,
gfp_t gfp_mask, int nid);
extern int mempool_resize(mempool_t *pool, int new_min_nr);
extern void mempool_destroy(mempool_t *pool);
extern void *mempool_alloc(mempool_t *pool, gfp_t gfp_mask) __attribute__((__malloc__));
extern void mempool_free(void *element, mempool_t *pool);
void *mempool_alloc_slab(gfp_t gfp_mask, void *pool_data);
void mempool_free_slab(void *element, void *pool_data);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) mempool_t *
mempool_create_slab_pool(int min_nr, struct kmem_cache *kc)
{
return mempool_create(min_nr, mempool_alloc_slab, mempool_free_slab,
(void *) kc);
}
void *mempool_kmalloc(gfp_t gfp_mask, void *pool_data);
void mempool_kfree(void *element, void *pool_data);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) mempool_t *mempool_create_kmalloc_pool(int min_nr, size_t size)
{
return mempool_create(min_nr, mempool_kmalloc, mempool_kfree,
(void *) size);
}
void *mempool_alloc_pages(gfp_t gfp_mask, void *pool_data);
void mempool_free_pages(void *element, void *pool_data);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) mempool_t *mempool_create_page_pool(int min_nr, int order)
{
return mempool_create(min_nr, mempool_alloc_pages, mempool_free_pages,
(void *)(long)order);
}
# 20 "./include/linux/blkdev.h" 2
# 1 "./include/linux/bio.h" 1
# 23 "./include/linux/bio.h"
# 1 "./include/linux/ioprio.h" 1
# 1 "./include/linux/iocontext.h" 1
enum {
ICQ_EXITED = 1 << 2,
ICQ_DESTROYED = 1 << 3,
};
# 73 "./include/linux/iocontext.h"
struct io_cq {
struct request_queue *q;
struct io_context *ioc;
union {
struct list_head q_node;
struct kmem_cache *__rcu_icq_cache;
};
union {
struct hlist_node ioc_node;
struct callback_head __rcu_head;
};
unsigned int flags;
};
struct io_context {
atomic_long_t refcount;
atomic_t active_ref;
atomic_t nr_tasks;
spinlock_t lock;
unsigned short ioprio;
int nr_batch_requests;
unsigned long last_waited;
struct radix_tree_root icq_tree;
struct io_cq *icq_hint;
struct hlist_head icq_list;
struct work_struct release_work;
};
# 130 "./include/linux/iocontext.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void get_io_context_active(struct io_context *ioc)
{
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(atomic_long_read(&ioc->refcount) <= 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/iocontext.h", 132); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(({ union { typeof((&ioc->active_ref)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&ioc->active_ref)->counter), __u.__c, sizeof((&ioc->active_ref)->counter)); else __read_once_size_nocheck(&((&ioc->active_ref)->counter), __u.__c, sizeof((&ioc->active_ref)->counter)); do { } while (0); __u.__val; }) <= 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/iocontext.h", 133); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
atomic_long_inc(&ioc->refcount);
atomic_add(1, &ioc->active_ref);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ioc_task_link(struct io_context *ioc)
{
get_io_context_active(ioc);
({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(({ union { typeof((&ioc->nr_tasks)->counter) __val; char __c[1]; } __u; if (1) __read_once_size(&((&ioc->nr_tasks)->counter), __u.__c, sizeof((&ioc->nr_tasks)->counter)); else __read_once_size_nocheck(&((&ioc->nr_tasks)->counter), __u.__c, sizeof((&ioc->nr_tasks)->counter)); do { } while (0); __u.__val; }) <= 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_null("include/linux/iocontext.h", 142); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); });
atomic_add(1, &ioc->nr_tasks);
}
struct task_struct;
void put_io_context(struct io_context *ioc);
void put_io_context_active(struct io_context *ioc);
void exit_io_context(struct task_struct *task);
struct io_context *get_task_io_context(struct task_struct *task,
gfp_t gfp_flags, int node);
# 7 "./include/linux/ioprio.h" 2
# 26 "./include/linux/ioprio.h"
enum {
IOPRIO_CLASS_NONE,
IOPRIO_CLASS_RT,
IOPRIO_CLASS_BE,
IOPRIO_CLASS_IDLE,
};
enum {
IOPRIO_WHO_PROCESS = 1,
IOPRIO_WHO_PGRP,
IOPRIO_WHO_USER,
};
# 53 "./include/linux/ioprio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int task_nice_ioprio(struct task_struct *task)
{
return (task_nice(task) + 20) / 5;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int task_nice_ioclass(struct task_struct *task)
{
if (task->policy == 5)
return IOPRIO_CLASS_IDLE;
else if (task->policy == 1 || task->policy == 2)
return IOPRIO_CLASS_RT;
else
return IOPRIO_CLASS_BE;
}
extern int ioprio_best(unsigned short aprio, unsigned short bprio);
extern int set_task_ioprio(struct task_struct *task, int ioprio);
# 24 "./include/linux/bio.h" 2
# 1 "./include/linux/bio-crypt-ctx.h" 1
enum blk_crypto_mode_num {
BLK_ENCRYPTION_MODE_INVALID,
BLK_ENCRYPTION_MODE_AES_256_XTS,
BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV,
BLK_ENCRYPTION_MODE_ADIANTUM,
BLK_ENCRYPTION_MODE_MAX,
};
# 1 "./include/linux/blk_types.h" 1
# 10 "./include/linux/blk_types.h"
# 1 "./include/linux/bvec.h" 1
# 30 "./include/linux/bvec.h"
struct bio_vec {
struct page *bv_page;
unsigned int bv_len;
unsigned int bv_offset;
};
struct bvec_iter {
sector_t bi_sector;
unsigned int bi_size;
unsigned int bi_idx;
unsigned int bi_done;
unsigned int bi_bvec_done;
};
# 72 "./include/linux/bvec.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bvec_iter_advance(const struct bio_vec *bv,
struct bvec_iter *iter, unsigned bytes)
{
if (({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(bytes > iter->bi_size); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_fmt("include/linux/bvec.h", 76, "Attempted to advance past end of bvec iter\n"); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); })) {
iter->bi_size = 0;
return false;
}
while (bytes) {
unsigned iter_len = ({ typeof((*iter).bi_size) __UNIQUE_ID_min1_63 = ((*iter).bi_size); typeof((&((bv))[((*iter)).bi_idx])->bv_len - (*iter).bi_bvec_done) __UNIQUE_ID_min2_64 = ((&((bv))[((*iter)).bi_idx])->bv_len - (*iter).bi_bvec_done); (void) (&__UNIQUE_ID_min1_63 == &__UNIQUE_ID_min2_64); __UNIQUE_ID_min1_63 < __UNIQUE_ID_min2_64 ? __UNIQUE_ID_min1_63 : __UNIQUE_ID_min2_64; });
unsigned len = ({ typeof(bytes) __UNIQUE_ID_min1_65 = (bytes); typeof(iter_len) __UNIQUE_ID_min2_66 = (iter_len); (void) (&__UNIQUE_ID_min1_65 == &__UNIQUE_ID_min2_66); __UNIQUE_ID_min1_65 < __UNIQUE_ID_min2_66 ? __UNIQUE_ID_min1_65 : __UNIQUE_ID_min2_66; });
bytes -= len;
iter->bi_size -= len;
iter->bi_bvec_done += len;
iter->bi_done += len;
if (iter->bi_bvec_done == (&(bv)[(*iter).bi_idx])->bv_len) {
iter->bi_bvec_done = 0;
iter->bi_idx++;
}
}
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bvec_iter_rewind(const struct bio_vec *bv,
struct bvec_iter *iter,
unsigned int bytes)
{
while (bytes) {
unsigned len = ({ typeof(bytes) __UNIQUE_ID_min1_67 = (bytes); typeof(iter->bi_bvec_done) __UNIQUE_ID_min2_68 = (iter->bi_bvec_done); (void) (&__UNIQUE_ID_min1_67 == &__UNIQUE_ID_min2_68); __UNIQUE_ID_min1_67 < __UNIQUE_ID_min2_68 ? __UNIQUE_ID_min1_67 : __UNIQUE_ID_min2_68; });
if (iter->bi_bvec_done == 0) {
if (({ static bool __attribute__ ((__section__(".data.unlikely"))) __warned; int __ret_warn_once = !!(iter->bi_idx == 0); if (__builtin_expect(!!(__ret_warn_once && !__warned), 0)) { __warned = true; ({ int __ret_warn_on = !!(1); if (__builtin_expect(!!(__ret_warn_on), 0)) warn_slowpath_fmt("include/linux/bvec.h", 108, "Attempted to rewind iter beyond " "bvec's boundaries\n"); __builtin_expect(!!(__ret_warn_on), 0); }); } __builtin_expect(!!(__ret_warn_once), 0); })) {
return false;
}
iter->bi_idx--;
iter->bi_bvec_done = (&(bv)[(*iter).bi_idx])->bv_len;
continue;
}
bytes -= len;
iter->bi_size += len;
iter->bi_bvec_done -= len;
}
return true;
}
# 11 "./include/linux/blk_types.h" 2
struct bio_set;
struct bio;
struct bio_integrity_payload;
struct page;
struct block_device;
struct io_context;
struct cgroup_subsys_state;
typedef void (bio_end_io_t) (struct bio *);
struct bio_crypt_ctx;
# 29 "./include/linux/blk_types.h"
typedef u8 blk_status_t;
# 48 "./include/linux/blk_types.h"
struct blk_issue_stat {
u64 stat;
};
struct bio {
struct bio *bi_next;
struct gendisk *bi_disk;
unsigned int bi_opf;
unsigned short bi_flags;
unsigned short bi_ioprio;
unsigned short bi_write_hint;
blk_status_t bi_status;
u8 bi_partno;
unsigned int bi_phys_segments;
unsigned int bi_seg_front_size;
unsigned int bi_seg_back_size;
struct bvec_iter bi_iter;
atomic_t __bi_remaining;
bio_end_io_t *bi_end_io;
void *bi_private;
struct io_context *bi_ioc;
struct cgroup_subsys_state *bi_css;
struct bio_crypt_ctx *bi_crypt_context;
bool bi_skip_dm_default_key;
union {
};
unsigned short bi_vcnt;
unsigned short bi_max_vecs;
atomic_t __bi_cnt;
struct bio_vec *bi_io_vec;
struct bio_set *bi_pool;
struct bio_vec bi_inline_vecs[0];
};
# 197 "./include/linux/blk_types.h"
enum req_opf {
REQ_OP_READ = 0,
REQ_OP_WRITE = 1,
REQ_OP_FLUSH = 2,
REQ_OP_DISCARD = 3,
REQ_OP_ZONE_REPORT = 4,
REQ_OP_SECURE_ERASE = 5,
REQ_OP_ZONE_RESET = 6,
REQ_OP_WRITE_SAME = 7,
REQ_OP_WRITE_ZEROES = 9,
REQ_OP_SCSI_IN = 32,
REQ_OP_SCSI_OUT = 33,
REQ_OP_DRV_IN = 34,
REQ_OP_DRV_OUT = 35,
REQ_OP_LAST,
};
enum req_flag_bits {
__REQ_FAILFAST_DEV =
8,
__REQ_FAILFAST_TRANSPORT,
__REQ_FAILFAST_DRIVER,
__REQ_SYNC,
__REQ_META,
__REQ_PRIO,
__REQ_NOMERGE,
__REQ_IDLE,
__REQ_INTEGRITY,
__REQ_FUA,
__REQ_PREFLUSH,
__REQ_RAHEAD,
__REQ_BACKGROUND,
__REQ_NOUNMAP,
__REQ_NOWAIT,
__REQ_NR_BITS,
};
# 279 "./include/linux/blk_types.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_set_op_attrs(struct bio *bio, unsigned op,
unsigned op_flags)
{
bio->bi_opf = op | op_flags;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool op_is_write(unsigned int op)
{
return (op & 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool op_is_flush(unsigned int op)
{
return op & ((1ULL << __REQ_FUA) | (1ULL << __REQ_PREFLUSH));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool op_is_sync(unsigned int op)
{
return (op & ((1 << 8) - 1)) == REQ_OP_READ ||
(op & ((1ULL << __REQ_SYNC) | (1ULL << __REQ_FUA) | (1ULL << __REQ_PREFLUSH)));
}
typedef unsigned int blk_qc_t;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_qc_t_valid(blk_qc_t cookie)
{
return cookie != -1U;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) blk_qc_t blk_tag_to_qc_t(unsigned int tag, unsigned int queue_num,
bool internal)
{
blk_qc_t ret = tag | (queue_num << 16);
if (internal)
ret |= (1U << 31);
return ret;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_qc_t_to_queue_num(blk_qc_t cookie)
{
return (cookie & ~(1U << 31)) >> 16;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_qc_t_to_tag(blk_qc_t cookie)
{
return cookie & ((1u << 16) - 1);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_qc_t_is_internal(blk_qc_t cookie)
{
return (cookie & (1U << 31)) != 0;
}
struct blk_rq_stat {
s64 mean;
u64 min;
u64 max;
s32 nr_samples;
s32 nr_batch;
u64 batch;
};
# 18 "./include/linux/bio-crypt-ctx.h" 2
# 41 "./include/linux/bio-crypt-ctx.h"
struct blk_crypto_key {
enum blk_crypto_mode_num crypto_mode;
unsigned int data_unit_size;
unsigned int data_unit_size_bits;
unsigned int size;
unsigned int hash;
bool is_hw_wrapped;
u8 raw[128];
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void
blk_crypto_key_set_hash_and_dun_bytes(struct blk_crypto_key *key,
u32 hash, unsigned int dun_bytes)
{
key->hash = (dun_bytes << 24) |
(hash & 0xffffff);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32
blk_crypto_key_hash(const struct blk_crypto_key *key)
{
return key->hash & 0xffffff;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int
blk_crypto_key_dun_bytes(const struct blk_crypto_key *key)
{
return key->hash >> 24;
}
# 95 "./include/linux/bio-crypt-ctx.h"
struct bio_crypt_ctx {
const struct blk_crypto_key *bc_key;
int bc_keyslot;
u64 bc_dun[(32/sizeof(u64))];
struct keyslot_manager *bc_ksm;
};
int bio_crypt_ctx_init(void);
struct bio_crypt_ctx *bio_crypt_alloc_ctx(gfp_t gfp_mask);
void bio_crypt_free_ctx(struct bio *bio);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_has_crypt_ctx(struct bio *bio)
{
return bio->bi_crypt_context;
}
void bio_crypt_clone(struct bio *dst, struct bio *src, gfp_t gfp_mask);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_crypt_set_ctx(struct bio *bio,
const struct blk_crypto_key *key,
u64 dun[(32/sizeof(u64))],
gfp_t gfp_mask)
{
struct bio_crypt_ctx *bc = bio_crypt_alloc_ctx(gfp_mask);
bc->bc_key = key;
memcpy(bc->bc_dun, dun, sizeof(bc->bc_dun));
bc->bc_ksm = ((void *)0);
bc->bc_keyslot = -1;
bio->bi_crypt_context = bc;
}
void bio_crypt_ctx_release_keyslot(struct bio_crypt_ctx *bc);
int bio_crypt_ctx_acquire_keyslot(struct bio_crypt_ctx *bc,
struct keyslot_manager *ksm);
struct request;
bool bio_crypt_should_process(struct request *rq);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_crypt_dun_is_contiguous(const struct bio_crypt_ctx *bc,
unsigned int bytes,
u64 next_dun[(32/sizeof(u64))])
{
int i = 0;
unsigned int inc = bytes >> bc->bc_key->data_unit_size_bits;
while (i < (32/sizeof(u64))) {
if (bc->bc_dun[i] + inc != next_dun[i])
return false;
inc = ((bc->bc_dun[i] + inc) < inc);
i++;
}
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_crypt_dun_increment(u64 dun[(32/sizeof(u64))],
unsigned int inc)
{
int i = 0;
while (inc && i < (32/sizeof(u64))) {
dun[i] += inc;
inc = (dun[i] < inc);
i++;
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_crypt_advance(struct bio *bio, unsigned int bytes)
{
struct bio_crypt_ctx *bc = bio->bi_crypt_context;
if (!bc)
return;
bio_crypt_dun_increment(bc->bc_dun,
bytes >> bc->bc_key->data_unit_size_bits);
}
bool bio_crypt_ctx_compatible(struct bio *b_1, struct bio *b_2);
bool bio_crypt_ctx_mergeable(struct bio *b_1, unsigned int b1_bytes,
struct bio *b_2);
# 224 "./include/linux/bio-crypt-ctx.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_set_skip_dm_default_key(struct bio *bio)
{
bio->bi_skip_dm_default_key = true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_should_skip_dm_default_key(const struct bio *bio)
{
return bio->bi_skip_dm_default_key;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_clone_skip_dm_default_key(struct bio *dst,
const struct bio *src)
{
dst->bi_skip_dm_default_key = src->bi_skip_dm_default_key;
}
# 26 "./include/linux/bio.h" 2
# 1 "./arch/arm/include/asm/io.h" 1
# 30 "./arch/arm/include/asm/io.h"
# 1 "./include/asm-generic/pci_iomap.h" 1
# 14 "./include/asm-generic/pci_iomap.h"
struct pci_dev;
# 36 "./include/asm-generic/pci_iomap.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *pci_iomap(struct pci_dev *dev, int bar, unsigned long max)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *pci_iomap_wc(struct pci_dev *dev, int bar, unsigned long max)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *pci_iomap_range(struct pci_dev *dev, int bar,
unsigned long offset,
unsigned long maxlen)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *pci_iomap_wc_range(struct pci_dev *dev, int bar,
unsigned long offset,
unsigned long maxlen)
{
return ((void *)0);
}
# 31 "./arch/arm/include/asm/io.h" 2
# 1 "./include/xen/xen.h" 1
enum xen_domain_type {
XEN_NATIVE,
XEN_PV_DOMAIN,
XEN_HVM_DOMAIN,
};
# 32 "./arch/arm/include/asm/io.h" 2
# 43 "./arch/arm/include/asm/io.h"
extern void atomic_io_modify(void *reg, u32 mask, u32 set);
extern void atomic_io_modify_relaxed(void *reg, u32 mask, u32 set);
void __raw_writesb(volatile void *addr, const void *data, int bytelen);
void __raw_writesw(volatile void *addr, const void *data, int wordlen);
void __raw_writesl(volatile void *addr, const void *data, int longlen);
void __raw_readsb(const volatile void *addr, void *data, int bytelen);
void __raw_readsw(const volatile void *addr, void *data, int wordlen);
void __raw_readsl(const volatile void *addr, void *data, int longlen);
# 73 "./arch/arm/include/asm/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_writew(u16 val, volatile void *addr)
{
asm volatile("strh %1, %0"
: : "Q" (*(volatile u16 *)addr), "r" (val));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u16 __raw_readw(const volatile void *addr)
{
u16 val;
asm volatile("ldrh %0, %1"
: "=r" (val)
: "Q" (*(volatile u16 *)addr));
return val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_writeb(u8 val, volatile void *addr)
{
asm volatile("strb %1, %0"
: : "Qo" (*(volatile u8 *)addr), "r" (val));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __raw_writel(u32 val, volatile void *addr)
{
asm volatile("str %1, %0"
: : "Qo" (*(volatile u32 *)addr), "r" (val));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u8 __raw_readb(const volatile void *addr)
{
u8 val;
asm volatile("ldrb %0, %1"
: "=r" (val)
: "Qo" (*(volatile u8 *)addr));
return val;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 __raw_readl(const volatile void *addr)
{
u32 val;
asm volatile("ldr %0, %1"
: "=r" (val)
: "Qo" (*(volatile u32 *)addr));
return val;
}
# 142 "./arch/arm/include/asm/io.h"
extern void *__arm_ioremap_caller(phys_addr_t, size_t, unsigned int,
void *);
extern void *__arm_ioremap_pfn(unsigned long, unsigned long, size_t, unsigned int);
extern void *__arm_ioremap_exec(phys_addr_t, size_t, bool cached);
extern void __iounmap(volatile void *addr);
extern void * (*arch_ioremap_caller)(phys_addr_t, size_t,
unsigned int, void *);
extern void (*arch_iounmap)(volatile void *);
extern void __readwrite_bug(const char *fn);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *__typesafe_io(unsigned long addr)
{
return (void *)addr;
}
# 184 "./arch/arm/include/asm/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void pci_ioremap_set_mem_type(int mem_type) {}
extern int pci_ioremap_io(unsigned int offset, phys_addr_t phys_addr);
# 198 "./arch/arm/include/asm/io.h"
void *pci_remap_cfgspace(resource_size_t res_cookie, size_t size);
# 1 "arch/arm/mach-rtd2851a/include/mach/io.h" 1
# 14 "arch/arm/mach-rtd2851a/include/mach/io.h"
# 1 "arch/arm/mach-rtd2851a/include/mach/iomap.h" 1
# 15 "arch/arm/mach-rtd2851a/include/mach/io.h" 2
# 1 "./arch/arm/include/asm/io.h" 1
# 16 "arch/arm/mach-rtd2851a/include/mach/io.h" 2
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned char rtd_inb(unsigned long addr)
{
if (addr < (0xB8000000 +0x00200000) && addr >= 0xB8000000)
return __raw_readb((void *)(addr-0xB8000000 +0xFE000000));
else
return __raw_readb((void *)addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned short rtd_inw(unsigned long addr)
{
if (addr < (0xB8000000 +0x00200000) && addr >= 0xB8000000)
return __raw_readw((void *)(addr-0xB8000000 +0xFE000000));
else
return __raw_readw((void *)addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int rtd_inl(unsigned long addr)
{
if (addr < (0xB8000000 +0x00200000) && addr >= 0xB8000000)
return __raw_readl((void *)(addr-0xB8000000 +0xFE000000));
else
return __raw_readl((void *)addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rtd_outb(unsigned long addr, unsigned char val)
{
if (addr < (0xB8000000 +0x00200000) && addr >= 0xB8000000)
__raw_writeb(val,(void *)(addr-0xB8000000 +0xFE000000));
else
__raw_writeb(val,(void *)addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rtd_outw(unsigned long addr, unsigned short val)
{
if (addr < (0xB8000000 +0x00200000) && addr >= 0xB8000000)
__raw_writew(val,(void *)(addr-0xB8000000 +0xFE000000));
else
__raw_writew(val,(void *)addr);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void rtd_outl(unsigned long addr, unsigned int val)
{
if (addr < (0xB8000000 +0x00200000) && addr >= 0xB8000000)
__raw_writel(val,(void *)(addr-0xB8000000 +0xFE000000));
else
__raw_writel(val,(void *)addr);
}
# 1 "arch/arm/mach-rtd2851a/include/mach/system.h" 1
# 13 "arch/arm/mach-rtd2851a/include/mach/system.h"
# 1 "./arch/arm/include/asm/io.h" 1
# 14 "arch/arm/mach-rtd2851a/include/mach/system.h" 2
# 1 "arch/arm/mach-rtd2851a/include/rbus/sys_reg_reg.h" 1
# 15 "arch/arm/mach-rtd2851a/include/rbus/sys_reg_reg.h"
# 1 "arch/arm/mach-rtd2851a/include/rbus/rbus_types.h" 1
# 16 "arch/arm/mach-rtd2851a/include/rbus/sys_reg_reg.h" 2
# 2969 "arch/arm/mach-rtd2851a/include/rbus/sys_reg_reg.h"
typedef union
{
unsigned int regValue;
struct{
unsigned int rstn_diseqc:1;
unsigned int rstn_cablerx_nouse:1;
unsigned int rstn_hdr1:1;
unsigned int rstn_hdr2:1;
unsigned int rstn_nnip:1;
unsigned int rstn_dscd_nouse:1;
unsigned int rstn_earc:1;
unsigned int rstn_atb_demod:1;
unsigned int rstn_atb_demod_wrap:1;
unsigned int rstn_hdic_apb:1;
unsigned int rstn_hdic_axi:1;
unsigned int rstn_hdic:1;
unsigned int rstn_dtv_frontend:1;
unsigned int rstn_kcpu_nouse:1;
unsigned int rstn_vde2_nouse:1;
unsigned int rstn_ve2_p2_nouse:1;
unsigned int rstn_ve2_nouse:1;
unsigned int rstn_vcpu2_nouse:1;
unsigned int rstn_rtk_demod:1;
unsigned int rstn_acpu:1;
unsigned int rstn_acpu2_nouse:1;
unsigned int rstn_ae2_nouse:1;
unsigned int rstn_ve_lgcy_p2:1;
unsigned int rstn_ve_lgcy:1;
unsigned int rstn_scpu_wrap:1;
unsigned int rstn_vde:1;
unsigned int rstn_ve_p2:1;
unsigned int rstn_ve:1;
unsigned int rstn_vcpu:1;
unsigned int rstn_ae:1;
unsigned int rstn_scpu:1;
unsigned int write_data:1;
};
}sys_reg_sys_srst0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int rstn_tvsb1:1;
unsigned int rstn_tvsb2:1;
unsigned int rstn_tvsb3:1;
unsigned int rstn_tvsb4:1;
unsigned int rstn_tvsb5:1;
unsigned int rstn_tve_nouse:1;
unsigned int rstn_dcphy:1;
unsigned int rstn_dcu:1;
unsigned int rstn_if_demod:1;
unsigned int rstn_vdec:1;
unsigned int rstn_vdec2:1;
unsigned int rstn_vbis0:1;
unsigned int res1:1;
unsigned int res2:1;
unsigned int res3:1;
unsigned int rstn_audio:1;
unsigned int rstn_ifadc:1;
unsigned int rstn_cbus_nouse:1;
unsigned int rstn_offms:1;
unsigned int rstn_me:1;
unsigned int rstn_apll_adc:1;
unsigned int rstn_hdmi:1;
unsigned int rstn_dispd:1;
unsigned int rstn_tp:1;
unsigned int rstn_cp:1;
unsigned int rstn_md:1;
unsigned int rstn_se:1;
unsigned int rstn_ve_blk_wrap:1;
unsigned int rstn_3d_gde:1;
unsigned int rstn_vodma1:1;
unsigned int rstn_dtv_demod_mb:1;
unsigned int write_data:1;
};
}sys_reg_sys_srst1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int rstn_vodma2:1;
unsigned int rstn_usb2_dwc:1;
unsigned int rstn_usb2_ex_dwc_nouse:1;
unsigned int rstn_tm:1;
unsigned int rstn_atvin2:1;
unsigned int rstn_atvin:1;
unsigned int rstn_dispm_cap:1;
unsigned int rstn_dispm_disp:1;
unsigned int rstn_emmc:1;
unsigned int rstn_if_demod_logic:1;
unsigned int rstn_bistreg:1;
unsigned int rstn_i2c4_nouse:1;
unsigned int rstn_pllreg:1;
unsigned int rstn_usb3_nouse:1;
unsigned int rstn_usb2_wrapper:1;
unsigned int rstn_pcmcia:1;
unsigned int rstn_hdr:1;
unsigned int rstn_i2c3_nouse:1;
unsigned int rstn_irtx_nouse:1;
unsigned int rstn_usb2_ex_wrapper_nouse:1;
unsigned int rstn_dtv_demod:1;
unsigned int rstn_misc:1;
unsigned int rstn_pwm:1;
unsigned int rstn_uart1:1;
unsigned int rstn_uart2_nouse:1;
unsigned int dummy18000108_6:1;
unsigned int rstn_gpio:1;
unsigned int rstn_i2c1:1;
unsigned int rstn_i2c2:1;
unsigned int rstn_nf:1;
unsigned int rstn_sc:1;
unsigned int write_data:1;
};
}sys_reg_sys_srst2_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int rstn_memc_nouse:1;
unsigned int rstn_memc_me_nouse:1;
unsigned int rstn_memc_rbus_nouse:1;
unsigned int rstn_lzma:1;
unsigned int rstn_memc_mc_nouse:1;
unsigned int res1:1;
unsigned int rstn_dcphy2_nouse:1;
unsigned int rstn_dcu2_nouse:1;
unsigned int rstn_dcphy_mc:1;
unsigned int rstn_disp_lg_hcic_nouse:1;
unsigned int rstn_disp_lg_mplus:1;
unsigned int rstn_d_boe_rgbw_pxl_nouse:1;
unsigned int rstn_d_boe_rgbw_apb_nouse:1;
unsigned int rstn_d_boe_rgbw_mcu_nouse:1;
unsigned int rstn_ddc_nouse:1;
unsigned int rstn_sd:1;
unsigned int rstn_siliconworks_rgbw_nouse:1;
unsigned int rstn_disp_lg_pod:1;
unsigned int rstn_dispi3:1;
unsigned int rstn_dispi2:1;
unsigned int rstn_dispi1:1;
unsigned int rstn_dispi_di:1;
unsigned int rstn_disp_lg_a10_nouse:1;
unsigned int res2:1;
unsigned int rstn_usb2_phy_p3_nouse:1;
unsigned int rstn_usb2_phy_p2:1;
unsigned int rstn_usb2_phy_p1:1;
unsigned int rstn_usb2_phy_p0:1;
unsigned int res3:1;
unsigned int rstn_usb3_phy_p0_nouse:1;
unsigned int rstn_usb3_mdio_p0_nouse:1;
unsigned int write_data:1;
};
}sys_reg_sys_srst3_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int clken_diseqc:1;
unsigned int clken_cablerx_nouse:1;
unsigned int clken_hdr1:1;
unsigned int clken_hdr2:1;
unsigned int clken_nnip:1;
unsigned int clken_dscd_nouse:1;
unsigned int clken_earc:1;
unsigned int clken_atb_demod:1;
unsigned int clken_atb_demod_wrap:1;
unsigned int clken_dtv_demod_mb:1;
unsigned int res1:1;
unsigned int clken_hdic:1;
unsigned int clken_dtv_frontend:1;
unsigned int clken_kcpu_nouse:1;
unsigned int res2:1;
unsigned int clken_ve2_p2_nouse:1;
unsigned int clken_ve2_nouse:1;
unsigned int clken_vcpu2_nouse:1;
unsigned int clken_rtk_demod:1;
unsigned int clken_acpu:1;
unsigned int clken_acpu2_nouse:1;
unsigned int res3:1;
unsigned int res4:1;
unsigned int clken_ve_blk_wrap:1;
unsigned int clken_ve_lgcy_p2:1;
unsigned int clken_ve_lgcy:1;
unsigned int clken_ve_p2:1;
unsigned int clken_ve:1;
unsigned int clken_vcpu:1;
unsigned int clken_scpu_wrap:1;
unsigned int clken_scpu:1;
unsigned int write_data:1;
};
}sys_reg_sys_clken0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int clken_tvsb1:1;
unsigned int clken_tvsb2:1;
unsigned int clken_tvsb3:1;
unsigned int clken_tvsb4:1;
unsigned int clken_tvsb5:1;
unsigned int clken_tve_nouse:1;
unsigned int clken_dcphy:1;
unsigned int clken_dcu:1;
unsigned int clken_if_demod:1;
unsigned int clken_vdec:1;
unsigned int clken_se2_nouse:1;
unsigned int clken_vbis0:1;
unsigned int clken_hdr1_comp:1;
unsigned int clken_hdr1_dm:1;
unsigned int clken_audio_pre512fs:1;
unsigned int clken_audio:1;
unsigned int clken_ifadc:1;
unsigned int clken_cbus_nouse:1;
unsigned int clken_offms:1;
unsigned int clken_me:1;
unsigned int clken_apll_adc:1;
unsigned int clken_hdmi:1;
unsigned int clken_dispd:1;
unsigned int clken_tp:1;
unsigned int clken_cp:1;
unsigned int clken_md:1;
unsigned int clken_se:1;
unsigned int clken_tp_div:1;
unsigned int clken_3d_gde:1;
unsigned int clken_vodma1:1;
unsigned int dummy18000114_1:1;
unsigned int write_data:1;
};
}sys_reg_sys_clken1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int clken_vodma2:1;
unsigned int clken_usb2_dwc:1;
unsigned int clken_usb2_ex_dwc_nouse:1;
unsigned int clken_tm:1;
unsigned int clken_atvin_vd:1;
unsigned int clken_atvin_ifd:1;
unsigned int clken_dispm_cap:1;
unsigned int clken_dispm_disp:1;
unsigned int clken_emmc:1;
unsigned int clken_efuse:1;
unsigned int clken_bistreg:1;
unsigned int clken_i2c4_nouse:1;
unsigned int clken_pllreg:1;
unsigned int clken_usb3_nouse:1;
unsigned int clken_usb2_wrapper:1;
unsigned int clken_pcmcia:1;
unsigned int clken_audio2_512fs:1;
unsigned int clken_i2c3_nouse:1;
unsigned int clken_irtx_nouse:1;
unsigned int clken_usb2_ex_wrapper_nouse:1;
unsigned int clken_dtv_demod:1;
unsigned int clken_misc:1;
unsigned int clken_pwm:1;
unsigned int clken_uart1:1;
unsigned int clken_uart2_nouse:1;
unsigned int clken_audio3_512fs:1;
unsigned int clken_gpio:1;
unsigned int clken_i2c1:1;
unsigned int clken_i2c2:1;
unsigned int clken_nf:1;
unsigned int clken_sc:1;
unsigned int write_data:1;
};
}sys_reg_sys_clken2_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int clken_memc_nouse:1;
unsigned int clken_memc_me_nouse:1;
unsigned int clken_memc_rbus_nouse:1;
unsigned int clken_lzma:1;
unsigned int clken_memc_mc_nouse:1;
unsigned int clken_dcphy_mc_mck:1;
unsigned int clken_dcphy2_nouse:1;
unsigned int clken_dcu2_nouse:1;
unsigned int clken_dcphy_mc:1;
unsigned int clken_disp_lg_hcic_nouse:1;
unsigned int clken_disp_lg_mplus:1;
unsigned int clken_d_boe_rgbw_pxl_nouse:1;
unsigned int clken_d_boe_rgbw_apb_nouse:1;
unsigned int clken_d_boe_rgbw_mcu_nouse:1;
unsigned int clken_ddc_nouse:1;
unsigned int clken_sd:1;
unsigned int clken_dss:1;
unsigned int clken_disp_lg_pod:1;
unsigned int clken_dispi:1;
unsigned int res1:3;
unsigned int clken_disp_lg_a10_nouse:1;
unsigned int clken_audio_arc_dma_r_512fs:1;
unsigned int clken_audio2_dma_r_512fs:1;
unsigned int clken_audio_dma_r_512fs:1;
unsigned int res2:1;
unsigned int res3:1;
unsigned int res4:1;
unsigned int res5:1;
unsigned int res6:1;
unsigned int write_data:1;
};
}sys_reg_sys_clken3_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:30;
unsigned int rstn_gdcmp:1;
unsigned int write_data:1;
};
}sys_reg_sys_srst4_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:30;
unsigned int clken_gdcmp:1;
unsigned int write_data:1;
};
}sys_reg_sys_clken4_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:25;
unsigned int ejtag_demod_sel_nouse:1;
unsigned int ejtag_demod_en_nouse:1;
unsigned int ejtag_mode:1;
unsigned int res2:1;
unsigned int ejtag_gpu_en:1;
unsigned int ejtag_gpu_sel_nouse:1;
unsigned int boot_cpu_clksel:1;
};
}sys_reg_sys_cpusel_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int pll_bypass_ref_clk_mode:1;
unsigned int mc_clk_sel_nouse:1;
unsigned int res1:1;
unsigned int ve_sel:1;
unsigned int sd_emmc_clk_sel:1;
unsigned int pcr108_clksel_nouse:1;
unsigned int dummy18000204_25:1;
unsigned int dpll_apr27m_sel:1;
unsigned int dpll_pcr27m_clksel:1;
unsigned int sd_clk_div:3;
unsigned int ve_rif_use_p2clk_nouse:1;
unsigned int ve2_rif_use_p2clk_nouse:1;
unsigned int atvin_clk_sel:1;
unsigned int atvin_vd_clk_sel:1;
unsigned int atvin_vdadc_tve_clk_sel:1;
unsigned int res2:1;
unsigned int tve_108m_clk_sel_nouse:1;
unsigned int cbus_clksel_nouse:1;
unsigned int hdmi_xtal_sel_nouse:1;
unsigned int syncp_xtal_sel_nouse:1;
unsigned int arm_xtal_sel_nouse:1;
unsigned int timer_xtal_sel_nouse:1;
unsigned int res3:1;
unsigned int hdmirx_2p_clk_sel_nouse:1;
unsigned int lzma_clk_sel:2;
unsigned int vdec_x27_clksel:1;
unsigned int vbis0_yppsel:1;
unsigned int inv_vbis0_sel:1;
unsigned int uart_clksel:1;
};
}sys_reg_sys_clksel_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dispd_osd_div_nouse:2;
unsigned int me_clk_bush_sel_nouse:1;
unsigned int res1:3;
unsigned int dclk_fract_b:6;
unsigned int dclk_fract_a:5;
unsigned int dispd_gdma_clk_sel:1;
unsigned int res2:1;
unsigned int res3:1;
unsigned int dummy18000208_11:1;
unsigned int dummy18000208_10:1;
unsigned int clken_disp_lg_avcom_nouse:1;
unsigned int dummy18000208_8:1;
unsigned int dummy18000208_7:1;
unsigned int res4:1;
unsigned int clken_disp_memc_nouse:1;
unsigned int res5:1;
unsigned int clken_disp_gdma:1;
unsigned int clken_disp_stage1:1;
unsigned int clken_disp_osd:1;
unsigned int dummy18000208_0:1;
};
}sys_reg_sys_dispclksel_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:15;
unsigned int clken_ext_ddr_en:1;
unsigned int clken_ext_15_en_nouse:1;
unsigned int clken_ext_14_en_nouse:1;
unsigned int clken_ext_13_en_nouse:1;
unsigned int clken_ext_12_en_nouse:1;
unsigned int clken_ext_11_en_nouse:1;
unsigned int clken_ext_10_en_nouse:1;
unsigned int clken_ext_9_en:1;
unsigned int clken_ext_8_en:1;
unsigned int clken_ext_7_en:1;
unsigned int clken_ext_6_en:1;
unsigned int clken_ext_5_en:1;
unsigned int clken_ext_4_en:1;
unsigned int clken_ext_3_en:1;
unsigned int clken_ext_2_en:1;
unsigned int clken_ext_1_en:1;
unsigned int clken_ext_0_en:1;
};
}sys_reg_sys_extclk_en_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int pwm7_clksel_nouse:1;
unsigned int pwm6_clksel_nouse:1;
unsigned int pwm5_clksel_nouse:1;
unsigned int pwm4_clksel_nouse:1;
unsigned int pwm3_clksel_nouse:1;
unsigned int pwm2_clksel_nouse:1;
unsigned int pwm1_clksel_nouse:1;
unsigned int pwm0_clksel:1;
unsigned int res1:6;
unsigned int dtv_demod_atb_clk_sel:1;
unsigned int i2c4_clksel_nouse:1;
unsigned int i2c3_clksel_nouse:1;
unsigned int dtv_demod_sel:1;
unsigned int i2c2_clksel:1;
unsigned int i2c1_clksel:1;
unsigned int dtv_demod_sel_msb:1;
unsigned int nf_clksel:3;
unsigned int pwm_pll_clksel:1;
unsigned int aud_dtv2_freq_sel_nouse:3;
unsigned int dummy18000214_3:1;
unsigned int aud_dtv_freq_sel_nouse:3;
};
}sys_reg_sys_clkdiv_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:15;
unsigned int res2:1;
unsigned int res3:4;
unsigned int res4:12;
};
}sys_reg_sys_arm_wd_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dummy1800021c_31_29:3;
unsigned int ib_rst_gck_memc_nouse:1;
unsigned int ib_rst_gck_gpu_nouse:1;
unsigned int ib_rst_gck_se2_nouse:1;
unsigned int ib_rst_gck_ve2_nouse:1;
unsigned int ib_rst_gck_ve_nouse:1;
unsigned int ib_rst_gck_tvsb5_nouse:1;
unsigned int ib_rst_gck_tvsb4_nouse:1;
unsigned int ib_rst_gck_tvsb3_nouse:1;
unsigned int ib_rst_gck_tvsb2_nouse:1;
unsigned int ib_rst_gck_tvsb1_nouse:1;
unsigned int ib_rst_gck_sb3_nouse:1;
unsigned int ib_rst_gck_sb2_nouse:1;
unsigned int ib_rst_gck_sb1_nouse:1;
unsigned int dummy1800021c_15_13:3;
unsigned int ib_clk_gck_memc_nouse:1;
unsigned int ib_clk_gck_gpu_nouse:1;
unsigned int ib_clk_gck_se2_nouse:1;
unsigned int ib_clk_gck_ve2_nouse:1;
unsigned int ib_clk_gck_ve_nouse:1;
unsigned int ib_clk_gck_tvsb5_nouse:1;
unsigned int ib_clk_gck_tvsb4_nouse:1;
unsigned int ib_clk_gck_tvsb3_nouse:1;
unsigned int ib_clk_gck_tvsb2_nouse:1;
unsigned int ib_clk_gck_tvsb1_nouse:1;
unsigned int ib_clk_gck_sb3_nouse:1;
unsigned int ib_clk_gck_sb2_nouse:1;
unsigned int ib_clk_gck_sb1_nouse:1;
};
}sys_reg_sys_ib_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int clkratio_acpu2_nouse:5;
unsigned int res2:2;
unsigned int clkratio_acpu2_active_nouse:1;
};
}sys_reg_sys_dyn_sw_acpu2_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:8;
unsigned int clkratio_vcpu:5;
unsigned int res2:2;
unsigned int clkratio_vcpu_active:1;
unsigned int res3:16;
};
}sys_reg_sys_dyn_sw_vcpu_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:16;
unsigned int clkratio_acpu:5;
unsigned int res2:2;
unsigned int clkratio_acpu_active:1;
unsigned int res3:8;
};
}sys_reg_sys_dyn_sw_acpu_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:9;
unsigned int dclk_osd_f2p_gate_sel_nouse:3;
unsigned int res2:1;
unsigned int dclk_osd_f1p_gate_sel:3;
unsigned int res3:1;
unsigned int dclk_s2_f2p_gate_sel_nouse:3;
unsigned int res4:1;
unsigned int dclk_s2_f1p_gate_sel:3;
unsigned int res5:1;
unsigned int dclk_s1_f2p_gate_sel_nouse:3;
unsigned int res6:1;
unsigned int dclk_s1_f1p_gate_sel:3;
};
}sys_reg_sys_dclk_gate_sel0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:5;
unsigned int dclk_s2_f1p_div2_gate_sel:3;
unsigned int res2:1;
unsigned int dclk_s2_f2p_div2_gate_sel_nouse:3;
unsigned int res3:1;
unsigned int dclk_mc_gate_sel_nouse:3;
unsigned int res4:1;
unsigned int dclk_mc_ippre2x_gate_sel_nouse:3;
unsigned int res5:1;
unsigned int dclk_mc_ippre_gate_sel_nouse:3;
unsigned int res6:1;
unsigned int dclk_memc_out_if_gate_sel_nouse:3;
unsigned int res7:1;
unsigned int dclk_memc_in_if_gate_sel_nouse:3;
};
}sys_reg_sys_dclk_gate_sel1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:31;
unsigned int efuse_ready:1;
};
}sys_reg_efuse_ctrl0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:5;
unsigned int res2:1;
unsigned int res3:1;
unsigned int res4:1;
unsigned int res5:1;
unsigned int res6:1;
unsigned int res7:1;
unsigned int res8:1;
unsigned int res9:1;
unsigned int res10:1;
unsigned int res11:1;
unsigned int res12:1;
unsigned int res13:3;
unsigned int atv_iso_en:1;
unsigned int res14:1;
unsigned int atv_str_status:1;
unsigned int res15:1;
unsigned int atv_str_pow:1;
unsigned int res16:3;
unsigned int dtv_iso_en:1;
unsigned int res17:1;
unsigned int dtv_str_status:1;
unsigned int res18:1;
unsigned int dtv_str_pow:1;
};
}sys_reg_power_ctrl0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int smartcard_int_scpu_routing_en_nouse:1;
unsigned int nnip_int_scpu_routing_en:1;
unsigned int ddc_int_scpu_routing_en:1;
unsigned int dctl_int_2_scpu_routing_en:1;
unsigned int standby_hdmi_clk_det_int_scpu_routing_en_nouse:1;
unsigned int standby_wdog_int_scpu_routing_en:1;
unsigned int standby_iso_misc_off_int_scpu_routing_en:1;
unsigned int standby_cbus_int_scpu_routing_en_nouse:1;
unsigned int standby_iso_misc_int_scpu_routing_en:1;
unsigned int standby_cec_int_scpu_routing_en:1;
unsigned int standby_rtc_int_scpu_routing_en:1;
unsigned int gpu_int_scpu_routing_en:1;
unsigned int usb3_int_scpu_routing_en_nouse:1;
unsigned int usb2_int_scpu_routing_en:1;
unsigned int etn_int_scpu_routing_en:1;
unsigned int sd_int_scpu_routing_en:1;
unsigned int audio_dma_irq_scpu_routing_en:1;
unsigned int audio_irq_scpu_routing_en:1;
unsigned int tv_sb_dc_int_scpu_routing_en:1;
unsigned int dcphy_int_scpu_routing_en:1;
unsigned int dc_int_scpu_routing_en:1;
unsigned int tve_int_scpu_routing_en_nouse:1;
unsigned int osd_int_scpu_routing_en:1;
unsigned int ultra_zoom_int_scpu_routing_en:1;
unsigned int dctl_int_scpu_routing_en:1;
unsigned int vbi_int_scpu_routing_en:1;
unsigned int vdec_int_scpu_routing_en:1;
unsigned int if_demod_int_scpu_routing_en:1;
unsigned int dispm_int_scpu_routing_en:1;
unsigned int dispi_int_scpu_routing_en:1;
unsigned int apll_dds_int_scpu_routing_en:1;
unsigned int write_data:1;
};
}sys_reg_int_ctrl_scpu_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dummy18000294_31:1;
unsigned int disp2tve_int_vcpu_routing_en_nouse:1;
unsigned int vodma_int_vcpu_routing_en:1;
unsigned int dctl_int_2_vcpu_routing_en:1;
unsigned int dc_int_vcpu2_routing_en_nouse:1;
unsigned int dummy18000294_26:1;
unsigned int dummy18000294_25:1;
unsigned int dummy18000294_24:1;
unsigned int dummy18000294_23:1;
unsigned int dummy18000294_22:1;
unsigned int dummy18000294_21:1;
unsigned int dummy18000294_20:1;
unsigned int dummy18000294_19:1;
unsigned int dummy18000294_18:1;
unsigned int dummy18000294_17:1;
unsigned int dummy18000294_16:1;
unsigned int me_int_vcpu_routing_en:1;
unsigned int ddc_int_vcpu_routing_en:1;
unsigned int tv_sb_dc_int_vcpu_routing_en:1;
unsigned int dcphy_int_vcpu_routing_en:1;
unsigned int dc_int_vcpu_routing_en:1;
unsigned int tve_int_vcpu_routing_en_nouse:1;
unsigned int osd_int_vcpu_routing_en:1;
unsigned int ultra_zoom_int_vcpu_routing_en:1;
unsigned int dctl_int_vcpu_routing_en:1;
unsigned int vbi_int_vcpu_routing_en:1;
unsigned int vdec_int_vcpu_routing_en:1;
unsigned int if_demod_int_vcpu_routing_en:1;
unsigned int dispm_int_vcpu_routing_en:1;
unsigned int dispi_int_vcpu_routing_en:1;
unsigned int apll_dds_int_vcpu_routing_en:1;
unsigned int write_data:1;
};
}sys_reg_int_ctrl_vcpu_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int earc_cmdc_int_acpu_routing_en:1;
unsigned int res1:1;
unsigned int res2:1;
unsigned int res3:1;
unsigned int res4:1;
unsigned int res5:1;
unsigned int res6:1;
unsigned int res7:1;
unsigned int res8:1;
unsigned int res9:1;
unsigned int res10:1;
unsigned int res11:1;
unsigned int res12:1;
unsigned int res13:1;
unsigned int res14:1;
unsigned int me_int_acpu_routing_en:1;
unsigned int audio_dma_irq_acpu_routing_en:1;
unsigned int audio_irq_acpu_routing_en:1;
unsigned int tv_sb_dc_int_acpu_routing_en:1;
unsigned int dcphy_int_acpu_routing_en:1;
unsigned int dc_int_acpu_routing_en:1;
unsigned int dummy18000298_10:1;
unsigned int dummy18000298_9:1;
unsigned int dummy18000298_8:1;
unsigned int audio_dma_irq_acpu2_routing_en_nouse:1;
unsigned int audio_irq_acpu2_routing_en_nouse:1;
unsigned int dummy18000298_5:1;
unsigned int dummy18000298_4:1;
unsigned int dc_int_acpu2_routing_en_nouse:1;
unsigned int dummy18000298_2:1;
unsigned int dummy18000298_1:1;
unsigned int write_data:1;
};
}sys_reg_int_ctrl_acpu_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int smartcard_int_kcpu_routing_en_nouse:1;
unsigned int disp2tve_int_kcpu_routing_en_nouse:1;
unsigned int vodma_int_kcpu_routing_en:1;
unsigned int dctl_int_2_kcpu_routing_en:1;
unsigned int standby_hdmi_clk_det_int_kcpu_routing_en_nouse:1;
unsigned int standby_wdog_int_kcpu_routing_en:1;
unsigned int standby_iso_misc_off_int_kcpu_routing_en:1;
unsigned int standby_cbus_int_kcpu_routing_en_nouse:1;
unsigned int standby_iso_misc_int_kcpu_routing_en:1;
unsigned int standby_cec_int_kcpu_routing_en:1;
unsigned int standby_rtc_int_kcpu_routing_en:1;
unsigned int gpu_int_kcpu_routing_en:1;
unsigned int usb3_int_kcpu_routing_en_nouse:1;
unsigned int usb2_int_kcpu_routing_en:1;
unsigned int etn_int_kcpu_routing_en:1;
unsigned int sd_int_kcpu_routing_en:1;
unsigned int dummy1800029c_15:1;
unsigned int ddc_int_kcpu_routing_en:1;
unsigned int tv_sb_dc_int_kcpu_routing_en:1;
unsigned int dcphy_int_kcpu_routing_en:1;
unsigned int dc_int_kcpu_routing_en:1;
unsigned int tve_int_kcpu_routing_en_nouse:1;
unsigned int osd_int_kcpu_routing_en:1;
unsigned int ultra_zoom_int_kcpu_routing_en:1;
unsigned int dctl_int_kcpu_routing_en:1;
unsigned int vbi_int_kcpu_routing_en:1;
unsigned int vdec_int_kcpu_routing_en:1;
unsigned int if_demod_int_kcpu_routing_en:1;
unsigned int dispm_int_kcpu_routing_en:1;
unsigned int dispi_int_kcpu_routing_en:1;
unsigned int apll_dds_int_kcpu_routing_en:1;
unsigned int write_data:1;
};
}sys_reg_int_ctrl_kcpu_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:1;
unsigned int res2:1;
unsigned int res3:1;
unsigned int memc_int_vcpu_routing_en_nouse:1;
unsigned int dummy180002a0_27:1;
unsigned int dummy180002a0_26:1;
unsigned int dummy180002a0_25:1;
unsigned int memc_int_vcpu2_routing_en_nouse:1;
unsigned int dummy180002a0_23:1;
unsigned int dummy180002a0_22:1;
unsigned int dummy180002a0_21:1;
unsigned int memc_int_scpu_routing_en_nouse:1;
unsigned int hdmi_airq_acpu_routing_en:1;
unsigned int hdmi_airq_scpu_routing_en:1;
unsigned int hdmi_irq_vcpu_routing_en:1;
unsigned int hdmi_irq_scpu_routing_en:1;
unsigned int res4:15;
unsigned int write_data:1;
};
}sys_reg_int_ctrl_memc_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int earc_cmdc_int_scpu_routing_en:1;
unsigned int nnip_security_int_scpu_routing_en:1;
unsigned int res1:13;
unsigned int mc_security_int_scpu_routing_en:1;
unsigned int usb2_ex_int_scpu_routing_en_nouse:1;
unsigned int dc_security_int_scpu_routing_en:1;
unsigned int res2:1;
unsigned int gpio_security_int_scpu_routing_en:1;
unsigned int vodma_int_scpu_routing_en:1;
unsigned int tm_security_int_scpu_routing_en:1;
unsigned int tp_security_int_scpu_routing_en:1;
unsigned int nf_security_int_scpu_routing_en:1;
unsigned int emmc_security_int_scpu_routing_en:1;
unsigned int md_security_int_scpu_routing_en:1;
unsigned int mcp_security_int_scpu_routing_en:1;
unsigned int sb2_security_int_scpu_routing_en:1;
unsigned int mis_security_int_scpu_routing_en:1;
unsigned int se_security_int_scpu_routing_en:1;
unsigned int dc2_security_int_scpu_routing_en_nouse:1;
unsigned int write_data:1;
};
}sys_reg_int_ctrl_scpu_2_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:15;
unsigned int mc_security_int_kcpu_routing_en:1;
unsigned int usb2_ex_int_kcpu_routing_en_nouse:1;
unsigned int dc_security_int_kcpu_routing_en:1;
unsigned int res2:3;
unsigned int tm_security_int_kcpu_routing_en:1;
unsigned int tp_security_int_kcpu_routing_en:1;
unsigned int nf_security_int_kcpu_routing_en:1;
unsigned int emmc_security_int_kcpu_routing_en:1;
unsigned int md_security_int_kcpu_routing_en:1;
unsigned int mcp_security_int_kcpu_routing_en:1;
unsigned int sb2_security_int_kcpu_routing_en:1;
unsigned int mis_security_int_kcpu_routing_en:1;
unsigned int se_security_int_kcpu_routing_en:1;
unsigned int dc2_security_int_kcpu_routing_en_nouse:1;
unsigned int write_data:1;
};
}sys_reg_int_ctrl_kcpu_2_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int write_en8:1;
unsigned int timer5_routing_mux_sel:3;
unsigned int write_en7:1;
unsigned int timer3_routing_mux_sel:3;
unsigned int write_en6:1;
unsigned int timer11_routing_mux_sel:3;
unsigned int write_en5:1;
unsigned int timer10_routing_mux_sel:3;
unsigned int write_en4:1;
unsigned int timer9_routing_mux_sel:3;
unsigned int write_en3:1;
unsigned int timer8_routing_mux_sel:3;
unsigned int write_en2:1;
unsigned int timer7_routing_mux_sel:3;
unsigned int write_en1:1;
unsigned int timer6_routing_mux_sel:3;
};
}sys_reg_int_ctrl_timer6_11_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:28;
unsigned int write_en1:1;
unsigned int timer4_routing_mux_sel:3;
};
}sys_reg_int_ctrl_timer4_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_main_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_main_ro_sel:3;
unsigned int dss_main_wire_sel:1;
unsigned int res2:3;
unsigned int dss_main_ready:1;
unsigned int dss_main_count_out:20;
};
}sys_reg_ss_main_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_main_speed_en:1;
unsigned int dss_main_wsort_go:1;
unsigned int dss_main_data_in:20;
};
}sys_reg_ss_main_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_demod_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_demod_ro_sel:3;
unsigned int dss_demod_wire_sel:1;
unsigned int res2:3;
unsigned int dss_demod_ready:1;
unsigned int dss_demod_count_out:20;
};
}sys_reg_ss_demod_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_demod_speed_en:1;
unsigned int dss_demod_wsort_go:1;
unsigned int dss_demod_data_in:20;
};
}sys_reg_ss_demod_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_tvsb1_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_tvsb1_ro_sel:3;
unsigned int dss_tvsb1_wire_sel:1;
unsigned int res2:3;
unsigned int dss_tvsb1_ready:1;
unsigned int dss_tvsb1_count_out:20;
};
}sys_reg_ss_tvsb1_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_tvsb1_speed_en:1;
unsigned int dss_tvsb1_wsort_go:1;
unsigned int dss_tvsb1_data_in:20;
};
}sys_reg_ss_tvsb1_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_hdmirx_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_hdmirx_ro_sel:3;
unsigned int dss_hdmirx_wire_sel:1;
unsigned int res2:3;
unsigned int dss_hdmirx_ready:1;
unsigned int dss_hdmirx_count_out:20;
};
}sys_reg_ss_hdmirx_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_hdmirx_speed_en:1;
unsigned int dss_hdmirx_wsort_go:1;
unsigned int dss_hdmirx_data_in:20;
};
}sys_reg_ss_hdmirx_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_nnip_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_nnip_ro_sel:3;
unsigned int dss_nnip_wire_sel:1;
unsigned int res2:3;
unsigned int dss_nnip_ready:1;
unsigned int dss_nnip_count_out:20;
};
}sys_reg_ss_nnip_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_nnip_speed_en:1;
unsigned int dss_nnip_wsort_go:1;
unsigned int dss_nnip_data_in:20;
};
}sys_reg_ss_nnip_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_vcpuve_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_vcpuve_ro_sel:3;
unsigned int dss_vcpuve_wire_sel:1;
unsigned int res2:3;
unsigned int dss_vcpuve_ready:1;
unsigned int dss_vcpuve_count_out:20;
};
}sys_reg_ss_vcpuve_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_vcpuve_speed_en:1;
unsigned int dss_vcpuve_wsort_go:1;
unsigned int dss_vcpuve_data_in:20;
};
}sys_reg_ss_vcpuve_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_dispd_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_dispd_ro_sel:3;
unsigned int dss_dispd_wire_sel:1;
unsigned int res2:3;
unsigned int dss_dispd_ready:1;
unsigned int dss_dispd_count_out:20;
};
}sys_reg_ss_acpu_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_dispd_speed_en:1;
unsigned int dss_dispd_wsort_go:1;
unsigned int dss_dispd_data_in:20;
};
}sys_reg_ss_acpu_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:16;
unsigned int res2:16;
};
}sys_reg_ss_acpu_rhvt_2_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dss_tvsb2_dss_rst_n:1;
unsigned int res1:3;
unsigned int dss_tvsb2_ro_sel:3;
unsigned int dss_tvsb2_wire_sel:1;
unsigned int res2:3;
unsigned int dss_tvsb2_ready:1;
unsigned int dss_tvsb2_count_out:20;
};
}sys_reg_ss_tvsb2_rhvt_0_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:10;
unsigned int dss_tvsb2_speed_en:1;
unsigned int dss_tvsb2_wsort_go:1;
unsigned int dss_tvsb2_data_in:20;
};
}sys_reg_ss_tvsb2_rhvt_1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:16;
unsigned int res2:16;
};
}sys_reg_ss_tvsb2_rhvt_2_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:20;
unsigned int demod_voltage_strobe_en:1;
unsigned int package_voltage_strobe_en:1;
unsigned int vcpu_ve_voltage_strobe_en:1;
unsigned int vcpu_voltage_strobe_en_nouse:1;
unsigned int res2:2;
unsigned int tvsb1_voltage_strobe_en:1;
unsigned int main_voltage_strobe_en:1;
unsigned int pllemmc_voltage_strobe_en:1;
unsigned int pllscpu_voltage_strobe_en:1;
unsigned int pllbus_voltage_strobe_en:1;
unsigned int tvsb2_voltage_strobe_en:1;
};
}sys_reg_voltage_strobe_ctrl_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:23;
unsigned int usb3_voltage_strobe_en:1;
unsigned int usb2_y1_voltage_strobe_en_nouse:1;
unsigned int usb2_y0_voltage_strobe_en_nouse:1;
unsigned int usb2_x1_voltage_strobe_en:1;
unsigned int usb2_x0_voltage_strobe_en:1;
unsigned int hdmirx2p1_voltage_strobe_en_nouse:1;
unsigned int hdmirx_voltage_strobe_en:1;
unsigned int pif_voltage_strobe_en:1;
unsigned int etn_voltage_strobe_en_nouse:1;
};
}sys_reg_voltage_strobe_ctrl_analog_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:2;
unsigned int usb3_tdf_dbg_en_nouse:1;
unsigned int pll_dbg_en:1;
unsigned int res2:9;
unsigned int dbg0_div_sel_en:1;
unsigned int dbg0_div_sel:2;
unsigned int res3:4;
unsigned int top_ssc_dbg_sel:4;
unsigned int crt_dbg_sel:8;
};
}sys_reg_sc_debug_0_RBUS;
# 15 "arch/arm/mach-rtd2851a/include/mach/system.h" 2
# 1 "arch/arm/mach-rtd2851a/include/mach/iomap.h" 1
# 16 "arch/arm/mach-rtd2851a/include/mach/system.h" 2
# 1 "arch/arm/mach-rtd2851a/include/mach/serial.h" 1
# 17 "arch/arm/mach-rtd2851a/include/mach/system.h" 2
# 68 "arch/arm/mach-rtd2851a/include/mach/io.h" 2
# 1 "arch/arm/mach-rtd2851a/include/rbus/misc_reg.h" 1
# 215 "arch/arm/mach-rtd2851a/include/rbus/misc_reg.h"
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:4;
unsigned int i2c1_int:1;
unsigned int i2c2_int:1;
unsigned int res2:1;
unsigned int sc0_int:1;
unsigned int res3:1;
unsigned int res4:1;
unsigned int pcmcia_int:1;
unsigned int res5:1;
unsigned int gpio_int:1;
unsigned int res6:3;
unsigned int res7:1;
unsigned int tc7_int:1;
unsigned int tc6_int:1;
unsigned int tc5_int:1;
unsigned int tc4_int:1;
unsigned int tc3_int:1;
unsigned int tc2_int:1;
unsigned int tc1_int:1;
unsigned int tc0_int:1;
unsigned int ur1_int:1;
unsigned int res8:1;
unsigned int res9:1;
unsigned int tc11_int:1;
unsigned int tc10_int:1;
unsigned int tc9_int:1;
unsigned int tc8_int:1;
};
}misc_isr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:26;
unsigned int sel0:4;
unsigned int res2:1;
unsigned int enable:1;
};
}misc_dbg_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:27;
unsigned int sc_od_mode:1;
unsigned int res2:1;
unsigned int uart_od_mode_en:1;
unsigned int res3:1;
unsigned int res4:1;
};
}misc_dummy_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int rvd6:3;
unsigned int i2c1_int_sel:1;
unsigned int i2c2_int_sel:1;
unsigned int res1:2;
unsigned int sc0_int_sel:1;
unsigned int res2:1;
unsigned int rvd5:1;
unsigned int pcmcia_int_sel:1;
unsigned int rvd4:1;
unsigned int gpio_int_sel:1;
unsigned int rvd3:3;
unsigned int res3:1;
unsigned int tc7_int_sel:1;
unsigned int tc6_int_sel:1;
unsigned int tc5_int_sel:1;
unsigned int tc3_int_sel:1;
unsigned int rvd2:4;
unsigned int ur1_int_sel:1;
unsigned int res4:1;
unsigned int rvd1:5;
};
}misc_isr_switch_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dummy2:32;
};
}misc_dummy2_RBUS;
# 70 "arch/arm/mach-rtd2851a/include/mach/io.h" 2
# 1 "arch/arm/mach-rtd2851a/include/rbus/misc_uart_reg.h" 1
# 326 "arch/arm/mach-rtd2851a/include/rbus/misc_uart_reg.h"
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int dll:8;
};
}misc_uart_u1rbr_thr_dll_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int dlh:8;
};
}misc_uart_u1ier_dlh_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int fifose:2;
unsigned int res2:2;
unsigned int iid:4;
};
}misc_uart_u1iir_fcr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int dlab:1;
unsigned int brk:1;
unsigned int res2:1;
unsigned int eps:1;
unsigned int pen:1;
unsigned int stb:1;
unsigned int wls:2;
};
}misc_uart_u1lcr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:27;
unsigned int loop:1;
unsigned int res2:2;
unsigned int rts:1;
unsigned int dtr:1;
};
}misc_uart_u1mcr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int rfe:1;
unsigned int temt:1;
unsigned int thre:1;
unsigned int bi:1;
unsigned int fe:1;
unsigned int pe:1;
unsigned int oe:1;
unsigned int dr:1;
};
}misc_uart_u1lsr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int dcd:1;
unsigned int ri:1;
unsigned int dsr:1;
unsigned int cts:1;
unsigned int ddcd:1;
unsigned int teri:1;
unsigned int ddsr:1;
unsigned int dcts:1;
};
}misc_uart_u1msr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:24;
unsigned int scr:8;
};
}misc_uart_u1scr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:25;
unsigned int dmaifen:1;
unsigned int txempthr:6;
};
}misc_uart_ur1_ctrl_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:27;
unsigned int tx_wrd_count:5;
};
}misc_uart_ur1_sts_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:29;
unsigned int ur_length_err:1;
unsigned int ur_tx_thd:1;
unsigned int write_data:1;
};
}misc_uart_dma_int_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:30;
unsigned int ur_length_err_en:1;
unsigned int ur_tx_thd_en:1;
};
}misc_uart_dma_int_en_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:1;
unsigned int tx_base:28;
unsigned int res2:3;
};
}misc_uart_dma_txbase_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:1;
unsigned int tx_limit:28;
unsigned int res2:3;
};
}misc_uart_dma_txlimit_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:1;
unsigned int tx_wrptr:31;
};
}misc_uart_dma_txwrptr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:1;
unsigned int tx_rdptr:31;
};
}misc_uart_dma_txrdptr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int tx_thd:32;
};
}misc_uart_dma_txthd_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:31;
unsigned int gating_en:1;
};
}misc_uart_dma_ctrl_RBUS;
# 71 "arch/arm/mach-rtd2851a/include/mach/io.h" 2
# 1 "arch/arm/mach-rtd2851a/include/rbus/wdog_reg.h" 1
# 125 "arch/arm/mach-rtd2851a/include/rbus/wdog_reg.h"
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:23;
unsigned int wdog_int:1;
unsigned int res2:8;
};
}wdog_msk_isr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int im_wdog_rst:1;
unsigned int res1:10;
unsigned int upper_bound_sel:1;
unsigned int res2:1;
unsigned int wd_rst_src:2;
unsigned int wdie:1;
unsigned int nmic:4;
unsigned int wdc:4;
unsigned int wden:8;
};
}wdog_tcwcr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:31;
unsigned int wdclr:1;
};
}wdog_tcwtr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int sel:32;
};
}wdog_tcwnmi_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int sel:32;
};
}wdog_tcwov_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:31;
unsigned int te:1;
};
}wdog_tcwtrg_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int wd_cnt:32;
};
}wdog_tcwsts_RBUS;
# 72 "arch/arm/mach-rtd2851a/include/mach/io.h" 2
# 1 "arch/arm/mach-rtd2851a/include/rbus/iso_misc_off_reg.h" 1
# 111 "arch/arm/mach-rtd2851a/include/rbus/iso_misc_off_reg.h"
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:8;
unsigned int res2:1;
unsigned int res3:1;
unsigned int res4:16;
unsigned int res5:1;
unsigned int wdog_ov_int:1;
unsigned int irda1_int:1;
unsigned int ur0_int:1;
unsigned int i2c0_int:1;
unsigned int res6:1;
};
}iso_misc_off_isr_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:3;
unsigned int i2c0_scpu_and_8051_int_sel:1;
unsigned int res2:1;
unsigned int i2c0_scpu_and_8051_int_en:1;
unsigned int res3:6;
unsigned int res4:3;
unsigned int irdarxdetec:1;
unsigned int res5:1;
unsigned int uart_od_mode_en:1;
unsigned int res6:10;
unsigned int dbg_sel:4;
};
}iso_misc_off_system_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int res1:30;
unsigned int uart0_int_to_emcu_en:1;
unsigned int uart0_int_to_scpu_en:1;
};
}iso_misc_off_int_ctrl_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dummy1:32;
};
}iso_misc_off_dummy1_RBUS;
typedef union
{
unsigned int regValue;
struct{
unsigned int dummy2:32;
};
}iso_misc_off_dummy2_RBUS;
# 73 "arch/arm/mach-rtd2851a/include/mach/io.h" 2
# 204 "./arch/arm/include/asm/io.h" 2
# 281 "./arch/arm/include/asm/io.h"
extern void _memcpy_fromio(void *, const volatile void *, size_t);
extern void _memcpy_toio(volatile void *, const void *, size_t);
extern void _memset_io(volatile void *, int, size_t);
# 325 "./arch/arm/include/asm/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void memset_io(volatile void *dst, unsigned c,
size_t count)
{
extern void mmioset(void *, unsigned int, size_t);
mmioset((void *)dst, c, count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void memcpy_fromio(void *to, const volatile void *from,
size_t count)
{
extern void mmiocpy(void *, const void *, size_t);
mmiocpy(to, (const void *)from, count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void memcpy_toio(volatile void *to, const void *from,
size_t count)
{
extern void mmiocpy(void *, const void *, size_t);
mmiocpy((void *)to, from, count);
}
# 400 "./arch/arm/include/asm/io.h"
void *ioremap(resource_size_t res_cookie, size_t size);
void *ioremap_cache(resource_size_t res_cookie, size_t size);
void *ioremap_cached(resource_size_t res_cookie, size_t size);
void *ioremap_wc(resource_size_t res_cookie, size_t size);
void iounmap(volatile void *iomem_cookie);
void *arch_memremap_wb(phys_addr_t phys_addr, size_t size);
# 437 "./arch/arm/include/asm/io.h"
extern void *ioport_map(unsigned long port, unsigned int nr);
extern void ioport_unmap(void *addr);
struct pci_dev;
extern void pci_iounmap(struct pci_dev *dev, void *addr);
# 460 "./arch/arm/include/asm/io.h"
# 1 "./include/asm-generic/io.h" 1
# 410 "./include/asm-generic/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u8 inb_p(unsigned long addr)
{
return ({ __u8 __v = __raw_readb(__typesafe_io((addr) & ((resource_size_t)0))); __asm__ __volatile__ ("dsb " "" : : : "memory"); __v; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u16 inw_p(unsigned long addr)
{
return ({ __u16 __v = (( __u16)(__le16)(( __le16) __raw_readw(__typesafe_io((addr) & ((resource_size_t)0))))); __asm__ __volatile__ ("dsb " "" : : : "memory"); __v; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 inl_p(unsigned long addr)
{
return ({ __u32 __v = (( __u32)(__le32)(( __le32) __raw_readl(__typesafe_io((addr) & ((resource_size_t)0))))); __asm__ __volatile__ ("dsb " "" : : : "memory"); __v; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outb_p(u8 value, unsigned long addr)
{
({ __asm__ __volatile__ ("dsb " "st" : : : "memory"); __raw_writeb(value,__typesafe_io((addr) & ((resource_size_t)0))); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outw_p(u16 value, unsigned long addr)
{
({ __asm__ __volatile__ ("dsb " "st" : : : "memory"); __raw_writew(( __u16) (( __le16)(__u16)(value)),__typesafe_io((addr) & ((resource_size_t)0))); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outl_p(u32 value, unsigned long addr)
{
({ __asm__ __volatile__ ("dsb " "st" : : : "memory"); __raw_writel(( __u32) (( __le32)(__u32)(value)),__typesafe_io((addr) & ((resource_size_t)0))); });
}
# 514 "./include/asm-generic/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void insb_p(unsigned long addr, void *buffer, unsigned int count)
{
__raw_readsb(__typesafe_io((addr) & ((resource_size_t)0)),buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void insw_p(unsigned long addr, void *buffer, unsigned int count)
{
__raw_readsw(__typesafe_io((addr) & ((resource_size_t)0)),buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void insl_p(unsigned long addr, void *buffer, unsigned int count)
{
__raw_readsl(__typesafe_io((addr) & ((resource_size_t)0)),buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outsb_p(unsigned long addr, const void *buffer,
unsigned int count)
{
__raw_writesb(__typesafe_io((addr) & ((resource_size_t)0)),buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outsw_p(unsigned long addr, const void *buffer,
unsigned int count)
{
__raw_writesw(__typesafe_io((addr) & ((resource_size_t)0)),buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void outsl_p(unsigned long addr, const void *buffer,
unsigned int count)
{
__raw_writesl(__typesafe_io((addr) & ((resource_size_t)0)),buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u8 ioread8(const volatile void *addr)
{
return ({ u8 __v = ({ u8 __r = __raw_readb(addr); __r; }); __asm__ __volatile__ ("dsb " "" : : : "memory"); __v; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u16 ioread16(const volatile void *addr)
{
return ({ u16 __v = ({ u16 __r = (( __u16)(__le16)(( __le16) __raw_readw(addr))); __r; }); __asm__ __volatile__ ("dsb " "" : : : "memory"); __v; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) u32 ioread32(const volatile void *addr)
{
return ({ u32 __v = ({ u32 __r = (( __u32)(__le32)(( __le32) __raw_readl(addr))); __r; }); __asm__ __volatile__ ("dsb " "" : : : "memory"); __v; });
}
# 600 "./include/asm-generic/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void iowrite8(u8 value, volatile void *addr)
{
({ __asm__ __volatile__ ("dsb " "st" : : : "memory"); __raw_writeb(value,addr); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void iowrite16(u16 value, volatile void *addr)
{
({ __asm__ __volatile__ ("dsb " "st" : : : "memory"); __raw_writew(( u16) (( __le16)(__u16)(value)),addr); });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void iowrite32(u32 value, volatile void *addr)
{
({ __asm__ __volatile__ ("dsb " "st" : : : "memory"); __raw_writel(( u32) (( __le32)(__u32)(value)),addr); });
}
# 686 "./include/asm-generic/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ioread8_rep(const volatile void *addr, void *buffer,
unsigned int count)
{
__raw_readsb(addr,buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ioread16_rep(const volatile void *addr,
void *buffer, unsigned int count)
{
__raw_readsw(addr,buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void ioread32_rep(const volatile void *addr,
void *buffer, unsigned int count)
{
__raw_readsl(addr,buffer,count);
}
# 724 "./include/asm-generic/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void iowrite8_rep(volatile void *addr,
const void *buffer,
unsigned int count)
{
__raw_writesb(addr,buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void iowrite16_rep(volatile void *addr,
const void *buffer,
unsigned int count)
{
__raw_writesw(addr,buffer,count);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void iowrite32_rep(volatile void *addr,
const void *buffer,
unsigned int count)
{
__raw_writesl(addr,buffer,count);
}
# 771 "./include/asm-generic/io.h"
struct pci_dev;
extern void *pci_iomap(struct pci_dev *dev, int bar, unsigned long max);
# 823 "./include/asm-generic/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *ioremap_uc(phys_addr_t offset, size_t size)
{
return ((void *)0);
}
# 939 "./include/asm-generic/io.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
{
}
# 461 "./arch/arm/include/asm/io.h" 2
struct bio_vec;
extern bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
const struct bio_vec *vec2);
extern int valid_phys_addr_range(phys_addr_t addr, size_t size);
extern int valid_mmap_phys_addr_range(unsigned long pfn, size_t size);
extern int devmem_is_allowed(unsigned long pfn);
extern void register_isa_ports(unsigned int mmio, unsigned int io,
unsigned int io_shift);
# 30 "./include/linux/bio.h" 2
# 83 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_has_data(struct bio *bio)
{
if (bio &&
bio->bi_iter.bi_size &&
((bio)->bi_opf & ((1 << 8) - 1)) != REQ_OP_DISCARD &&
((bio)->bi_opf & ((1 << 8) - 1)) != REQ_OP_SECURE_ERASE &&
((bio)->bi_opf & ((1 << 8) - 1)) != REQ_OP_WRITE_ZEROES)
return true;
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_no_advance_iter(struct bio *bio)
{
return ((bio)->bi_opf & ((1 << 8) - 1)) == REQ_OP_DISCARD ||
((bio)->bi_opf & ((1 << 8) - 1)) == REQ_OP_SECURE_ERASE ||
((bio)->bi_opf & ((1 << 8) - 1)) == REQ_OP_WRITE_SAME ||
((bio)->bi_opf & ((1 << 8) - 1)) == REQ_OP_WRITE_ZEROES;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_mergeable(struct bio *bio)
{
if (bio->bi_opf & ((1ULL << __REQ_NOMERGE) | (1ULL << __REQ_PREFLUSH) | (1ULL << __REQ_FUA)))
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int bio_cur_bytes(struct bio *bio)
{
if (bio_has_data(bio))
return ((struct bio_vec) { .bv_page = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_page), .bv_len = ({ typeof(((((bio)->bi_iter))).bi_size) __UNIQUE_ID_min1_69 = (((((bio)->bi_iter))).bi_size); typeof((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done) __UNIQUE_ID_min2_70 = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done); (void) (&__UNIQUE_ID_min1_69 == &__UNIQUE_ID_min2_70); __UNIQUE_ID_min1_69 < __UNIQUE_ID_min2_70 ? __UNIQUE_ID_min1_69 : __UNIQUE_ID_min2_70; }), .bv_offset = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_offset + ((((bio)->bi_iter))).bi_bvec_done), }).bv_len;
else
return bio->bi_iter.bi_size;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *bio_data(struct bio *bio)
{
if (bio_has_data(bio))
return page_address(((&((((bio))->bi_io_vec))[((((bio)->bi_iter))).bi_idx])->bv_page)) + ((&((((bio))->bi_io_vec))[((((bio)->bi_iter))).bi_idx])->bv_offset + (((bio)->bi_iter)).bi_bvec_done);
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_full(struct bio *bio)
{
return bio->bi_vcnt >= bio->bi_max_vecs;
}
# 177 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_advance_iter(struct bio *bio, struct bvec_iter *iter,
unsigned bytes)
{
iter->bi_sector += bytes >> 9;
if (bio_no_advance_iter(bio)) {
iter->bi_size -= bytes;
iter->bi_done += bytes;
} else {
bvec_iter_advance(bio->bi_io_vec, iter, bytes);
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
unsigned int bytes)
{
iter->bi_sector -= bytes >> 9;
if (bio_no_advance_iter(bio)) {
iter->bi_size += bytes;
iter->bi_done -= bytes;
return true;
}
return bvec_iter_rewind(bio->bi_io_vec, iter, bytes);
}
# 216 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned bio_segments(struct bio *bio)
{
unsigned segs = 0;
struct bio_vec bv;
struct bvec_iter iter;
switch (((bio)->bi_opf & ((1 << 8) - 1))) {
case REQ_OP_DISCARD:
case REQ_OP_SECURE_ERASE:
case REQ_OP_WRITE_ZEROES:
return 0;
case REQ_OP_WRITE_SAME:
return 1;
default:
break;
}
for (iter = ((bio)->bi_iter); (iter).bi_size && ((bv = ((struct bio_vec) { .bv_page = ((&(((((bio))->bi_io_vec)))[(((((iter))))).bi_idx])->bv_page), .bv_len = ({ typeof(((((iter)))).bi_size) __UNIQUE_ID_min1_71 = (((((iter)))).bi_size); typeof((&(((((bio))->bi_io_vec)))[(((((iter))))).bi_idx])->bv_len - ((((iter)))).bi_bvec_done) __UNIQUE_ID_min2_72 = ((&(((((bio))->bi_io_vec)))[(((((iter))))).bi_idx])->bv_len - ((((iter)))).bi_bvec_done); (void) (&__UNIQUE_ID_min1_71 == &__UNIQUE_ID_min2_72); __UNIQUE_ID_min1_71 < __UNIQUE_ID_min2_72 ? __UNIQUE_ID_min1_71 : __UNIQUE_ID_min2_72; }), .bv_offset = ((&(((((bio))->bi_io_vec)))[(((((iter))))).bi_idx])->bv_offset + ((((iter)))).bi_bvec_done), })), 1); bio_advance_iter((bio), &(iter), (bv).bv_len))
segs++;
return segs;
}
# 258 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_get(struct bio *bio)
{
bio->bi_flags |= (1 << 9);
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
atomic_add(1, &bio->__bi_cnt);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_cnt_set(struct bio *bio, unsigned int count)
{
if (count != 1) {
bio->bi_flags |= (1 << 9);
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
}
({ union { typeof(((&bio->__bi_cnt)->counter)) __val; char __c[1]; } __u = { .__val = ( typeof(((&bio->__bi_cnt)->counter))) ((count)) }; __write_once_size(&(((&bio->__bi_cnt)->counter)), __u.__c, sizeof(((&bio->__bi_cnt)->counter))); __u.__val; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_flagged(struct bio *bio, unsigned int bit)
{
return (bio->bi_flags & (1U << bit)) != 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_set_flag(struct bio *bio, unsigned int bit)
{
bio->bi_flags |= (1U << bit);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_clear_flag(struct bio *bio, unsigned int bit)
{
bio->bi_flags &= ~(1U << bit);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_get_first_bvec(struct bio *bio, struct bio_vec *bv)
{
*bv = ((struct bio_vec) { .bv_page = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_page), .bv_len = ({ typeof(((((bio)->bi_iter))).bi_size) __UNIQUE_ID_min1_73 = (((((bio)->bi_iter))).bi_size); typeof((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done) __UNIQUE_ID_min2_74 = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done); (void) (&__UNIQUE_ID_min1_73 == &__UNIQUE_ID_min2_74); __UNIQUE_ID_min1_73 < __UNIQUE_ID_min2_74 ? __UNIQUE_ID_min1_73 : __UNIQUE_ID_min2_74; }), .bv_offset = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_offset + ((((bio)->bi_iter))).bi_bvec_done), });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_get_last_bvec(struct bio *bio, struct bio_vec *bv)
{
struct bvec_iter iter = bio->bi_iter;
int idx;
if (__builtin_expect(!!(!((bio)->bi_iter.bi_size != ((struct bio_vec) { .bv_page = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_page), .bv_len = ({ typeof(((((bio)->bi_iter))).bi_size) __UNIQUE_ID_min1_75 = (((((bio)->bi_iter))).bi_size); typeof((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done) __UNIQUE_ID_min2_76 = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done); (void) (&__UNIQUE_ID_min1_75 == &__UNIQUE_ID_min2_76); __UNIQUE_ID_min1_75 < __UNIQUE_ID_min2_76 ? __UNIQUE_ID_min1_75 : __UNIQUE_ID_min2_76; }), .bv_offset = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_offset + ((((bio)->bi_iter))).bi_bvec_done), }).bv_len)), 0)) {
*bv = ((struct bio_vec) { .bv_page = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_page), .bv_len = ({ typeof(((((bio)->bi_iter))).bi_size) __UNIQUE_ID_min1_77 = (((((bio)->bi_iter))).bi_size); typeof((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done) __UNIQUE_ID_min2_78 = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_len - ((((bio)->bi_iter))).bi_bvec_done); (void) (&__UNIQUE_ID_min1_77 == &__UNIQUE_ID_min2_78); __UNIQUE_ID_min1_77 < __UNIQUE_ID_min2_78 ? __UNIQUE_ID_min1_77 : __UNIQUE_ID_min2_78; }), .bv_offset = ((&(((((bio))->bi_io_vec)))[(((((bio)->bi_iter)))).bi_idx])->bv_offset + ((((bio)->bi_iter))).bi_bvec_done), });
return;
}
bio_advance_iter(bio, &iter, iter.bi_size);
if (!iter.bi_bvec_done)
idx = iter.bi_idx - 1;
else
idx = iter.bi_idx;
*bv = bio->bi_io_vec[idx];
if (iter.bi_bvec_done)
bv->bv_len = iter.bi_bvec_done;
}
enum bip_flags {
BIP_BLOCK_INTEGRITY = 1 << 0,
BIP_MAPPED_INTEGRITY = 1 << 1,
BIP_CTRL_NOCHECK = 1 << 2,
BIP_DISK_NOCHECK = 1 << 3,
BIP_IP_CHECKSUM = 1 << 4,
};
struct bio_integrity_payload {
struct bio *bip_bio;
struct bvec_iter bip_iter;
unsigned short bip_slab;
unsigned short bip_vcnt;
unsigned short bip_max_vcnt;
unsigned short bip_flags;
struct work_struct bip_work;
struct bio_vec *bip_vec;
struct bio_vec bip_inline_vecs[0];
};
# 381 "./include/linux/bio.h"
extern void bio_trim(struct bio *bio, int offset, int size);
extern struct bio *bio_split(struct bio *bio, int sectors,
gfp_t gfp, struct bio_set *bs);
# 395 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *bio_next_split(struct bio *bio, int sectors,
gfp_t gfp, struct bio_set *bs)
{
if (sectors >= ((bio)->bi_iter.bi_size >> 9))
return bio;
return bio_split(bio, sectors, gfp, bs);
}
extern struct bio_set *bioset_create(unsigned int, unsigned int, int flags);
enum {
BIOSET_NEED_BVECS = (1UL << (0)),
BIOSET_NEED_RESCUER = (1UL << (1)),
};
extern void bioset_free(struct bio_set *);
extern mempool_t *biovec_create_pool(int pool_entries);
extern struct bio *bio_alloc_bioset(gfp_t, unsigned int, struct bio_set *);
extern void bio_put(struct bio *);
extern void __bio_clone_fast(struct bio *, struct bio *);
extern struct bio *bio_clone_fast(struct bio *, gfp_t, struct bio_set *);
extern struct bio *bio_clone_bioset(struct bio *, gfp_t, struct bio_set *bs);
extern struct bio_set *fs_bio_set;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *bio_alloc(gfp_t gfp_mask, unsigned int nr_iovecs)
{
return bio_alloc_bioset(gfp_mask, nr_iovecs, fs_bio_set);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *bio_kmalloc(gfp_t gfp_mask, unsigned int nr_iovecs)
{
return bio_alloc_bioset(gfp_mask, nr_iovecs, ((void *)0));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *bio_clone_kmalloc(struct bio *bio, gfp_t gfp_mask)
{
return bio_clone_bioset(bio, gfp_mask, ((void *)0));
}
extern blk_qc_t submit_bio(struct bio *);
extern void bio_endio(struct bio *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_io_error(struct bio *bio)
{
bio->bi_status = (( blk_status_t)10);
bio_endio(bio);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_wouldblock_error(struct bio *bio)
{
bio->bi_status = (( blk_status_t)12);
bio_endio(bio);
}
struct request_queue;
extern int bio_phys_segments(struct request_queue *, struct bio *);
extern int submit_bio_wait(struct bio *bio);
extern void bio_advance(struct bio *, unsigned);
extern void bio_init(struct bio *bio, struct bio_vec *table,
unsigned short max_vecs);
extern void bio_uninit(struct bio *);
extern void bio_reset(struct bio *);
void bio_chain(struct bio *, struct bio *);
extern int bio_add_page(struct bio *, struct page *, unsigned int,unsigned int);
extern int bio_add_pc_page(struct request_queue *, struct bio *, struct page *,
unsigned int, unsigned int);
bool __bio_try_merge_page(struct bio *bio, struct page *page,
unsigned int len, unsigned int off);
void __bio_add_page(struct bio *bio, struct page *page,
unsigned int len, unsigned int off);
int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter);
struct rq_map_data;
extern struct bio *bio_map_user_iov(struct request_queue *,
const struct iov_iter *, gfp_t);
extern void bio_unmap_user(struct bio *);
extern struct bio *bio_map_kern(struct request_queue *, void *, unsigned int,
gfp_t);
extern struct bio *bio_copy_kern(struct request_queue *, void *, unsigned int,
gfp_t, int);
extern void bio_set_pages_dirty(struct bio *bio);
extern void bio_check_pages_dirty(struct bio *bio);
void generic_start_io_acct(struct request_queue *q, int rw,
unsigned long sectors, struct hd_struct *part);
void generic_end_io_acct(struct request_queue *q, int rw,
struct hd_struct *part,
unsigned long start_time);
extern void bio_flush_dcache_pages(struct bio *bi);
extern void bio_copy_data(struct bio *dst, struct bio *src);
extern int bio_alloc_pages(struct bio *bio, gfp_t gfp);
extern void bio_free_pages(struct bio *bio);
extern struct bio *bio_copy_user_iov(struct request_queue *,
struct rq_map_data *,
const struct iov_iter *,
gfp_t);
extern int bio_uncopy_user(struct bio *);
void zero_fill_bio(struct bio *bio);
extern struct bio_vec *bvec_alloc(gfp_t, int, unsigned long *, mempool_t *);
extern void bvec_free(mempool_t *, struct bio_vec *, unsigned int);
extern unsigned int bvec_nr_vecs(unsigned short idx);
extern const char *bio_devname(struct bio *bio, char *buffer);
# 534 "./include/linux/bio.h"
int bio_associate_blkcg(struct bio *bio, struct cgroup_subsys_state *blkcg_css);
int bio_associate_current(struct bio *bio);
void bio_disassociate_task(struct bio *bio);
void bio_clone_blkcg_association(struct bio *dst, struct bio *src);
# 552 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) char *bvec_kmap_irq(struct bio_vec *bvec, unsigned long *flags)
{
unsigned long addr;
do { do { ({ unsigned long __dummy; typeof(*flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); *flags = arch_local_irq_save(); } while (0); } while (0);
addr = (unsigned long) kmap_atomic(bvec->bv_page);
do { if (__builtin_expect(!!(addr & ~(~((1 << 12) - 1))), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/bio.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "563" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
return (char *) addr + bvec->bv_offset;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bvec_kunmap_irq(char *buffer, unsigned long *flags)
{
unsigned long ptr = (unsigned long) buffer & (~((1 << 12) - 1));
do { do { bool __cond = !(!(__builtin_types_compatible_p(typeof(((void *) ptr)), typeof(struct page *)))); extern void __compiletime_assert_79(void) ; if (__cond) __compiletime_assert_79(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); __kunmap_atomic((void *) ptr); } while (0);
do { do { ({ unsigned long __dummy; typeof(*flags) __dummy2; (void)(&__dummy == &__dummy2); 1; }); arch_local_irq_restore(*flags); } while (0); } while (0);
}
# 588 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) char *__bio_kmap_irq(struct bio *bio, struct bvec_iter iter,
unsigned long *flags)
{
return bvec_kmap_irq(&((struct bio_vec) { .bv_page = ((&((((bio)->bi_io_vec)))[((((iter)))).bi_idx])->bv_page), .bv_len = ({ typeof((((iter))).bi_size) __UNIQUE_ID_min1_80 = ((((iter))).bi_size); typeof((&((((bio)->bi_io_vec)))[((((iter)))).bi_idx])->bv_len - (((iter))).bi_bvec_done) __UNIQUE_ID_min2_81 = ((&((((bio)->bi_io_vec)))[((((iter)))).bi_idx])->bv_len - (((iter))).bi_bvec_done); (void) (&__UNIQUE_ID_min1_80 == &__UNIQUE_ID_min2_81); __UNIQUE_ID_min1_80 < __UNIQUE_ID_min2_81 ? __UNIQUE_ID_min1_80 : __UNIQUE_ID_min2_81; }), .bv_offset = ((&((((bio)->bi_io_vec)))[((((iter)))).bi_idx])->bv_offset + (((iter))).bi_bvec_done), }), flags);
}
# 606 "./include/linux/bio.h"
struct bio_list {
struct bio *head;
struct bio *tail;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bio_list_empty(const struct bio_list *bl)
{
return bl->head == ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_list_init(struct bio_list *bl)
{
bl->head = bl->tail = ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned bio_list_size(const struct bio_list *bl)
{
unsigned sz = 0;
struct bio *bio;
for (bio = (bl)->head; bio; bio = bio->bi_next)
sz++;
return sz;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_list_add(struct bio_list *bl, struct bio *bio)
{
bio->bi_next = ((void *)0);
if (bl->tail)
bl->tail->bi_next = bio;
else
bl->head = bio;
bl->tail = bio;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_list_add_head(struct bio_list *bl, struct bio *bio)
{
bio->bi_next = bl->head;
bl->head = bio;
if (!bl->tail)
bl->tail = bio;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_list_merge(struct bio_list *bl, struct bio_list *bl2)
{
if (!bl2->head)
return;
if (bl->tail)
bl->tail->bi_next = bl2->head;
else
bl->head = bl2->head;
bl->tail = bl2->tail;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_list_merge_head(struct bio_list *bl,
struct bio_list *bl2)
{
if (!bl2->head)
return;
if (bl->head)
bl2->tail->bi_next = bl->head;
else
bl->tail = bl2->tail;
bl->head = bl2->head;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *bio_list_peek(struct bio_list *bl)
{
return bl->head;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *bio_list_pop(struct bio_list *bl)
{
struct bio *bio = bl->head;
if (bio) {
bl->head = bl->head->bi_next;
if (!bl->head)
bl->tail = ((void *)0);
bio->bi_next = ((void *)0);
}
return bio;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *bio_list_get(struct bio_list *bl)
{
struct bio *bio = bl->head;
bl->head = bl->tail = ((void *)0);
return bio;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_inc_remaining(struct bio *bio)
{
bio_set_flag(bio, 8);
__asm__ __volatile__ ("dmb " "ish" : : : "memory");
atomic_add(1, &bio->__bi_remaining);
}
# 734 "./include/linux/bio.h"
struct bio_set {
struct kmem_cache *bio_slab;
unsigned int front_pad;
mempool_t *bio_pool;
mempool_t *bvec_pool;
# 749 "./include/linux/bio.h"
spinlock_t rescue_lock;
struct bio_list rescue_list;
struct work_struct rescue_work;
struct workqueue_struct *rescue_workqueue;
};
struct biovec_slab {
int nr_vecs;
char *name;
struct kmem_cache *slab;
};
# 788 "./include/linux/bio.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *bio_integrity(struct bio *bio)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bioset_integrity_create(struct bio_set *bs, int pool_size)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bioset_integrity_free (struct bio_set *bs)
{
return;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_integrity_prep(struct bio *bio)
{
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bio_integrity_clone(struct bio *bio, struct bio *bio_src,
gfp_t gfp_mask)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_integrity_advance(struct bio *bio,
unsigned int bytes_done)
{
return;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_integrity_trim(struct bio *bio)
{
return;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bio_integrity_init(void)
{
return;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *bio_integrity_alloc(struct bio * bio, gfp_t gfp,
unsigned int nr)
{
return ERR_PTR(-22);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bio_integrity_add_page(struct bio *bio, struct page *page,
unsigned int len, unsigned int offset)
{
return 0;
}
# 22 "./include/linux/blkdev.h" 2
# 1 "./include/linux/bsg.h" 1
# 1 "./include/uapi/linux/bsg.h" 1
# 22 "./include/uapi/linux/bsg.h"
struct sg_io_v4 {
__s32 guard;
__u32 protocol;
__u32 subprotocol;
__u32 request_len;
__u64 request;
__u64 request_tag;
__u32 request_attr;
__u32 request_priority;
__u32 request_extra;
__u32 max_response_len;
__u64 response;
__u32 dout_iovec_count;
__u32 dout_xfer_len;
__u32 din_iovec_count;
__u32 din_xfer_len;
__u64 dout_xferp;
__u64 din_xferp;
__u32 timeout;
__u32 flags;
__u64 usr_ptr;
__u32 spare_in;
__u32 driver_status;
__u32 transport_status;
__u32 device_status;
__u32 retry_delay;
__u32 info;
__u32 duration;
__u32 response_len;
__s32 din_resid;
__s32 dout_resid;
__u64 generated_tag;
__u32 spare_out;
__u32 padding;
};
# 6 "./include/linux/bsg.h" 2
# 23 "./include/linux/bsg.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bsg_register_queue(struct request_queue *q,
struct device *parent, const char *name,
void (*release)(struct device *))
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void bsg_unregister_queue(struct request_queue *q)
{
}
# 25 "./include/linux/blkdev.h" 2
# 1 "./include/linux/scatterlist.h" 1
# 11 "./include/linux/scatterlist.h"
struct scatterlist {
unsigned long page_link;
unsigned int offset;
unsigned int length;
dma_addr_t dma_address;
};
# 39 "./include/linux/scatterlist.h"
struct sg_table {
struct scatterlist *sgl;
unsigned int nents;
unsigned int orig_nents;
};
# 83 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sg_assign_page(struct scatterlist *sg, struct page *page)
{
unsigned long page_link = sg->page_link & 0x3;
do { if (__builtin_expect(!!((unsigned long) page & 0x03), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"include/linux/scatterlist.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "91" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
sg->page_link = page_link | (unsigned long) page;
}
# 113 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sg_set_page(struct scatterlist *sg, struct page *page,
unsigned int len, unsigned int offset)
{
sg_assign_page(sg, page);
sg->offset = offset;
sg->length = len;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *sg_page(struct scatterlist *sg)
{
return (struct page *)((sg)->page_link & ~0x3);
}
# 137 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sg_set_buf(struct scatterlist *sg, const void *buf,
unsigned int buflen)
{
sg_set_page(sg, (mem_map + ((((((unsigned long)(buf) - (0xC0000000UL)) >> 12) + ((unsigned long)((0x00000000UL) >> 12)))) - ((unsigned long)((0x00000000UL) >> 12)))), buflen, ((unsigned long)(buf) & ~(~((1 << 12) - 1))));
}
# 162 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sg_chain(struct scatterlist *prv, unsigned int prv_nents,
struct scatterlist *sgl)
{
prv[prv_nents - 1].offset = 0;
prv[prv_nents - 1].length = 0;
prv[prv_nents - 1].page_link = ((unsigned long) sgl | 0x01) & ~0x02;
}
# 187 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sg_mark_end(struct scatterlist *sg)
{
sg->page_link |= 0x02;
sg->page_link &= ~0x01;
}
# 207 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void sg_unmark_end(struct scatterlist *sg)
{
sg->page_link &= ~0x02;
}
# 225 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) dma_addr_t sg_phys(struct scatterlist *sg)
{
return (((phys_addr_t)(((unsigned long)((sg_page(sg)) - mem_map) + ((unsigned long)((0x00000000UL) >> 12)))) << 12)) + sg->offset;
}
# 240 "./include/linux/scatterlist.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *sg_virt(struct scatterlist *sg)
{
return page_address(sg_page(sg)) + sg->offset;
}
int sg_nents(struct scatterlist *sg);
int sg_nents_for_len(struct scatterlist *sg, u64 len);
struct scatterlist *sg_next(struct scatterlist *);
struct scatterlist *sg_last(struct scatterlist *s, unsigned int);
void sg_init_table(struct scatterlist *, unsigned int);
void sg_init_one(struct scatterlist *, const void *, unsigned int);
int sg_split(struct scatterlist *in, const int in_mapped_nents,
const off_t skip, const int nb_splits,
const size_t *split_sizes,
struct scatterlist **out, int *out_mapped_nents,
gfp_t gfp_mask);
typedef struct scatterlist *(sg_alloc_fn)(unsigned int, gfp_t);
typedef void (sg_free_fn)(struct scatterlist *, unsigned int);
void __sg_free_table(struct sg_table *, unsigned int, bool, sg_free_fn *);
void sg_free_table(struct sg_table *);
int __sg_alloc_table(struct sg_table *, unsigned int, unsigned int,
struct scatterlist *, gfp_t, sg_alloc_fn *);
int sg_alloc_table(struct sg_table *, unsigned int, gfp_t);
int sg_alloc_table_from_pages(struct sg_table *sgt,
struct page **pages, unsigned int n_pages,
unsigned long offset, unsigned long size,
gfp_t gfp_mask);
# 280 "./include/linux/scatterlist.h"
size_t sg_copy_buffer(struct scatterlist *sgl, unsigned int nents, void *buf,
size_t buflen, off_t skip, bool to_buffer);
size_t sg_copy_from_buffer(struct scatterlist *sgl, unsigned int nents,
const void *buf, size_t buflen);
size_t sg_copy_to_buffer(struct scatterlist *sgl, unsigned int nents,
void *buf, size_t buflen);
size_t sg_pcopy_from_buffer(struct scatterlist *sgl, unsigned int nents,
const void *buf, size_t buflen, off_t skip);
size_t sg_pcopy_to_buffer(struct scatterlist *sgl, unsigned int nents,
void *buf, size_t buflen, off_t skip);
size_t sg_zero_buffer(struct scatterlist *sgl, unsigned int nents,
size_t buflen, off_t skip);
# 321 "./include/linux/scatterlist.h"
void sg_free_table_chained(struct sg_table *table, bool first_chunk);
int sg_alloc_table_chained(struct sg_table *table, int nents,
struct scatterlist *first_chunk);
# 336 "./include/linux/scatterlist.h"
struct sg_page_iter {
struct scatterlist *sg;
unsigned int sg_pgoffset;
unsigned int __nents;
int __pg_advance;
};
bool __sg_page_iter_next(struct sg_page_iter *piter);
void __sg_page_iter_start(struct sg_page_iter *piter,
struct scatterlist *sglist, unsigned int nents,
unsigned long pgoffset);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct page *sg_page_iter_page(struct sg_page_iter *piter)
{
return (mem_map + ((((unsigned long)(((sg_page(piter->sg))) - mem_map) + ((unsigned long)((0x00000000UL) >> 12))) + (piter->sg_pgoffset)) - ((unsigned long)((0x00000000UL) >> 12))));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) dma_addr_t sg_page_iter_dma_address(struct sg_page_iter *piter)
{
return ((piter->sg)->dma_address) + (piter->sg_pgoffset << 12);
}
# 400 "./include/linux/scatterlist.h"
struct sg_mapping_iter {
struct page *page;
void *addr;
size_t length;
size_t consumed;
struct sg_page_iter piter;
unsigned int __offset;
unsigned int __remaining;
unsigned int __flags;
};
void sg_miter_start(struct sg_mapping_iter *miter, struct scatterlist *sgl,
unsigned int nents, unsigned int flags);
bool sg_miter_skip(struct sg_mapping_iter *miter, off_t offset);
bool sg_miter_next(struct sg_mapping_iter *miter);
void sg_miter_stop(struct sg_mapping_iter *miter);
# 29 "./include/linux/blkdev.h" 2
# 1 "./include/uapi/linux/blkzoned.h" 1
# 32 "./include/uapi/linux/blkzoned.h"
enum blk_zone_type {
BLK_ZONE_TYPE_CONVENTIONAL = 0x1,
BLK_ZONE_TYPE_SEQWRITE_REQ = 0x2,
BLK_ZONE_TYPE_SEQWRITE_PREF = 0x3,
};
# 65 "./include/uapi/linux/blkzoned.h"
enum blk_zone_cond {
BLK_ZONE_COND_NOT_WP = 0x0,
BLK_ZONE_COND_EMPTY = 0x1,
BLK_ZONE_COND_IMP_OPEN = 0x2,
BLK_ZONE_COND_EXP_OPEN = 0x3,
BLK_ZONE_COND_CLOSED = 0x4,
BLK_ZONE_COND_READONLY = 0xD,
BLK_ZONE_COND_FULL = 0xE,
BLK_ZONE_COND_OFFLINE = 0xF,
};
# 94 "./include/uapi/linux/blkzoned.h"
struct blk_zone {
__u64 start;
__u64 len;
__u64 wp;
__u8 type;
__u8 cond;
__u8 non_seq;
__u8 reset;
__u8 reserved[36];
};
# 115 "./include/uapi/linux/blkzoned.h"
struct blk_zone_report {
__u64 sector;
__u32 nr_zones;
__u8 reserved[4];
struct blk_zone zones[0];
} __attribute__((packed));
struct blk_zone_range {
__u64 sector;
__u64 nr_sectors;
};
# 30 "./include/linux/blkdev.h" 2
struct module;
struct scsi_ioctl_command;
struct request_queue;
struct elevator_queue;
struct blk_trace;
struct request;
struct sg_io_hdr;
struct bsg_job;
struct blkcg_gq;
struct blk_flush_queue;
struct pr_ops;
struct rq_wb;
struct blk_queue_stats;
struct blk_stat_callback;
struct keyslot_manager;
# 60 "./include/linux/blkdev.h"
typedef void (rq_end_io_fn)(struct request *, blk_status_t);
struct request_list {
struct request_queue *q;
struct blkcg_gq *blkg;
int count[2];
int starved[2];
mempool_t *rq_pool;
wait_queue_head_t wait[2];
unsigned int flags;
};
typedef __u32 req_flags_t;
# 136 "./include/linux/blkdev.h"
struct request {
struct list_head queuelist;
union {
struct __call_single_data csd;
u64 fifo_time;
};
struct request_queue *q;
struct blk_mq_ctx *mq_ctx;
int cpu;
unsigned int cmd_flags;
req_flags_t rq_flags;
int internal_tag;
unsigned long atomic_flags;
unsigned int __data_len;
int tag;
sector_t __sector;
struct bio *bio;
struct bio *biotail;
# 169 "./include/linux/blkdev.h"
union {
struct hlist_node hash;
struct list_head ipi_list;
};
union {
struct rb_node rb_node;
struct bio_vec special_vec;
void *completion_data;
int error_count;
};
union {
struct {
struct io_cq *icq;
void *priv[2];
} elv;
struct {
unsigned int seq;
struct list_head list;
rq_end_io_fn *saved_end_io;
} flush;
};
struct gendisk *rq_disk;
struct hd_struct *part;
unsigned long start_time;
struct blk_issue_stat issue_stat;
struct request_list *rl;
unsigned long long start_time_ns;
unsigned long long io_start_time_ns;
unsigned short nr_phys_segments;
unsigned short ioprio;
unsigned int timeout;
void *special;
unsigned int extra_len;
unsigned short write_hint;
unsigned long deadline;
struct list_head timeout_list;
rq_end_io_fn *end_io;
void *end_io_data;
struct request *next_rq;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_op_is_scsi(unsigned int op)
{
return op == REQ_OP_SCSI_IN || op == REQ_OP_SCSI_OUT;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_op_is_private(unsigned int op)
{
return op == REQ_OP_DRV_IN || op == REQ_OP_DRV_OUT;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_rq_is_scsi(struct request *rq)
{
return blk_op_is_scsi(((rq)->cmd_flags & ((1 << 8) - 1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_rq_is_private(struct request *rq)
{
return blk_op_is_private(((rq)->cmd_flags & ((1 << 8) - 1)));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_rq_is_passthrough(struct request *rq)
{
return blk_rq_is_scsi(rq) || blk_rq_is_private(rq);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_is_passthrough(struct bio *bio)
{
unsigned op = ((bio)->bi_opf & ((1 << 8) - 1));
return blk_op_is_scsi(op) || blk_op_is_private(op);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned short req_get_ioprio(struct request *req)
{
return req->ioprio;
}
# 1 "./include/linux/elevator.h" 1
# 1 "./include/linux/hashtable.h" 1
# 34 "./include/linux/hashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void __hash_init(struct hlist_head *ht, unsigned int sz)
{
unsigned int i;
for (i = 0; i < sz; i++)
((&ht[i])->first = ((void *)0));
}
# 76 "./include/linux/hashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool hash_hashed(struct hlist_node *node)
{
return !hlist_unhashed(node);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool __hash_empty(struct hlist_head *ht, unsigned int sz)
{
unsigned int i;
for (i = 0; i < sz; i++)
if (!hlist_empty(&ht[i]))
return false;
return true;
}
# 105 "./include/linux/hashtable.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hash_del(struct hlist_node *node)
{
hlist_del_init(node);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void hash_del_rcu(struct hlist_node *node)
{
hlist_del_init_rcu(node);
}
# 7 "./include/linux/elevator.h" 2
struct io_cq;
struct elevator_type;
enum elv_merge {
ELEVATOR_NO_MERGE = 0,
ELEVATOR_FRONT_MERGE = 1,
ELEVATOR_BACK_MERGE = 2,
ELEVATOR_DISCARD_MERGE = 3,
};
typedef enum elv_merge (elevator_merge_fn) (struct request_queue *, struct request **,
struct bio *);
typedef void (elevator_merge_req_fn) (struct request_queue *, struct request *, struct request *);
typedef void (elevator_merged_fn) (struct request_queue *, struct request *, enum elv_merge);
typedef int (elevator_allow_bio_merge_fn) (struct request_queue *,
struct request *, struct bio *);
typedef int (elevator_allow_rq_merge_fn) (struct request_queue *,
struct request *, struct request *);
typedef void (elevator_bio_merged_fn) (struct request_queue *,
struct request *, struct bio *);
typedef int (elevator_dispatch_fn) (struct request_queue *, int);
typedef void (elevator_add_req_fn) (struct request_queue *, struct request *);
typedef struct request *(elevator_request_list_fn) (struct request_queue *, struct request *);
typedef void (elevator_completed_req_fn) (struct request_queue *, struct request *);
typedef int (elevator_may_queue_fn) (struct request_queue *, unsigned int);
typedef void (elevator_init_icq_fn) (struct io_cq *);
typedef void (elevator_exit_icq_fn) (struct io_cq *);
typedef int (elevator_set_req_fn) (struct request_queue *, struct request *,
struct bio *, gfp_t);
typedef void (elevator_put_req_fn) (struct request *);
typedef void (elevator_activate_req_fn) (struct request_queue *, struct request *);
typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct request *);
typedef int (elevator_init_fn) (struct request_queue *,
struct elevator_type *e);
typedef void (elevator_exit_fn) (struct elevator_queue *);
typedef void (elevator_registered_fn) (struct request_queue *);
struct elevator_ops
{
elevator_merge_fn *elevator_merge_fn;
elevator_merged_fn *elevator_merged_fn;
elevator_merge_req_fn *elevator_merge_req_fn;
elevator_allow_bio_merge_fn *elevator_allow_bio_merge_fn;
elevator_allow_rq_merge_fn *elevator_allow_rq_merge_fn;
elevator_bio_merged_fn *elevator_bio_merged_fn;
elevator_dispatch_fn *elevator_dispatch_fn;
elevator_add_req_fn *elevator_add_req_fn;
elevator_activate_req_fn *elevator_activate_req_fn;
elevator_deactivate_req_fn *elevator_deactivate_req_fn;
elevator_completed_req_fn *elevator_completed_req_fn;
elevator_request_list_fn *elevator_former_req_fn;
elevator_request_list_fn *elevator_latter_req_fn;
elevator_init_icq_fn *elevator_init_icq_fn;
elevator_exit_icq_fn *elevator_exit_icq_fn;
elevator_set_req_fn *elevator_set_req_fn;
elevator_put_req_fn *elevator_put_req_fn;
elevator_may_queue_fn *elevator_may_queue_fn;
elevator_init_fn *elevator_init_fn;
elevator_exit_fn *elevator_exit_fn;
elevator_registered_fn *elevator_registered_fn;
};
struct blk_mq_alloc_data;
struct blk_mq_hw_ctx;
struct elevator_mq_ops {
int (*init_sched)(struct request_queue *, struct elevator_type *);
void (*exit_sched)(struct elevator_queue *);
int (*init_hctx)(struct blk_mq_hw_ctx *, unsigned int);
void (*exit_hctx)(struct blk_mq_hw_ctx *, unsigned int);
bool (*allow_merge)(struct request_queue *, struct request *, struct bio *);
bool (*bio_merge)(struct blk_mq_hw_ctx *, struct bio *);
int (*request_merge)(struct request_queue *q, struct request **, struct bio *);
void (*request_merged)(struct request_queue *, struct request *, enum elv_merge);
void (*requests_merged)(struct request_queue *, struct request *, struct request *);
void (*limit_depth)(unsigned int, struct blk_mq_alloc_data *);
void (*prepare_request)(struct request *, struct bio *bio);
void (*finish_request)(struct request *);
void (*insert_requests)(struct blk_mq_hw_ctx *, struct list_head *, bool);
struct request *(*dispatch_request)(struct blk_mq_hw_ctx *);
bool (*has_work)(struct blk_mq_hw_ctx *);
void (*completed_request)(struct request *);
void (*started_request)(struct request *);
void (*requeue_request)(struct request *);
struct request *(*former_request)(struct request_queue *, struct request *);
struct request *(*next_request)(struct request_queue *, struct request *);
void (*init_icq)(struct io_cq *);
void (*exit_icq)(struct io_cq *);
};
struct elv_fs_entry {
struct attribute attr;
ssize_t (*show)(struct elevator_queue *, char *);
ssize_t (*store)(struct elevator_queue *, const char *, size_t);
};
struct elevator_type
{
struct kmem_cache *icq_cache;
union {
struct elevator_ops sq;
struct elevator_mq_ops mq;
} ops;
size_t icq_size;
size_t icq_align;
struct elv_fs_entry *elevator_attrs;
char elevator_name[(16)];
struct module *elevator_owner;
bool uses_mq;
char icq_cache_name[(16) + 6];
struct list_head list;
};
void elv_rqhash_del(struct request_queue *q, struct request *rq);
void elv_rqhash_add(struct request_queue *q, struct request *rq);
void elv_rqhash_reposition(struct request_queue *q, struct request *rq);
struct request *elv_rqhash_find(struct request_queue *q, sector_t offset);
struct elevator_queue
{
struct elevator_type *type;
void *elevator_data;
struct kobject kobj;
struct mutex sysfs_lock;
unsigned int registered:1;
unsigned int uses_mq:1;
struct hlist_head hash[1 << (6)];
};
extern void elv_dispatch_sort(struct request_queue *, struct request *);
extern void elv_dispatch_add_tail(struct request_queue *, struct request *);
extern void elv_add_request(struct request_queue *, struct request *, int);
extern void __elv_add_request(struct request_queue *, struct request *, int);
extern enum elv_merge elv_merge(struct request_queue *, struct request **,
struct bio *);
extern void elv_merge_requests(struct request_queue *, struct request *,
struct request *);
extern void elv_merged_request(struct request_queue *, struct request *,
enum elv_merge);
extern void elv_bio_merged(struct request_queue *q, struct request *,
struct bio *);
extern bool elv_attempt_insert_merge(struct request_queue *, struct request *);
extern void elv_requeue_request(struct request_queue *, struct request *);
extern struct request *elv_former_request(struct request_queue *, struct request *);
extern struct request *elv_latter_request(struct request_queue *, struct request *);
extern int elv_register_queue(struct request_queue *q);
extern void elv_unregister_queue(struct request_queue *q);
extern int elv_may_queue(struct request_queue *, unsigned int);
extern void elv_completed_request(struct request_queue *, struct request *);
extern int elv_set_request(struct request_queue *q, struct request *rq,
struct bio *bio, gfp_t gfp_mask);
extern void elv_put_request(struct request_queue *, struct request *);
extern void elv_drain_elevator(struct request_queue *);
extern void __attribute__ ((__section__(".init.text"))) load_default_elevator_module(void);
extern int elv_register(struct elevator_type *);
extern void elv_unregister(struct elevator_type *);
extern ssize_t elv_iosched_show(struct request_queue *, char *);
extern ssize_t elv_iosched_store(struct request_queue *, const char *, size_t);
extern int elevator_init(struct request_queue *, char *);
extern void elevator_exit(struct request_queue *, struct elevator_queue *);
extern bool elv_bio_merge_ok(struct request *, struct bio *);
extern struct elevator_queue *elevator_alloc(struct request_queue *,
struct elevator_type *);
extern struct request *elv_rb_former_request(struct request_queue *, struct request *);
extern struct request *elv_rb_latter_request(struct request_queue *, struct request *);
extern void elv_rb_add(struct rb_root *, struct request *);
extern void elv_rb_del(struct rb_root *, struct request *);
extern struct request *elv_rb_find(struct rb_root *, sector_t);
# 254 "./include/linux/elevator.h"
enum {
ELV_MQUEUE_MAY,
ELV_MQUEUE_NO,
ELV_MQUEUE_MUST,
};
# 283 "./include/linux/blkdev.h" 2
struct blk_queue_ctx;
typedef void (request_fn_proc) (struct request_queue *q);
typedef blk_qc_t (make_request_fn) (struct request_queue *q, struct bio *bio);
typedef int (prep_rq_fn) (struct request_queue *, struct request *);
typedef void (unprep_rq_fn) (struct request_queue *, struct request *);
struct bio_vec;
typedef void (softirq_done_fn)(struct request *);
typedef int (dma_drain_needed_fn)(struct request *);
typedef int (lld_busy_fn) (struct request_queue *q);
typedef int (bsg_job_fn) (struct bsg_job *);
typedef int (init_rq_fn)(struct request_queue *, struct request *, gfp_t);
typedef void (exit_rq_fn)(struct request_queue *, struct request *);
enum blk_eh_timer_return {
BLK_EH_NOT_HANDLED,
BLK_EH_HANDLED,
BLK_EH_RESET_TIMER,
};
typedef enum blk_eh_timer_return (rq_timed_out_fn)(struct request *);
enum blk_queue_state {
Queue_down,
Queue_up,
};
struct blk_queue_tag {
struct request **tag_index;
unsigned long *tag_map;
int max_depth;
int real_max_depth;
atomic_t refcnt;
int alloc_policy;
int next_tag;
};
# 330 "./include/linux/blkdev.h"
enum blk_zoned_model {
BLK_ZONED_NONE,
BLK_ZONED_HA,
BLK_ZONED_HM,
};
struct queue_limits {
unsigned long bounce_pfn;
unsigned long seg_boundary_mask;
unsigned long virt_boundary_mask;
unsigned int max_hw_sectors;
unsigned int max_dev_sectors;
unsigned int chunk_sectors;
unsigned int max_sectors;
unsigned int max_segment_size;
unsigned int physical_block_size;
unsigned int logical_block_size;
unsigned int alignment_offset;
unsigned int io_min;
unsigned int io_opt;
unsigned int max_discard_sectors;
unsigned int max_hw_discard_sectors;
unsigned int max_write_same_sectors;
unsigned int max_write_zeroes_sectors;
unsigned int discard_granularity;
unsigned int discard_alignment;
unsigned short max_segments;
unsigned short max_integrity_segments;
unsigned short max_discard_segments;
unsigned char misaligned;
unsigned char discard_misaligned;
unsigned char cluster;
unsigned char raid_partial_stripes_expensive;
enum blk_zoned_model zoned;
};
# 389 "./include/linux/blkdev.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blkdev_report_zones_ioctl(struct block_device *bdev,
fmode_t mode, unsigned int cmd,
unsigned long arg)
{
return -25;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blkdev_reset_zones_ioctl(struct block_device *bdev,
fmode_t mode, unsigned int cmd,
unsigned long arg)
{
return -25;
}
struct request_queue {
struct list_head queue_head;
struct request *last_merge;
struct elevator_queue *elevator;
int nr_rqs[2];
int nr_rqs_elvpriv;
atomic_t shared_hctx_restart;
struct blk_queue_stats *stats;
struct rq_wb *rq_wb;
struct request_list root_rl;
request_fn_proc *request_fn;
make_request_fn *make_request_fn;
prep_rq_fn *prep_rq_fn;
unprep_rq_fn *unprep_rq_fn;
softirq_done_fn *softirq_done_fn;
rq_timed_out_fn *rq_timed_out_fn;
dma_drain_needed_fn *dma_drain_needed;
lld_busy_fn *lld_busy_fn;
init_rq_fn *init_rq_fn;
exit_rq_fn *exit_rq_fn;
void (*initialize_rq_fn)(struct request *rq);
const struct blk_mq_ops *mq_ops;
unsigned int *mq_map;
struct blk_mq_ctx *queue_ctx;
unsigned int nr_queues;
unsigned int queue_depth;
struct blk_mq_hw_ctx **queue_hw_ctx;
unsigned int nr_hw_queues;
sector_t end_sector;
struct request *boundary_rq;
struct delayed_work delay_work;
struct backing_dev_info *backing_dev_info;
void *queuedata;
unsigned long queue_flags;
int id;
gfp_t bounce_gfp;
spinlock_t __queue_lock;
spinlock_t *queue_lock;
struct kobject kobj;
struct kobject mq_kobj;
struct device *dev;
int rpm_status;
unsigned int nr_pending;
unsigned long nr_requests;
unsigned int nr_congestion_on;
unsigned int nr_congestion_off;
unsigned int nr_batching;
unsigned int dma_drain_size;
void *dma_drain_buffer;
unsigned int dma_pad_mask;
unsigned int dma_alignment;
struct blk_queue_tag *queue_tags;
struct list_head tag_busy_list;
unsigned int nr_sorted;
unsigned int in_flight[2];
unsigned int request_fn_active;
struct keyslot_manager *ksm;
unsigned int rq_timeout;
int poll_nsec;
struct blk_stat_callback *poll_cb;
struct blk_rq_stat poll_stat[16];
struct timer_list timeout;
struct work_struct timeout_work;
struct list_head timeout_list;
struct list_head icq_list;
unsigned long blkcg_pols[(((3) + ((sizeof(long) * 8)) - 1) / ((sizeof(long) * 8)))];
struct blkcg_gq *root_blkg;
struct list_head blkg_list;
struct queue_limits limits;
unsigned int sg_timeout;
unsigned int sg_reserved_size;
int node;
struct blk_flush_queue *fq;
struct list_head requeue_list;
spinlock_t requeue_lock;
struct delayed_work requeue_work;
struct mutex sysfs_lock;
int bypass_depth;
atomic_t mq_freeze_depth;
# 601 "./include/linux/blkdev.h"
struct throtl_data *td;
struct callback_head callback_head;
wait_queue_head_t mq_freeze_wq;
struct percpu_ref q_usage_counter;
struct list_head all_q_node;
struct blk_mq_tag_set *tag_set;
struct list_head tag_set_list;
struct bio_set *bio_split;
bool mq_sysfs_init_done;
size_t cmd_size;
void *rq_alloc_data;
struct work_struct release_work;
u64 write_hints[5];
};
# 676 "./include/linux/blkdev.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void queue_lockdep_assert_held(struct request_queue *q)
{
if (q->queue_lock)
do { (void)(q->queue_lock); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void queue_flag_set_unlocked(unsigned int flag,
struct request_queue *q)
{
__set_bit(flag, &q->queue_flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_flag_test_and_clear(unsigned int flag,
struct request_queue *q)
{
queue_lockdep_assert_held(q);
if (test_bit(flag, &q->queue_flags)) {
__clear_bit(flag, &q->queue_flags);
return 1;
}
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_flag_test_and_set(unsigned int flag,
struct request_queue *q)
{
queue_lockdep_assert_held(q);
if (!test_bit(flag, &q->queue_flags)) {
__set_bit(flag, &q->queue_flags);
return 0;
}
return 1;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void queue_flag_set(unsigned int flag, struct request_queue *q)
{
queue_lockdep_assert_held(q);
__set_bit(flag, &q->queue_flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void queue_flag_clear_unlocked(unsigned int flag,
struct request_queue *q)
{
__clear_bit(flag, &q->queue_flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_in_flight(struct request_queue *q)
{
return q->in_flight[0] + q->in_flight[1];
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void queue_flag_clear(unsigned int flag, struct request_queue *q)
{
queue_lockdep_assert_held(q);
__clear_bit(flag, &q->queue_flags);
}
# 763 "./include/linux/blkdev.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_account_rq(struct request *rq)
{
return (rq->rq_flags & (( req_flags_t)(1 << 1))) && !blk_rq_is_passthrough(rq);
}
# 781 "./include/linux/blkdev.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool queue_is_rq_based(struct request_queue *q)
{
return q->request_fn || q->mq_ops;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_queue_cluster(struct request_queue *q)
{
return q->limits.cluster;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) enum blk_zoned_model
blk_queue_zoned_model(struct request_queue *q)
{
return q->limits.zoned;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_queue_is_zoned(struct request_queue *q)
{
switch (blk_queue_zoned_model(q)) {
case BLK_ZONED_HA:
case BLK_ZONED_HM:
return true;
default:
return false;
}
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_queue_zone_sectors(struct request_queue *q)
{
return blk_queue_is_zoned(q) ? q->limits.chunk_sectors : 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rq_is_sync(struct request *rq)
{
return op_is_sync(rq->cmd_flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_rl_full(struct request_list *rl, bool sync)
{
unsigned int flag = sync ? (1U << 0) : (1U << 1);
return rl->flags & flag;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_set_rl_full(struct request_list *rl, bool sync)
{
unsigned int flag = sync ? (1U << 0) : (1U << 1);
rl->flags |= flag;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_clear_rl_full(struct request_list *rl, bool sync)
{
unsigned int flag = sync ? (1U << 0) : (1U << 1);
rl->flags &= ~flag;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool rq_mergeable(struct request *rq)
{
if (blk_rq_is_passthrough(rq))
return false;
if (((rq)->cmd_flags & ((1 << 8) - 1)) == REQ_OP_FLUSH)
return false;
if (((rq)->cmd_flags & ((1 << 8) - 1)) == REQ_OP_WRITE_ZEROES)
return false;
if (rq->cmd_flags & ((1ULL << __REQ_NOMERGE) | (1ULL << __REQ_PREFLUSH) | (1ULL << __REQ_FUA)))
return false;
if (rq->rq_flags & ((( req_flags_t)(1 << 1)) | (( req_flags_t)(1 << 3)) | (( req_flags_t)(1 << 4)) | (( req_flags_t)(1 << 18))))
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_write_same_mergeable(struct bio *a, struct bio *b)
{
if (((&((((a))->bi_io_vec))[((((a)->bi_iter))).bi_idx])->bv_page) == ((&((((b))->bi_io_vec))[((((b)->bi_iter))).bi_idx])->bv_page) &&
((&((((a))->bi_io_vec))[((((a)->bi_iter))).bi_idx])->bv_offset + (((a)->bi_iter)).bi_bvec_done) == ((&((((b))->bi_io_vec))[((((b)->bi_iter))).bi_idx])->bv_offset + (((b)->bi_iter)).bi_bvec_done))
return true;
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_queue_depth(struct request_queue *q)
{
if (q->queue_depth)
return q->queue_depth;
return q->nr_requests;
}
enum {
BLKPREP_OK,
BLKPREP_KILL,
BLKPREP_DEFER,
BLKPREP_INVALID,
};
extern unsigned long blk_max_low_pfn, blk_max_pfn;
# 909 "./include/linux/blkdev.h"
struct rq_map_data {
struct page **pages;
int page_order;
int nr_entries;
unsigned long offset;
int null_mapped;
int from_user;
};
struct req_iterator {
struct bvec_iter iter;
struct bio *bio;
};
# 942 "./include/linux/blkdev.h"
extern void rq_flush_dcache_pages(struct request *rq);
# 960 "./include/linux/blkdev.h"
extern int blk_register_queue(struct gendisk *disk);
extern void blk_unregister_queue(struct gendisk *disk);
extern blk_qc_t generic_make_request(struct bio *bio);
extern void blk_rq_init(struct request_queue *q, struct request *rq);
extern void blk_init_request_from_bio(struct request *req, struct bio *bio);
extern void blk_put_request(struct request *);
extern void __blk_put_request(struct request_queue *, struct request *);
extern struct request *blk_get_request(struct request_queue *, unsigned int op,
gfp_t gfp_mask);
extern void blk_requeue_request(struct request_queue *, struct request *);
extern int blk_lld_busy(struct request_queue *q);
extern int blk_rq_prep_clone(struct request *rq, struct request *rq_src,
struct bio_set *bs, gfp_t gfp_mask,
int (*bio_ctr)(struct bio *, struct bio *, void *),
void *data);
extern void blk_rq_unprep_clone(struct request *rq);
extern blk_status_t blk_insert_cloned_request(struct request_queue *q,
struct request *rq);
extern int blk_rq_append_bio(struct request *rq, struct bio **bio);
extern void blk_delay_queue(struct request_queue *, unsigned long);
extern void blk_queue_split(struct request_queue *, struct bio **);
extern void blk_recount_segments(struct request_queue *, struct bio *);
extern int scsi_verify_blk_ioctl(struct block_device *, unsigned int);
extern int scsi_cmd_blk_ioctl(struct block_device *, fmode_t,
unsigned int, void *);
extern int scsi_cmd_ioctl(struct request_queue *, struct gendisk *, fmode_t,
unsigned int, void *);
extern int sg_scsi_ioctl(struct request_queue *, struct gendisk *, fmode_t,
struct scsi_ioctl_command *);
extern int blk_queue_enter(struct request_queue *q, bool nowait);
extern void blk_queue_exit(struct request_queue *q);
extern void blk_start_queue(struct request_queue *q);
extern void blk_start_queue_async(struct request_queue *q);
extern void blk_stop_queue(struct request_queue *q);
extern void blk_sync_queue(struct request_queue *q);
extern void __blk_stop_queue(struct request_queue *q);
extern void __blk_run_queue(struct request_queue *q);
extern void __blk_run_queue_uncond(struct request_queue *q);
extern void blk_run_queue(struct request_queue *);
extern void blk_run_queue_async(struct request_queue *q);
extern int blk_rq_map_user(struct request_queue *, struct request *,
struct rq_map_data *, void *, unsigned long,
gfp_t);
extern int blk_rq_unmap_user(struct bio *);
extern int blk_rq_map_kern(struct request_queue *, struct request *, void *, unsigned int, gfp_t);
extern int blk_rq_map_user_iov(struct request_queue *, struct request *,
struct rq_map_data *, const struct iov_iter *,
gfp_t);
extern void blk_execute_rq(struct request_queue *, struct gendisk *,
struct request *, int);
extern void blk_execute_rq_nowait(struct request_queue *, struct gendisk *,
struct request *, int, rq_end_io_fn *);
int blk_status_to_errno(blk_status_t status);
blk_status_t errno_to_blk_status(int errno);
bool blk_mq_poll(struct request_queue *q, blk_qc_t cookie);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct request_queue *bdev_get_queue(struct block_device *bdev)
{
return bdev->bd_disk->queue;
}
# 1032 "./include/linux/blkdev.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) sector_t blk_rq_pos(const struct request *rq)
{
return rq->__sector;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_rq_bytes(const struct request *rq)
{
return rq->__data_len;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blk_rq_cur_bytes(const struct request *rq)
{
return rq->bio ? bio_cur_bytes(rq->bio) : 0;
}
extern unsigned int blk_rq_err_bytes(const struct request *rq);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_rq_sectors(const struct request *rq)
{
return blk_rq_bytes(rq) >> 9;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_rq_cur_sectors(const struct request *rq)
{
return blk_rq_cur_bytes(rq) >> 9;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_rq_payload_bytes(struct request *rq)
{
if (rq->rq_flags & (( req_flags_t)(1 << 18)))
return rq->special_vec.bv_len;
return blk_rq_bytes(rq);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_queue_get_max_sectors(struct request_queue *q,
int op)
{
if (__builtin_expect(!!(op == REQ_OP_DISCARD || op == REQ_OP_SECURE_ERASE), 0))
return ({ typeof(q->limits.max_discard_sectors) __UNIQUE_ID_min1_82 = (q->limits.max_discard_sectors); typeof((~0U) >> 9) __UNIQUE_ID_min2_83 = ((~0U) >> 9); (void) (&__UNIQUE_ID_min1_82 == &__UNIQUE_ID_min2_83); __UNIQUE_ID_min1_82 < __UNIQUE_ID_min2_83 ? __UNIQUE_ID_min1_82 : __UNIQUE_ID_min2_83; });
if (__builtin_expect(!!(op == REQ_OP_WRITE_SAME), 0))
return q->limits.max_write_same_sectors;
if (__builtin_expect(!!(op == REQ_OP_WRITE_ZEROES), 0))
return q->limits.max_write_zeroes_sectors;
return q->limits.max_sectors;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_max_size_offset(struct request_queue *q,
sector_t offset)
{
if (!q->limits.chunk_sectors)
return q->limits.max_sectors;
return ({ typeof(q->limits.max_sectors) __UNIQUE_ID_min1_84 = (q->limits.max_sectors); typeof((unsigned int)(q->limits.chunk_sectors - (offset & (q->limits.chunk_sectors - 1)))) __UNIQUE_ID_min2_85 = ((unsigned int)(q->limits.chunk_sectors - (offset & (q->limits.chunk_sectors - 1)))); (void) (&__UNIQUE_ID_min1_84 == &__UNIQUE_ID_min2_85); __UNIQUE_ID_min1_84 < __UNIQUE_ID_min2_85 ? __UNIQUE_ID_min1_84 : __UNIQUE_ID_min2_85; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_rq_get_max_sectors(struct request *rq,
sector_t offset)
{
struct request_queue *q = rq->q;
if (blk_rq_is_passthrough(rq))
return q->limits.max_hw_sectors;
if (!q->limits.chunk_sectors ||
((rq)->cmd_flags & ((1 << 8) - 1)) == REQ_OP_DISCARD ||
((rq)->cmd_flags & ((1 << 8) - 1)) == REQ_OP_SECURE_ERASE)
return blk_queue_get_max_sectors(q, ((rq)->cmd_flags & ((1 << 8) - 1)));
return ({ typeof(blk_max_size_offset(q, offset)) __UNIQUE_ID_min1_86 = (blk_max_size_offset(q, offset)); typeof(blk_queue_get_max_sectors(q, ((rq)->cmd_flags & ((1 << 8) - 1)))) __UNIQUE_ID_min2_87 = (blk_queue_get_max_sectors(q, ((rq)->cmd_flags & ((1 << 8) - 1)))); (void) (&__UNIQUE_ID_min1_86 == &__UNIQUE_ID_min2_87); __UNIQUE_ID_min1_86 < __UNIQUE_ID_min2_87 ? __UNIQUE_ID_min1_86 : __UNIQUE_ID_min2_87; });
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blk_rq_count_bios(struct request *rq)
{
unsigned int nr_bios = 0;
struct bio *bio;
if ((rq->bio)) for (bio = (rq)->bio; bio; bio = bio->bi_next)
nr_bios++;
return nr_bios;
}
extern struct request *blk_peek_request(struct request_queue *q);
extern void blk_start_request(struct request *rq);
extern struct request *blk_fetch_request(struct request_queue *q);
# 1149 "./include/linux/blkdev.h"
extern bool blk_update_request(struct request *rq, blk_status_t error,
unsigned int nr_bytes);
extern void blk_finish_request(struct request *rq, blk_status_t error);
extern bool blk_end_request(struct request *rq, blk_status_t error,
unsigned int nr_bytes);
extern void blk_end_request_all(struct request *rq, blk_status_t error);
extern bool __blk_end_request(struct request *rq, blk_status_t error,
unsigned int nr_bytes);
extern void __blk_end_request_all(struct request *rq, blk_status_t error);
extern bool __blk_end_request_cur(struct request *rq, blk_status_t error);
extern void blk_complete_request(struct request *);
extern void __blk_complete_request(struct request *);
extern void blk_abort_request(struct request *);
extern void blk_unprep_request(struct request *);
extern struct request_queue *blk_init_queue_node(request_fn_proc *rfn,
spinlock_t *lock, int node_id);
extern struct request_queue *blk_init_queue(request_fn_proc *, spinlock_t *);
extern int blk_init_allocated_queue(struct request_queue *);
extern void blk_cleanup_queue(struct request_queue *);
extern void blk_queue_make_request(struct request_queue *, make_request_fn *);
extern void blk_queue_bounce_limit(struct request_queue *, u64);
extern void blk_queue_max_hw_sectors(struct request_queue *, unsigned int);
extern void blk_queue_chunk_sectors(struct request_queue *, unsigned int);
extern void blk_queue_max_segments(struct request_queue *, unsigned short);
extern void blk_queue_max_discard_segments(struct request_queue *,
unsigned short);
extern void blk_queue_max_segment_size(struct request_queue *, unsigned int);
extern void blk_queue_max_discard_sectors(struct request_queue *q,
unsigned int max_discard_sectors);
extern void blk_queue_max_write_same_sectors(struct request_queue *q,
unsigned int max_write_same_sectors);
extern void blk_queue_max_write_zeroes_sectors(struct request_queue *q,
unsigned int max_write_same_sectors);
extern void blk_queue_logical_block_size(struct request_queue *, unsigned int);
extern void blk_queue_physical_block_size(struct request_queue *, unsigned int);
extern void blk_queue_alignment_offset(struct request_queue *q,
unsigned int alignment);
extern void blk_limits_io_min(struct queue_limits *limits, unsigned int min);
extern void blk_queue_io_min(struct request_queue *q, unsigned int min);
extern void blk_limits_io_opt(struct queue_limits *limits, unsigned int opt);
extern void blk_queue_io_opt(struct request_queue *q, unsigned int opt);
extern void blk_set_queue_depth(struct request_queue *q, unsigned int depth);
extern void blk_set_default_limits(struct queue_limits *lim);
extern void blk_set_stacking_limits(struct queue_limits *lim);
extern int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
sector_t offset);
extern int bdev_stack_limits(struct queue_limits *t, struct block_device *bdev,
sector_t offset);
extern void disk_stack_limits(struct gendisk *disk, struct block_device *bdev,
sector_t offset);
extern void blk_queue_stack_limits(struct request_queue *t, struct request_queue *b);
extern void blk_queue_dma_pad(struct request_queue *, unsigned int);
extern void blk_queue_update_dma_pad(struct request_queue *, unsigned int);
extern int blk_queue_dma_drain(struct request_queue *q,
dma_drain_needed_fn *dma_drain_needed,
void *buf, unsigned int size);
extern void blk_queue_lld_busy(struct request_queue *q, lld_busy_fn *fn);
extern void blk_queue_segment_boundary(struct request_queue *, unsigned long);
extern void blk_queue_virt_boundary(struct request_queue *, unsigned long);
extern void blk_queue_prep_rq(struct request_queue *, prep_rq_fn *pfn);
extern void blk_queue_unprep_rq(struct request_queue *, unprep_rq_fn *ufn);
extern void blk_queue_dma_alignment(struct request_queue *, int);
extern void blk_queue_update_dma_alignment(struct request_queue *, int);
extern void blk_queue_softirq_done(struct request_queue *, softirq_done_fn *);
extern void blk_queue_rq_timed_out(struct request_queue *, rq_timed_out_fn *);
extern void blk_queue_rq_timeout(struct request_queue *, unsigned int);
extern void blk_queue_flush_queueable(struct request_queue *q, bool queueable);
extern void blk_queue_write_cache(struct request_queue *q, bool enabled, bool fua);
# 1232 "./include/linux/blkdev.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned short blk_rq_nr_phys_segments(struct request *rq)
{
if (rq->rq_flags & (( req_flags_t)(1 << 18)))
return 1;
return rq->nr_phys_segments;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned short blk_rq_nr_discard_segments(struct request *rq)
{
return ({ unsigned short __UNIQUE_ID_min1_88 = (rq->nr_phys_segments); unsigned short __UNIQUE_ID_min2_89 = (1); (void) (&__UNIQUE_ID_min1_88 == &__UNIQUE_ID_min2_89); __UNIQUE_ID_min1_88 > __UNIQUE_ID_min2_89 ? __UNIQUE_ID_min1_88 : __UNIQUE_ID_min2_89; });
}
extern int blk_rq_map_sg(struct request_queue *, struct request *, struct scatterlist *);
extern void blk_dump_rq_flags(struct request *, char *);
extern long nr_blockdev_pages(void);
bool __attribute__((warn_unused_result)) blk_get_queue(struct request_queue *);
struct request_queue *blk_alloc_queue(gfp_t);
struct request_queue *blk_alloc_queue_node(gfp_t, int);
extern void blk_put_queue(struct request_queue *);
extern void blk_set_queue_dying(struct request_queue *);
extern void blk_pm_runtime_init(struct request_queue *q, struct device *dev);
extern int blk_pre_runtime_suspend(struct request_queue *q);
extern void blk_post_runtime_suspend(struct request_queue *q, int err);
extern void blk_pre_runtime_resume(struct request_queue *q);
extern void blk_post_runtime_resume(struct request_queue *q, int err);
extern void blk_set_runtime_active(struct request_queue *q);
# 1293 "./include/linux/blkdev.h"
struct blk_plug {
struct list_head list;
struct list_head mq_list;
struct list_head cb_list;
};
struct blk_plug_cb;
typedef void (*blk_plug_cb_fn)(struct blk_plug_cb *, bool);
struct blk_plug_cb {
struct list_head list;
blk_plug_cb_fn callback;
void *data;
};
extern struct blk_plug_cb *blk_check_plugged(blk_plug_cb_fn unplug,
void *data, int size);
extern void blk_start_plug(struct blk_plug *);
extern void blk_finish_plug(struct blk_plug *);
extern void blk_flush_plug_list(struct blk_plug *, bool);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_flush_plug(struct task_struct *tsk)
{
struct blk_plug *plug = tsk->plug;
if (plug)
blk_flush_plug_list(plug, false);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_schedule_flush_plug(struct task_struct *tsk)
{
struct blk_plug *plug = tsk->plug;
if (plug)
blk_flush_plug_list(plug, true);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_needs_flush_plug(struct task_struct *tsk)
{
struct blk_plug *plug = tsk->plug;
return plug &&
(!list_empty(&plug->list) ||
!list_empty(&plug->mq_list) ||
!list_empty(&plug->cb_list));
}
extern int blk_queue_start_tag(struct request_queue *, struct request *);
extern struct request *blk_queue_find_tag(struct request_queue *, int);
extern void blk_queue_end_tag(struct request_queue *, struct request *);
extern int blk_queue_init_tags(struct request_queue *, int, struct blk_queue_tag *, int);
extern void blk_queue_free_tags(struct request_queue *);
extern int blk_queue_resize_tags(struct request_queue *, int);
extern void blk_queue_invalidate_tags(struct request_queue *);
extern struct blk_queue_tag *blk_init_tags(int, int);
extern void blk_free_tags(struct blk_queue_tag *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct request *blk_map_queue_find_tag(struct blk_queue_tag *bqt,
int tag)
{
if (__builtin_expect(!!(bqt == ((void *)0) || tag >= bqt->real_max_depth), 0))
return ((void *)0);
return bqt->tag_index[tag];
}
extern int blkdev_issue_flush(struct block_device *, gfp_t, sector_t *);
extern int blkdev_issue_write_same(struct block_device *bdev, sector_t sector,
sector_t nr_sects, gfp_t gfp_mask, struct page *page);
extern int blkdev_issue_discard(struct block_device *bdev, sector_t sector,
sector_t nr_sects, gfp_t gfp_mask, unsigned long flags);
extern int __blkdev_issue_discard(struct block_device *bdev, sector_t sector,
sector_t nr_sects, gfp_t gfp_mask, int flags,
struct bio **biop);
extern int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector,
sector_t nr_sects, gfp_t gfp_mask, struct bio **biop,
unsigned flags);
extern int blkdev_issue_zeroout(struct block_device *bdev, sector_t sector,
sector_t nr_sects, gfp_t gfp_mask, unsigned flags);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int sb_issue_discard(struct super_block *sb, sector_t block,
sector_t nr_blocks, gfp_t gfp_mask, unsigned long flags)
{
return blkdev_issue_discard(sb->s_bdev, block << (sb->s_blocksize_bits - 9),
nr_blocks << (sb->s_blocksize_bits - 9),
gfp_mask, flags);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int sb_issue_zeroout(struct super_block *sb, sector_t block,
sector_t nr_blocks, gfp_t gfp_mask)
{
return blkdev_issue_zeroout(sb->s_bdev,
block << (sb->s_blocksize_bits - 9),
nr_blocks << (sb->s_blocksize_bits - 9),
gfp_mask, 0);
}
extern int blk_verify_command(unsigned char *cmd, fmode_t has_write_perm);
enum blk_default_limits {
BLK_MAX_SEGMENTS = 128,
BLK_SAFE_MAX_SECTORS = 255,
BLK_DEF_MAX_SECTORS = 2560,
BLK_MAX_SEGMENT_SIZE = 65536,
BLK_SEG_BOUNDARY_MASK = 0xFFFFFFFFUL,
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long queue_segment_boundary(struct request_queue *q)
{
return q->limits.seg_boundary_mask;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long queue_virt_boundary(struct request_queue *q)
{
return q->limits.virt_boundary_mask;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int queue_max_sectors(struct request_queue *q)
{
return q->limits.max_sectors;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int queue_max_hw_sectors(struct request_queue *q)
{
return q->limits.max_hw_sectors;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned short queue_max_segments(struct request_queue *q)
{
return q->limits.max_segments;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned short queue_max_discard_segments(struct request_queue *q)
{
return q->limits.max_discard_segments;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int queue_max_segment_size(struct request_queue *q)
{
return q->limits.max_segment_size;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned queue_logical_block_size(struct request_queue *q)
{
int retval = 512;
if (q && q->limits.logical_block_size)
retval = q->limits.logical_block_size;
return retval;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int bdev_logical_block_size(struct block_device *bdev)
{
return queue_logical_block_size(bdev_get_queue(bdev));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int queue_physical_block_size(struct request_queue *q)
{
return q->limits.physical_block_size;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int bdev_physical_block_size(struct block_device *bdev)
{
return queue_physical_block_size(bdev_get_queue(bdev));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int queue_io_min(struct request_queue *q)
{
return q->limits.io_min;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bdev_io_min(struct block_device *bdev)
{
return queue_io_min(bdev_get_queue(bdev));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int queue_io_opt(struct request_queue *q)
{
return q->limits.io_opt;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bdev_io_opt(struct block_device *bdev)
{
return queue_io_opt(bdev_get_queue(bdev));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_alignment_offset(struct request_queue *q)
{
if (q->limits.misaligned)
return -1;
return q->limits.alignment_offset;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_limit_alignment_offset(struct queue_limits *lim, sector_t sector)
{
unsigned int granularity = ({ typeof(lim->physical_block_size) __UNIQUE_ID_max1_90 = (lim->physical_block_size); typeof(lim->io_min) __UNIQUE_ID_max2_91 = (lim->io_min); (void) (&__UNIQUE_ID_max1_90 == &__UNIQUE_ID_max2_91); __UNIQUE_ID_max1_90 > __UNIQUE_ID_max2_91 ? __UNIQUE_ID_max1_90 : __UNIQUE_ID_max2_91; });
unsigned int alignment = ({ unsigned int __r, __b = (granularity >> 9); if (!__builtin_constant_p(__b) || __b == 0 || (7 < 4 && (__b & (__b - 1)) != 0)) { __r = ({ register unsigned int __base asm("r4") = __b; register unsigned long long __n asm("r0") = sector; register unsigned long long __res asm("r2"); register unsigned int __rem asm("r1"); asm( ".ifnc " "%0" "," "r1" "; " ".ifnc " "%0" "r1" ",fpr11; " ".ifnc " "%0" "r1" ",r11fp; " ".ifnc " "%0" "r1" ",ipr12; " ".ifnc " "%0" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r0" "; " ".ifnc " "%2" "r0" ",fpr11; " ".ifnc " "%2" "r0" ",r11fp; " ".ifnc " "%2" "r0" ",ipr12; " ".ifnc " "%2" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r4" "; " ".ifnc " "%3" "r4" ",fpr11; " ".ifnc " "%3" "r4" ",r11fp; " ".ifnc " "%3" "r4" ",ipr12; " ".ifnc " "%3" "r4" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __do_div64" : "=r" (__rem), "=r" (__res) : "r" (__n), "r" (__base) : "ip", "lr", "cc"); sector = __res; __rem; }); } else if ((__b & (__b - 1)) == 0) { __r = sector; __r &= (__b - 1); sector /= __b; } else { unsigned long long __res, __x, __t, __m, __n = sector; unsigned int __c, __p, __z = 0; __r = __n; __p = 1 << ({ unsigned int __left = (__b), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m = (~0ULL / __b) * __p; __m += (((~0ULL % __b + 1) * __p) + __b - 1) / __b; __x = ~0ULL / __b * __b - 1; __res = (__m & 0xffffffff) * (__x & 0xffffffff); __res >>= 32; __res += (__m & 0xffffffff) * (__x >> 32); __t = __res; __res += (__x & 0xffffffff) * (__m >> 32); __t = (__res < __t) ? (1ULL << 32) : 0; __res = (__res >> 32) + __t; __res += (__m >> 32) * (__x >> 32); __res /= __p; if (~0ULL % (__b / (__b & -__b)) == 0) { __n /= (__b & -__b); __m = ~0ULL / (__b / (__b & -__b)); __p = 1; __c = 1; } else if (__res != __x / __b) { __c = 1; __m = (~0ULL / __b) * __p; __m += ((~0ULL % __b + 1) * __p) / __b; } else { unsigned int __bits = -(__m & -__m); __bits |= __m >> 32; __bits = (~__bits) << 1; if (!__bits) { __p /= (__m & -__m); __m /= (__m & -__m); } else { __p >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); } __c = 0; } if (!__c) { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "=&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { __res = __m; asm ( "umlal %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "cmn %Q0, %Q1\n\t" "adcs %R0, %R0, %R1\n\t" "adc %Q0, %3, #0" : "=&r" (__res) : "r" (__m), "r" (__n), "r" (__z) : "cc" ); } if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { asm ( "umlal %R0, %Q0, %R1, %Q2\n\t" "umlal %R0, %Q0, %Q1, %R2\n\t" "mov %R0, #0\n\t" "umlal %Q0, %R0, %R1, %R2" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umlal %R0, %Q0, %R2, %Q3\n\t" "umlal %R0, %1, %Q2, %R3\n\t" "mov %R0, #0\n\t" "adds %Q0, %1, %Q0\n\t" "adc %R0, %R0, #0\n\t" "umlal %Q0, %R0, %R2, %R3" : "+&r" (__res), "+&r" (__z) : "r" (__m), "r" (__n) : "cc" ); } __res /= __p; { unsigned int __res0 = __res; unsigned int __b0 = __b; __r -= __res0 * __b0; } sector = __res; } __r; }) << 9;
return (granularity + lim->alignment_offset - alignment) % granularity;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bdev_alignment_offset(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (q->limits.misaligned)
return -1;
if (bdev != bdev->bd_contains)
return bdev->bd_part->alignment_offset;
return q->limits.alignment_offset;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_discard_alignment(struct request_queue *q)
{
if (q->limits.discard_misaligned)
return -1;
return q->limits.discard_alignment;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_limit_discard_alignment(struct queue_limits *lim, sector_t sector)
{
unsigned int alignment, granularity, offset;
if (!lim->max_discard_sectors)
return 0;
alignment = lim->discard_alignment >> 9;
granularity = lim->discard_granularity >> 9;
if (!granularity)
return 0;
offset = ({ unsigned int __r, __b = (granularity); if (!__builtin_constant_p(__b) || __b == 0 || (7 < 4 && (__b & (__b - 1)) != 0)) { __r = ({ register unsigned int __base asm("r4") = __b; register unsigned long long __n asm("r0") = sector; register unsigned long long __res asm("r2"); register unsigned int __rem asm("r1"); asm( ".ifnc " "%0" "," "r1" "; " ".ifnc " "%0" "r1" ",fpr11; " ".ifnc " "%0" "r1" ",r11fp; " ".ifnc " "%0" "r1" ",ipr12; " ".ifnc " "%0" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%2" "," "r0" "; " ".ifnc " "%2" "r0" ",fpr11; " ".ifnc " "%2" "r0" ",r11fp; " ".ifnc " "%2" "r0" ",ipr12; " ".ifnc " "%2" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" ".ifnc " "%3" "," "r4" "; " ".ifnc " "%3" "r4" ",fpr11; " ".ifnc " "%3" "r4" ",r11fp; " ".ifnc " "%3" "r4" ",ipr12; " ".ifnc " "%3" "r4" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t" "bl __do_div64" : "=r" (__rem), "=r" (__res) : "r" (__n), "r" (__base) : "ip", "lr", "cc"); sector = __res; __rem; }); } else if ((__b & (__b - 1)) == 0) { __r = sector; __r &= (__b - 1); sector /= __b; } else { unsigned long long __res, __x, __t, __m, __n = sector; unsigned int __c, __p, __z = 0; __r = __n; __p = 1 << ({ unsigned int __left = (__b), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m = (~0ULL / __b) * __p; __m += (((~0ULL % __b + 1) * __p) + __b - 1) / __b; __x = ~0ULL / __b * __b - 1; __res = (__m & 0xffffffff) * (__x & 0xffffffff); __res >>= 32; __res += (__m & 0xffffffff) * (__x >> 32); __t = __res; __res += (__x & 0xffffffff) * (__m >> 32); __t = (__res < __t) ? (1ULL << 32) : 0; __res = (__res >> 32) + __t; __res += (__m >> 32) * (__x >> 32); __res /= __p; if (~0ULL % (__b / (__b & -__b)) == 0) { __n /= (__b & -__b); __m = ~0ULL / (__b / (__b & -__b)); __p = 1; __c = 1; } else if (__res != __x / __b) { __c = 1; __m = (~0ULL / __b) * __p; __m += ((~0ULL % __b + 1) * __p) / __b; } else { unsigned int __bits = -(__m & -__m); __bits |= __m >> 32; __bits = (~__bits) << 1; if (!__bits) { __p /= (__m & -__m); __m /= (__m & -__m); } else { __p >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); } __c = 0; } if (!__c) { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "=&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { __res = __m; asm ( "umlal %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "cmn %Q0, %Q1\n\t" "adcs %R0, %R0, %R1\n\t" "adc %Q0, %3, #0" : "=&r" (__res) : "r" (__m), "r" (__n), "r" (__z) : "cc" ); } if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { asm ( "umlal %R0, %Q0, %R1, %Q2\n\t" "umlal %R0, %Q0, %Q1, %R2\n\t" "mov %R0, #0\n\t" "umlal %Q0, %R0, %R1, %R2" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umlal %R0, %Q0, %R2, %Q3\n\t" "umlal %R0, %1, %Q2, %R3\n\t" "mov %R0, #0\n\t" "adds %Q0, %1, %Q0\n\t" "adc %R0, %R0, #0\n\t" "umlal %Q0, %R0, %R2, %R3" : "+&r" (__res), "+&r" (__z) : "r" (__m), "r" (__n) : "cc" ); } __res /= __p; { unsigned int __res0 = __res; unsigned int __b0 = __b; __r -= __res0 * __b0; } sector = __res; } __r; });
offset = (granularity + alignment - offset) % granularity;
return offset << 9;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int bdev_discard_alignment(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (bdev != bdev->bd_contains)
return bdev->bd_part->discard_alignment;
return q->limits.discard_alignment;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int bdev_write_same(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (q)
return q->limits.max_write_same_sectors;
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int bdev_write_zeroes_sectors(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (q)
return q->limits.max_write_zeroes_sectors;
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) enum blk_zoned_model bdev_zoned_model(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (q)
return blk_queue_zoned_model(q);
return BLK_ZONED_NONE;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bdev_is_zoned(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (q)
return blk_queue_is_zoned(q);
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int bdev_zone_sectors(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (q)
return blk_queue_zone_sectors(q);
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int queue_dma_alignment(struct request_queue *q)
{
return q ? q->dma_alignment : 511;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blk_rq_aligned(struct request_queue *q, unsigned long addr,
unsigned int len)
{
unsigned int alignment = queue_dma_alignment(q) | q->dma_pad_mask;
return !(addr & alignment) && !(len & alignment);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int blksize_bits(unsigned int size)
{
unsigned int bits = 8;
do {
bits++;
size >>= 1;
} while (size > 256);
return bits;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned int block_size(struct block_device *bdev)
{
return bdev->bd_block_size;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool queue_flush_queueable(struct request_queue *q)
{
return !test_bit(22, &q->queue_flags);
}
typedef struct {struct page *v;} Sector;
unsigned char *read_dev_sector(struct block_device *, sector_t, Sector *);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void put_dev_sector(Sector p)
{
put_page(p.v);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool __bvec_gap_to_prev(struct request_queue *q,
struct bio_vec *bprv, unsigned int offset)
{
return offset ||
((bprv->bv_offset + bprv->bv_len) & queue_virt_boundary(q));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bvec_gap_to_prev(struct request_queue *q,
struct bio_vec *bprv, unsigned int offset)
{
if (!queue_virt_boundary(q))
return false;
return __bvec_gap_to_prev(q, bprv, offset);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bios_segs_mergeable(struct request_queue *q,
struct bio *prev, struct bio_vec *prev_last_bv,
struct bio_vec *next_first_bv)
{
if (!(((((((phys_addr_t)(((unsigned long)((((prev_last_bv))->bv_page) - mem_map) + ((unsigned long)((0x00000000UL) >> 12)))) << 12)) + (unsigned long) ((prev_last_bv))->bv_offset) + (prev_last_bv)->bv_len) == ((((phys_addr_t)(((unsigned long)((((next_first_bv))->bv_page) - mem_map) + ((unsigned long)((0x00000000UL) >> 12)))) << 12)) + (unsigned long) ((next_first_bv))->bv_offset)) && (!(XEN_NATIVE != XEN_NATIVE) || xen_biovec_phys_mergeable(prev_last_bv, next_first_bv))))
return false;
if (!(((((((phys_addr_t)(((unsigned long)((((prev_last_bv))->bv_page) - mem_map) + ((unsigned long)((0x00000000UL) >> 12)))) << 12)) + (unsigned long) ((prev_last_bv))->bv_offset)) | (queue_segment_boundary((q)))) == (((((((phys_addr_t)(((unsigned long)((((next_first_bv))->bv_page) - mem_map) + ((unsigned long)((0x00000000UL) >> 12)))) << 12)) + (unsigned long) ((next_first_bv))->bv_offset) + (next_first_bv)->bv_len) - 1) | (queue_segment_boundary((q))))))
return false;
if (prev->bi_seg_back_size + next_first_bv->bv_len >
queue_max_segment_size(q))
return false;
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool bio_will_gap(struct request_queue *q,
struct request *prev_rq,
struct bio *prev,
struct bio *next)
{
if (bio_has_data(prev) && queue_virt_boundary(q)) {
struct bio_vec pb, nb;
if (prev_rq)
bio_get_first_bvec(prev_rq->bio, &pb);
else
bio_get_first_bvec(prev, &pb);
if (pb.bv_offset)
return true;
# 1720 "./include/linux/blkdev.h"
bio_get_last_bvec(prev, &pb);
bio_get_first_bvec(next, &nb);
if (!bios_segs_mergeable(q, prev, &pb, &nb))
return __bvec_gap_to_prev(q, &pb, nb.bv_offset);
}
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool req_gap_back_merge(struct request *req, struct bio *bio)
{
return bio_will_gap(req->q, req, req->biotail, bio);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool req_gap_front_merge(struct request *req, struct bio *bio)
{
return bio_will_gap(req->q, ((void *)0), bio, req->bio);
}
int kblockd_schedule_work(struct work_struct *work);
int kblockd_schedule_work_on(int cpu, struct work_struct *work);
int kblockd_schedule_delayed_work(struct delayed_work *dwork, unsigned long delay);
int kblockd_schedule_delayed_work_on(int cpu, struct delayed_work *dwork, unsigned long delay);
int kblockd_mod_delayed_work_on(int cpu, struct delayed_work *dwork, unsigned long delay);
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_start_time_ns(struct request *req)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
req->start_time_ns = sched_clock();
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void set_io_start_time_ns(struct request *req)
{
do { __preempt_count_add(1); __asm__ __volatile__("": : :"memory"); } while (0);
req->io_start_time_ns = sched_clock();
do { __asm__ __volatile__("": : :"memory"); if (__builtin_expect(!!(__preempt_count_dec_and_test()), 0)) preempt_schedule(); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) uint64_t rq_start_time_ns(struct request *req)
{
return req->start_time_ns;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) uint64_t rq_io_start_time_ns(struct request *req)
{
return req->io_start_time_ns;
}
# 1885 "./include/linux/blkdev.h"
struct bio;
struct block_device;
struct gendisk;
struct blk_integrity;
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blk_integrity_rq(struct request *rq)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blk_rq_count_integrity_sg(struct request_queue *q,
struct bio *b)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blk_rq_map_integrity_sg(struct request_queue *q,
struct bio *b,
struct scatterlist *s)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct blk_integrity *bdev_get_integrity(struct block_device *b)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct blk_integrity *blk_get_integrity(struct gendisk *disk)
{
return ((void *)0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int blk_integrity_compare(struct gendisk *a, struct gendisk *b)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_integrity_register(struct gendisk *d,
struct blk_integrity *b)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_integrity_unregister(struct gendisk *d)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void blk_queue_max_integrity_segments(struct request_queue *q,
unsigned int segs)
{
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned short queue_max_integrity_segments(struct request_queue *q)
{
return 0;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_integrity_merge_rq(struct request_queue *rq,
struct request *r1,
struct request *r2)
{
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool blk_integrity_merge_bio(struct request_queue *rq,
struct request *r,
struct bio *b)
{
return true;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool integrity_req_gap_back_merge(struct request *req,
struct bio *next)
{
return false;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) bool integrity_req_gap_front_merge(struct request *req,
struct bio *bio)
{
return false;
}
struct block_device_operations {
int (*open) (struct block_device *, fmode_t);
void (*release) (struct gendisk *, fmode_t);
int (*rw_page)(struct block_device *, sector_t, struct page *, bool);
int (*ioctl) (struct block_device *, fmode_t, unsigned, unsigned long);
int (*compat_ioctl) (struct block_device *, fmode_t, unsigned, unsigned long);
unsigned int (*check_events) (struct gendisk *disk,
unsigned int clearing);
int (*media_changed) (struct gendisk *);
void (*unlock_native_capacity) (struct gendisk *);
int (*revalidate_disk) (struct gendisk *);
int (*getgeo)(struct block_device *, struct hd_geometry *);
void (*swap_slot_free_notify) (struct block_device *, unsigned long);
struct module *owner;
const struct pr_ops *pr_ops;
};
extern int __blkdev_driver_ioctl(struct block_device *, fmode_t, unsigned int,
unsigned long);
extern int bdev_read_page(struct block_device *, sector_t, struct page *);
extern int bdev_write_page(struct block_device *, sector_t, struct page *,
struct writeback_control *);
# 14 "drivers/md/dm-exception-store.h" 2
# 1 "./include/linux/device-mapper.h" 1
# 16 "./include/linux/device-mapper.h"
struct dm_dev;
struct dm_target;
struct dm_table;
struct mapped_device;
struct bio_vec;
enum dm_queue_mode {
DM_TYPE_NONE = 0,
DM_TYPE_BIO_BASED = 1,
DM_TYPE_REQUEST_BASED = 2,
DM_TYPE_MQ_REQUEST_BASED = 3,
DM_TYPE_DAX_BIO_BASED = 4,
};
typedef enum { STATUSTYPE_INFO, STATUSTYPE_TABLE } status_type_t;
union map_info {
void *ptr;
};
typedef int (*dm_ctr_fn) (struct dm_target *target,
unsigned int argc, char **argv);
typedef void (*dm_dtr_fn) (struct dm_target *ti);
# 59 "./include/linux/device-mapper.h"
typedef int (*dm_map_fn) (struct dm_target *ti, struct bio *bio);
typedef int (*dm_clone_and_map_request_fn) (struct dm_target *ti,
struct request *rq,
union map_info *map_context,
struct request **clone);
typedef void (*dm_release_clone_request_fn) (struct request *clone);
# 74 "./include/linux/device-mapper.h"
typedef int (*dm_endio_fn) (struct dm_target *ti,
struct bio *bio, blk_status_t *error);
typedef int (*dm_request_endio_fn) (struct dm_target *ti,
struct request *clone, blk_status_t error,
union map_info *map_context);
typedef void (*dm_presuspend_fn) (struct dm_target *ti);
typedef void (*dm_presuspend_undo_fn) (struct dm_target *ti);
typedef void (*dm_postsuspend_fn) (struct dm_target *ti);
typedef int (*dm_preresume_fn) (struct dm_target *ti);
typedef void (*dm_resume_fn) (struct dm_target *ti);
typedef void (*dm_status_fn) (struct dm_target *ti, status_type_t status_type,
unsigned status_flags, char *result, unsigned maxlen);
typedef int (*dm_message_fn) (struct dm_target *ti, unsigned argc, char **argv);
typedef int (*dm_prepare_ioctl_fn) (struct dm_target *ti,
struct block_device **bdev, fmode_t *mode);
# 104 "./include/linux/device-mapper.h"
typedef int (*iterate_devices_callout_fn) (struct dm_target *ti,
struct dm_dev *dev,
sector_t start, sector_t len,
void *data);
typedef int (*dm_iterate_devices_fn) (struct dm_target *ti,
iterate_devices_callout_fn fn,
void *data);
typedef void (*dm_io_hints_fn) (struct dm_target *ti,
struct queue_limits *limits);
typedef int (*dm_busy_fn) (struct dm_target *ti);
typedef long (*dm_dax_direct_access_fn) (struct dm_target *ti, unsigned long pgoff,
long nr_pages, void **kaddr, pfn_t *pfn);
typedef size_t (*dm_dax_copy_from_iter_fn)(struct dm_target *ti, unsigned long pgoff,
void *addr, size_t bytes, struct iov_iter *i);
void dm_error(const char *message);
struct dm_dev {
struct block_device *bdev;
struct dax_device *dax_dev;
fmode_t mode;
char name[16];
};
dev_t dm_get_dev_t(const char *path);
int dm_get_device(struct dm_target *ti, const char *path, fmode_t mode,
struct dm_dev **result);
void dm_put_device(struct dm_target *ti, struct dm_dev *d);
struct target_type {
uint64_t features;
const char *name;
struct module *module;
unsigned version[3];
dm_ctr_fn ctr;
dm_dtr_fn dtr;
dm_map_fn map;
dm_clone_and_map_request_fn clone_and_map_rq;
dm_release_clone_request_fn release_clone_rq;
dm_endio_fn end_io;
dm_request_endio_fn rq_end_io;
dm_presuspend_fn presuspend;
dm_presuspend_undo_fn presuspend_undo;
dm_postsuspend_fn postsuspend;
dm_preresume_fn preresume;
dm_resume_fn resume;
dm_status_fn status;
dm_message_fn message;
dm_prepare_ioctl_fn prepare_ioctl;
dm_busy_fn busy;
dm_iterate_devices_fn iterate_devices;
dm_io_hints_fn io_hints;
dm_dax_direct_access_fn direct_access;
dm_dax_copy_from_iter_fn dax_copy_from_iter;
struct list_head list;
};
# 229 "./include/linux/device-mapper.h"
typedef unsigned (*dm_num_write_bios_fn) (struct dm_target *ti, struct bio *bio);
# 249 "./include/linux/device-mapper.h"
struct dm_target {
struct dm_table *table;
struct target_type *type;
sector_t begin;
sector_t len;
uint32_t max_io_len;
# 268 "./include/linux/device-mapper.h"
unsigned num_flush_bios;
unsigned num_discard_bios;
unsigned num_write_same_bios;
unsigned num_write_zeroes_bios;
unsigned per_io_data_size;
dm_num_write_bios_fn num_write_bios;
void *private;
char *error;
bool flush_supported:1;
bool discards_supported:1;
bool split_discard_bios:1;
bool may_passthrough_inline_crypto:1;
};
struct dm_target_callbacks {
struct list_head list;
int (*congested_fn) (struct dm_target_callbacks *, int);
};
# 345 "./include/linux/device-mapper.h"
struct dm_target_io {
struct dm_io *io;
struct dm_target *ti;
unsigned target_bio_nr;
unsigned *len_ptr;
struct bio clone;
};
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void *dm_per_bio_data(struct bio *bio, size_t data_size)
{
return (char *)bio - __builtin_offsetof(struct dm_target_io, clone) - data_size;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) struct bio *dm_bio_from_per_bio_data(void *data, size_t data_size)
{
return (struct bio *)((char *)data + data_size + __builtin_offsetof(struct dm_target_io, clone));
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned dm_bio_get_target_bio_nr(const struct bio *bio)
{
return ({ void *__mptr = (void *)(bio); do { bool __cond = !(!(!__builtin_types_compatible_p(typeof(*(bio)), typeof(((struct dm_target_io *)0)->clone)) && !__builtin_types_compatible_p(typeof(*(bio)), typeof(void)))); extern void __compiletime_assert_92(void) ; if (__cond) __compiletime_assert_92(); do { ((void)sizeof(char[1 - 2 * __cond])); } while (0); } while (0); ((struct dm_target_io *)(__mptr - __builtin_offsetof(struct dm_target_io, clone))); })->target_bio_nr;
}
int dm_register_target(struct target_type *t);
void dm_unregister_target(struct target_type *t);
struct dm_arg_set {
unsigned argc;
char **argv;
};
struct dm_arg {
unsigned min;
unsigned max;
char *error;
};
int dm_read_arg(const struct dm_arg *arg, struct dm_arg_set *arg_set,
unsigned *value, char **error);
int dm_read_arg_group(const struct dm_arg *arg, struct dm_arg_set *arg_set,
unsigned *num_args, char **error);
const char *dm_shift_arg(struct dm_arg_set *as);
void dm_consume_args(struct dm_arg_set *as, unsigned num_args);
# 423 "./include/linux/device-mapper.h"
int dm_create(int minor, struct mapped_device **md);
struct mapped_device *dm_get_md(dev_t dev);
void dm_get(struct mapped_device *md);
int dm_hold(struct mapped_device *md);
void dm_put(struct mapped_device *md);
void dm_set_mdptr(struct mapped_device *md, void *ptr);
void *dm_get_mdptr(struct mapped_device *md);
int dm_suspend(struct mapped_device *md, unsigned suspend_flags);
int dm_resume(struct mapped_device *md);
uint32_t dm_get_event_nr(struct mapped_device *md);
int dm_wait_event(struct mapped_device *md, int event_nr);
uint32_t dm_next_uevent_seq(struct mapped_device *md);
void dm_uevent_add(struct mapped_device *md, struct list_head *elist);
const char *dm_device_name(struct mapped_device *md);
int dm_copy_name_and_uuid(struct mapped_device *md, char *name, char *uuid);
struct gendisk *dm_disk(struct mapped_device *md);
int dm_suspended(struct dm_target *ti);
int dm_noflush_suspending(struct dm_target *ti);
void dm_accept_partial_bio(struct bio *bio, unsigned n_sectors);
void dm_remap_zone_report(struct dm_target *ti, struct bio *bio,
sector_t start);
union map_info *dm_get_rq_mapinfo(struct request *rq);
struct queue_limits *dm_get_queue_limits(struct mapped_device *md);
int dm_get_geometry(struct mapped_device *md, struct hd_geometry *geo);
int dm_set_geometry(struct mapped_device *md, struct hd_geometry *geo);
# 481 "./include/linux/device-mapper.h"
int dm_table_create(struct dm_table **result, fmode_t mode,
unsigned num_targets, struct mapped_device *md);
int dm_table_add_target(struct dm_table *t, const char *type,
sector_t start, sector_t len, char *params);
void dm_table_add_target_callbacks(struct dm_table *t, struct dm_target_callbacks *cb);
void dm_table_set_type(struct dm_table *t, enum dm_queue_mode type);
int dm_table_complete(struct dm_table *t);
int __attribute__((warn_unused_result)) dm_set_target_max_io_len(struct dm_target *ti, sector_t len);
struct dm_table *dm_get_live_table(struct mapped_device *md, int *srcu_idx);
void dm_put_live_table(struct mapped_device *md, int srcu_idx);
void dm_sync_table(struct mapped_device *md);
sector_t dm_table_get_size(struct dm_table *t);
unsigned int dm_table_get_num_targets(struct dm_table *t);
fmode_t dm_table_get_mode(struct dm_table *t);
struct mapped_device *dm_table_get_md(struct dm_table *t);
void dm_table_event(struct dm_table *t);
void dm_table_run_md_queue_async(struct dm_table *t);
struct dm_table *dm_swap_table(struct mapped_device *md,
struct dm_table *t);
void *dm_vcalloc(unsigned long nmemb, unsigned long elem_size);
# 639 "./include/linux/device-mapper.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) sector_t to_sector(unsigned long long n)
{
return (n >> 9);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned long to_bytes(sector_t n)
{
return (n << 9);
}
# 15 "drivers/md/dm-exception-store.h" 2
typedef sector_t chunk_t;
# 29 "drivers/md/dm-exception-store.h"
struct dm_exception {
struct list_head hash_list;
chunk_t old_chunk;
chunk_t new_chunk;
};
struct dm_exception_store;
struct dm_exception_store_type {
const char *name;
struct module *module;
int (*ctr) (struct dm_exception_store *store, char *options);
void (*dtr) (struct dm_exception_store *store);
int (*read_metadata) (struct dm_exception_store *store,
int (*callback)(void *callback_context,
chunk_t old, chunk_t new),
void *callback_context);
int (*prepare_exception) (struct dm_exception_store *store,
struct dm_exception *e);
void (*commit_exception) (struct dm_exception_store *store,
struct dm_exception *e, int valid,
void (*callback) (void *, int success),
void *callback_context);
# 84 "drivers/md/dm-exception-store.h"
int (*prepare_merge) (struct dm_exception_store *store,
chunk_t *last_old_chunk, chunk_t *last_new_chunk);
int (*commit_merge) (struct dm_exception_store *store, int nr_merged);
void (*drop_snapshot) (struct dm_exception_store *store);
unsigned (*status) (struct dm_exception_store *store,
status_type_t status, char *result,
unsigned maxlen);
void (*usage) (struct dm_exception_store *store,
sector_t *total_sectors, sector_t *sectors_allocated,
sector_t *metadata_sectors);
struct list_head list;
};
struct dm_snapshot;
struct dm_exception_store {
struct dm_exception_store_type *type;
struct dm_snapshot *snap;
unsigned chunk_size;
unsigned chunk_mask;
unsigned chunk_shift;
void *context;
bool userspace_supports_overflow;
};
struct dm_dev *dm_snap_origin(struct dm_snapshot *snap);
struct dm_dev *dm_snap_cow(struct dm_snapshot *snap);
# 142 "drivers/md/dm-exception-store.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) chunk_t dm_chunk_number(chunk_t chunk)
{
return chunk & (chunk_t)((1ULL << 56) - 1ULL);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned dm_consecutive_chunk_count(struct dm_exception *e)
{
return e->new_chunk >> 56;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dm_consecutive_chunk_count_inc(struct dm_exception *e)
{
e->new_chunk += (1ULL << 56);
do { if (__builtin_expect(!!(!dm_consecutive_chunk_count(e)), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"drivers/md/dm-exception-store.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "156" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) void dm_consecutive_chunk_count_dec(struct dm_exception *e)
{
do { if (__builtin_expect(!!(!dm_consecutive_chunk_count(e)), 0)) do { asm volatile("1:\t" ".long " "((0xe7f001f2) & 0xFFFFFFFF)" "\n\t" "\n" ".pushsection .rodata.str, \"aMS\", %progbits, 1\n" "2:\t.asciz " "\"drivers/md/dm-exception-store.h\"" "\n" ".popsection\n" ".pushsection __bug_table,\"aw\"\n" ".align 2\n" "3:\t.word 1b, 2b\n" "\t.hword " "161" ", 0\n" ".popsection"); do { ; do { } while (1); } while (0); } while (0); } while (0);
e->new_chunk -= (1ULL << 56);
}
# 192 "drivers/md/dm-exception-store.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) sector_t get_dev_size(struct block_device *bdev)
{
return i_size_read(bdev->bd_inode) >> 9;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) chunk_t sector_to_chunk(struct dm_exception_store *store,
sector_t sector)
{
return sector >> store->chunk_shift;
}
int dm_exception_store_type_register(struct dm_exception_store_type *type);
int dm_exception_store_type_unregister(struct dm_exception_store_type *type);
int dm_exception_store_set_chunk_size(struct dm_exception_store *store,
unsigned chunk_size,
char **error);
int dm_exception_store_create(struct dm_target *ti, int argc, char **argv,
struct dm_snapshot *snap,
unsigned *args_used,
struct dm_exception_store **store);
void dm_exception_store_destroy(struct dm_exception_store *store);
int dm_exception_store_init(void);
void dm_exception_store_exit(void);
int dm_persistent_snapshot_init(void);
void dm_persistent_snapshot_exit(void);
int dm_transient_snapshot_init(void);
void dm_transient_snapshot_exit(void);
# 9 "drivers/md/dm-snap-persistent.c" 2
# 1 "./include/linux/ctype.h" 1
# 19 "./include/linux/ctype.h"
extern const unsigned char _ctype[];
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int isdigit(int c)
{
return '0' <= c && c <= '9';
}
# 42 "./include/linux/ctype.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned char __tolower(unsigned char c)
{
if ((((_ctype[(int)(unsigned char)(c)])&(0x01)) != 0))
c -= 'A'-'a';
return c;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) unsigned char __toupper(unsigned char c)
{
if ((((_ctype[(int)(unsigned char)(c)])&(0x02)) != 0))
c -= 'a'-'A';
return c;
}
# 63 "./include/linux/ctype.h"
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) char _tolower(const char c)
{
return c | 0x20;
}
static inline __attribute__((always_inline, unused)) __attribute__((no_instrument_function)) int isodigit(const char c)
{
return c >= '0' && c <= '7';
}
# 11 "drivers/md/dm-snap-persistent.c" 2
# 1 "./include/linux/dm-io.h" 1
# 17 "./include/linux/dm-io.h"
struct dm_io_region {
struct block_device *bdev;
sector_t sector;
sector_t count;
};
struct page_list {
struct page_list *next;
struct page *page;
};
typedef void (*io_notify_fn)(unsigned long error, void *context);
enum dm_io_mem_type {
DM_IO_PAGE_LIST,
DM_IO_BIO,
DM_IO_VMA,
DM_IO_KMEM,
};
struct dm_io_memory {
enum dm_io_mem_type type;
unsigned offset;
union {
struct page_list *pl;
struct bio *bio;
void *vma;
void *addr;
} ptr;
};
struct dm_io_notify {
io_notify_fn fn;
void *context;
};
struct dm_io_client;
struct dm_io_request {
int bi_op;
int bi_op_flags;
struct dm_io_memory mem;
struct dm_io_notify notify;
struct dm_io_client *client;
};
struct dm_io_client *dm_io_client_create(void);
void dm_io_client_destroy(struct dm_io_client *client);
int dm_io(struct dm_io_request *io_req, unsigned num_regions,
struct dm_io_region *region, unsigned long *sync_error_bits);
# 17 "drivers/md/dm-snap-persistent.c" 2
# 1 "drivers/md/dm-bufio.h" 1
# 17 "drivers/md/dm-bufio.h"
struct dm_bufio_client;
struct dm_buffer;
struct dm_bufio_client *
dm_bufio_client_create(struct block_device *bdev, unsigned block_size,
unsigned reserved_buffers, unsigned aux_size,
void (*alloc_callback)(struct dm_buffer *),
void (*write_callback)(struct dm_buffer *));
void dm_bufio_client_destroy(struct dm_bufio_client *c);
void dm_bufio_set_sector_offset(struct dm_bufio_client *c, sector_t start);
# 55 "drivers/md/dm-bufio.h"
void *dm_bufio_read(struct dm_bufio_client *c, sector_t block,
struct dm_buffer **bp);
void *dm_bufio_get(struct dm_bufio_client *c, sector_t block,
struct dm_buffer **bp);
void *dm_bufio_new(struct dm_bufio_client *c, sector_t block,
struct dm_buffer **bp);
void dm_bufio_prefetch(struct dm_bufio_client *c,
sector_t block, unsigned n_blocks);
void dm_bufio_release(struct dm_buffer *b);
# 94 "drivers/md/dm-bufio.h"
void dm_bufio_mark_buffer_dirty(struct dm_buffer *b);
void dm_bufio_mark_partial_buffer_dirty(struct dm_buffer *b,
unsigned start, unsigned end);
void dm_bufio_write_dirty_buffers_async(struct dm_bufio_client *c);
int dm_bufio_write_dirty_buffers(struct dm_bufio_client *c);
int dm_bufio_issue_flush(struct dm_bufio_client *c);
void dm_bufio_release_move(struct dm_buffer *b, sector_t new_block);
void dm_bufio_forget(struct dm_bufio_client *c, sector_t block);
void dm_bufio_set_minimum_buffers(struct dm_bufio_client *c, unsigned n);
unsigned dm_bufio_get_block_size(struct dm_bufio_client *c);
sector_t dm_bufio_get_device_size(struct dm_bufio_client *c);
sector_t dm_bufio_get_block_number(struct dm_buffer *b);
void *dm_bufio_get_block_data(struct dm_buffer *b);
void *dm_bufio_get_aux_data(struct dm_buffer *b);
struct dm_bufio_client *dm_bufio_get_client(struct dm_buffer *b);
# 18 "drivers/md/dm-snap-persistent.c" 2
# 65 "drivers/md/dm-snap-persistent.c"
struct disk_header {
__le32 magic;
__le32 valid;
__le32 version;
__le32 chunk_size;
} __attribute__((packed));
struct disk_exception {
__le64 old_chunk;
__le64 new_chunk;
} __attribute__((packed));
struct core_exception {
uint64_t old_chunk;
uint64_t new_chunk;
};
struct commit_callback {
void (*callback)(void *, int success);
void *context;
};
struct pstore {
struct dm_exception_store *store;
int version;
int valid;
uint32_t exceptions_per_area;
void *area;
void *zero_area;
void *header_area;
chunk_t current_area;
# 151 "drivers/md/dm-snap-persistent.c"
chunk_t next_free;
uint32_t current_committed;
atomic_t pending_count;
uint32_t callback_count;
struct commit_callback *callbacks;
struct dm_io_client *io_client;
struct workqueue_struct *metadata_wq;
};
void skip_metadata(struct pstore *ps)
{
uint32_t stride = ps->exceptions_per_area + 1;
chunk_t next_free = ps->next_free;
if (({ unsigned int __r, __b = (stride);
if (!__builtin_constant_p(__b) || __b == 0 || (7 < 4 && (__b & (__b - 1)) != 0))
{ __r = ({ register unsigned int __base asm("r4") = __b;
register unsigned long long __n asm("r0") = next_free;
register unsigned long long __res asm("r2");
register unsigned int __rem asm("r1");
asm(
".ifnc " "%0" "," "r1" "; " ".ifnc " "%0" "r1" ",fpr11; " ".ifnc " "%0" "r1" ",r11fp; " ".ifnc " "%0" "r1" ",ipr12; " ".ifnc " "%0" "r1" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t"
".ifnc " "%1" "," "r2" "; " ".ifnc " "%1" "r2" ",fpr11; " ".ifnc " "%1" "r2" ",r11fp; " ".ifnc " "%1" "r2" ",ipr12; " ".ifnc " "%1" "r2" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t"
".ifnc " "%2" "," "r0" "; " ".ifnc " "%2" "r0" ",fpr11; " ".ifnc " "%2" "r0" ",r11fp; " ".ifnc " "%2" "r0" ",ipr12; " ".ifnc " "%2" "r0" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t"
".ifnc " "%3" "," "r4" "; " ".ifnc " "%3" "r4" ",fpr11; " ".ifnc " "%3" "r4" ",r11fp; " ".ifnc " "%3" "r4" ",ipr12; " ".ifnc " "%3" "r4" ",r12ip; " ".err; " ".endif; " ".endif; " ".endif; " ".endif; " ".endif\n\t"
"bl __do_div64"
: "=r" (__rem), "=r" (__res)
: "r"(__n), "r" (__base)
: "ip", "lr", "cc");
next_free = __res;
__rem; }); }
else if ((__b & (__b - 1)) == 0)
{ __r = next_free; __r &= (__b - 1); next_free /= __b; }
else
{ unsigned long long __res, __x, __t, __m, __n = next_free;
unsigned int __c, __p, __z = 0;
__r = __n;
__p = 1 << ({ unsigned int __left = (__b), __nr = 0;
if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00)
__nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m = (~0ULL / __b) * __p; __m += (((~0ULL % __b + 1) * __p) + __b - 1) / __b; __x = ~0ULL / __b * __b - 1; __res = (__m & 0xffffffff) * (__x & 0xffffffff); __res >>= 32; __res += (__m & 0xffffffff) * (__x >> 32); __t = __res; __res += (__x & 0xffffffff) * (__m >> 32); __t = (__res < __t) ? (1ULL << 32) : 0; __res = (__res >> 32) + __t; __res += (__m >> 32) * (__x >> 32); __res /= __p; if (~0ULL % (__b / (__b & -__b)) == 0) { __n /= (__b & -__b); __m = ~0ULL / (__b / (__b & -__b)); __p = 1; __c = 1; } else if (__res != __x / __b) { __c = 1; __m = (~0ULL / __b) * __p; __m += ((~0ULL % __b + 1) * __p) / __b; } else { unsigned int __bits = -(__m & -__m); __bits |= __m >> 32; __bits = (~__bits) << 1; if (!__bits) { __p /= (__m & -__m); __m /= (__m & -__m); } else { __p >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); __m >>= ({ unsigned int __left = (__bits), __nr = 0; if (__left & 0xffff0000) __nr += 16, __left >>= 16; if (__left & 0x0000ff00) __nr += 8, __left >>= 8; if (__left & 0x000000f0) __nr += 4, __left >>= 4; if (__left & 0x0000000c) __nr += 2, __left >>= 2; if (__left & 0x00000002) __nr += 1; __nr; }); } __c = 0; } if (!__c) { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "=&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { __res = __m; asm ( "umlal %Q0, %R0, %Q1, %Q2\n\t" "mov %Q0, #0" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umull %Q0, %R0, %Q1, %Q2\n\t" "cmn %Q0, %Q1\n\t" "adcs %R0, %R0, %R1\n\t" "adc %Q0, %3, #0" : "=&r" (__res) : "r" (__m), "r" (__n), "r" (__z) : "cc" ); } if (!(__m & ((1ULL << 63) | (1ULL << 31)))) { asm ( "umlal %R0, %Q0, %R1, %Q2\n\t" "umlal %R0, %Q0, %Q1, %R2\n\t" "mov %R0, #0\n\t" "umlal %Q0, %R0, %R1, %R2" : "+&r" (__res) : "r" (__m), "r" (__n) : "cc" ); } else { asm ( "umlal %R0, %Q0, %R2, %Q3\n\t" "umlal %R0, %1, %Q2, %R3\n\t" "mov %R0, #0\n\t" "adds %Q0, %1, %Q0\n\t" "adc %R0, %R0, #0\n\t" "umlal %Q0, %R0, %R2, %R3" : "+&r" (__res), "+&r" (__z) : "r" (__m), "r" (__n) : "cc" ); } __res /= __p; { unsigned int __res0 = __res; unsigned int __b0 = __b; __r -= __res0 * __b0; } next_free = __res; }
__r; }) == 1)
ps->next_free++;
}
[-- Attachment #3: dm-snap-persistent_clang.trim.s --]
[-- Type: text/plain, Size: 1982 bytes --]
.text
.syntax unified
.eabi_attribute 67, "2.09" @ Tag_conformance
.eabi_attribute 6, 14 @ Tag_CPU_arch
.eabi_attribute 7, 65 @ Tag_CPU_arch_profile
.eabi_attribute 8, 1 @ Tag_ARM_ISA_use
.eabi_attribute 9, 2 @ Tag_THUMB_ISA_use
.fpu crypto-neon-fp-armv8
.eabi_attribute 12, 4 @ Tag_Advanced_SIMD_arch
.eabi_attribute 36, 1 @ Tag_FP_HP_extension
.eabi_attribute 42, 1 @ Tag_MPextension_use
.eabi_attribute 34, 1 @ Tag_CPU_unaligned_access
.eabi_attribute 68, 3 @ Tag_Virtualization_use
.eabi_attribute 17, 1 @ Tag_ABI_PCS_GOT_use
.eabi_attribute 20, 1 @ Tag_ABI_FP_denormal
.eabi_attribute 21, 0 @ Tag_ABI_FP_exceptions
.eabi_attribute 23, 3 @ Tag_ABI_FP_number_model
.eabi_attribute 24, 1 @ Tag_ABI_align_needed
.eabi_attribute 25, 1 @ Tag_ABI_align_preserved
.eabi_attribute 38, 1 @ Tag_ABI_FP_16bit_format
.eabi_attribute 18, 4 @ Tag_ABI_PCS_wchar_t
.eabi_attribute 26, 2 @ Tag_ABI_enum_size
.eabi_attribute 14, 0 @ Tag_ABI_PCS_R9_use
.file "dm-snap-persistent.c"
@ Start of file scope inline assembly
@ End of file scope inline assembly
.globl skip_metadata @ -- Begin function skip_metadata
.p2align 2
.type skip_metadata,%function
.code 32 @ @skip_metadata
skip_metadata:
.fnstart
@ %bb.0: @ %entry
.save {r4, r5, r6, lr}
push {r4, r5, r6, lr}
mov r5, r0
ldr r0, [r5, #40]!
ldr r6, [r5, #4]
ldr r1, [r5, #-28]
add r4, r1, #1
mov r1, r6
@APP
bl __do_div64
@NO_APP
cmp r1, #1
popne {r4, r5, r6, pc}
adds r0, r0, #1
adc r1, r6, #0
strd r0, r1, [r5]
pop {r4, r5, r6, pc}
.Lfunc_end0:
.size skip_metadata, .Lfunc_end0-skip_metadata
.cantunwind
.fnend
@ -- End function
.ident "clang version 11.0.0 (Realtek LLVM-11 Build 999g+)"
.section ".note.GNU-stack","",%progbits
.addrsig
.eabi_attribute 30, 1 @ Tag_ABI_optimization_goals
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-24 7:42 ` Antony Yu
@ 2020-11-24 10:14 ` Antony Yu
2020-11-24 21:06 ` Nick Desaulniers
0 siblings, 1 reply; 17+ messages in thread
From: Antony Yu @ 2020-11-24 10:14 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Russell King, Nick Desaulniers, linux-arm-kernel, linux-kernel,
clang-built-linux, Pen-Yung Yu
Antony Yu <swpenim@gmail.com> 於 2020年11月24日 週二 下午3:43寫道：
>
> On Mon, Nov 23, 2020 at 11:16:02AM -0700, Nathan Chancellor wrote:
> > On Mon, Nov 23, 2020 at 03:36:32PM +0800, Antony Yu wrote:
> > > __do_div64 clobbers the input register r0 in little endian system.
> > > According to the inline assembly document, if an input operand is
> > > modified, it should be tied to a output operand. This patch can
> > > prevent compilers from reusing r0 register after asm statements.
> > >
> > > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > > ---
> > > arch/arm/include/asm/div64.h | 5 +++--
> > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > > index 898e9c78a7e7..809efc51e90f 100644
> > > --- a/arch/arm/include/asm/div64.h
> > > +++ b/arch/arm/include/asm/div64.h
> > > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > > asm( __asmeq("%0", __xh)
> > > __asmeq("%1", "r2")
> > > __asmeq("%2", "r0")
> > > - __asmeq("%3", "r4")
> > > + __asmeq("%3", "r0")
> > > + __asmeq("%4", "r4")
> > > "bl __do_div64"
> > > - : "=r" (__rem), "=r" (__res)
> > > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > > : "r" (__n), "r" (__base)
> > > : "ip", "lr", "cc");
> > > *n = __res;
> > > --
> > > 2.23.0
> > >
> >
> > I am not sure that I am qualified to review this (my assembly knowledge
> > is not the best) but your commit title mentions an error when compiling
> > with clang. What is the exact error, what configuration generates it,
> > and what version of clang? We have done fairly decent testing for
> > 32-bit ARM, I would like to know what we are missing.
> >
> > Cheers,
> > Nathan
>
> We have run fail on android R vts vts_libsnapshot_test with kernel 4.14.
> This bug is triggered accidently by a workaround patch in our code base.
> It is fine on a pure clean 4.14 branch since __do_div64 may not be
> executed in skip_metadata.
>
> The attachment are .i and generated .s file. .s file can be reproduced
> with clang -target arm-linux-eabi -march=armv8.2-a -O2.
>
> In function skip_metadata, it loads some value to r0, calls __do_div64,
> adds 1 to r0 and stores it to [r5]. It gets wrong value since __do_div64
> clobbers r0 register.
>
> We have tried clang-10, clang-11 and android prebuilt clang-r383902b. All
> of them have the same problem.
Sorry for the large attachment.
I put .i and .s files on
https://gist.github.com/penyung/274b0c697062a1c776994bb40243cfff
Antony Yu
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-24 10:14 ` Antony Yu
@ 2020-11-24 21:06 ` Nick Desaulniers
2020-11-30 8:20 ` [PATCH v2] " Antony Yu
0 siblings, 1 reply; 17+ messages in thread
From: Nick Desaulniers @ 2020-11-24 21:06 UTC (permalink / raw)
To: Antony Yu
Cc: Nathan Chancellor, Russell King, Linux ARM, LKML, clang-built-linux
Thanks for the report, it probably was not fun to debug. I'll take a
closer look at this after the Thanksgiving holiday.
On Tue, Nov 24, 2020 at 2:14 AM Antony Yu <swpenim@gmail.com> wrote:
>
> Antony Yu <swpenim@gmail.com> 於 2020年11月24日 週二 下午3:43寫道：
> >
> > On Mon, Nov 23, 2020 at 11:16:02AM -0700, Nathan Chancellor wrote:
> > > On Mon, Nov 23, 2020 at 03:36:32PM +0800, Antony Yu wrote:
> > > > __do_div64 clobbers the input register r0 in little endian system.
> > > > According to the inline assembly document, if an input operand is
> > > > modified, it should be tied to a output operand. This patch can
> > > > prevent compilers from reusing r0 register after asm statements.
> > > >
> > > > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > > > ---
> > > > arch/arm/include/asm/div64.h | 5 +++--
> > > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > > > index 898e9c78a7e7..809efc51e90f 100644
> > > > --- a/arch/arm/include/asm/div64.h
> > > > +++ b/arch/arm/include/asm/div64.h
> > > > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > > > asm( __asmeq("%0", __xh)
> > > > __asmeq("%1", "r2")
> > > > __asmeq("%2", "r0")
> > > > - __asmeq("%3", "r4")
> > > > + __asmeq("%3", "r0")
> > > > + __asmeq("%4", "r4")
> > > > "bl __do_div64"
> > > > - : "=r" (__rem), "=r" (__res)
> > > > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > > > : "r" (__n), "r" (__base)
> > > > : "ip", "lr", "cc");
> > > > *n = __res;
> > > > --
> > > > 2.23.0
> > > >
> > >
> > > I am not sure that I am qualified to review this (my assembly knowledge
> > > is not the best) but your commit title mentions an error when compiling
> > > with clang. What is the exact error, what configuration generates it,
> > > and what version of clang? We have done fairly decent testing for
> > > 32-bit ARM, I would like to know what we are missing.
> > >
> > > Cheers,
> > > Nathan
> >
> > We have run fail on android R vts vts_libsnapshot_test with kernel 4.14.
> > This bug is triggered accidently by a workaround patch in our code base.
> > It is fine on a pure clean 4.14 branch since __do_div64 may not be
> > executed in skip_metadata.
> >
> > The attachment are .i and generated .s file. .s file can be reproduced
> > with clang -target arm-linux-eabi -march=armv8.2-a -O2.
> >
> > In function skip_metadata, it loads some value to r0, calls __do_div64,
> > adds 1 to r0 and stores it to [r5]. It gets wrong value since __do_div64
> > clobbers r0 register.
> >
> > We have tried clang-10, clang-11 and android prebuilt clang-r383902b. All
> > of them have the same problem.
>
> Sorry for the large attachment.
> I put .i and .s files on
> https://gist.github.com/penyung/274b0c697062a1c776994bb40243cfff
>
> Antony Yu
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-23 7:36 [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang Antony Yu
2020-11-23 18:16 ` Nathan Chancellor
@ 2020-11-24 23:16 ` kernel test robot
2020-11-30 10:11 ` Ard Biesheuvel
2 siblings, 0 replies; 17+ messages in thread
From: kernel test robot @ 2020-11-24 23:16 UTC (permalink / raw)
To: Antony Yu
Cc: kbuild-all, clang-built-linux, swpenim, Russell King,
Nathan Chancellor, Nick Desaulniers, linux-arm-kernel,
linux-kernel
[-- Attachment #1: Type: text/plain, Size: 5215 bytes --]
Hi Antony,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.10-rc5 next-20201124]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Antony-Yu/ARM-fix-__div64_32-error-when-compiling-with-clang/20201123-154454
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 418baf2c28f3473039f2f7377760bd8f6897ae18
config: arm-randconfig-r036-20201124 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project df9ae5992889560a8f3c6760b54d5051b47c7bf5)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm cross compiling tool for clang build
# apt-get install binutils-arm-linux-gnueabi
# https://github.com/0day-ci/linux/commit/83df1f983ed4219556020bf30cc819e66bb7e69c
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Antony-Yu/ARM-fix-__div64_32-error-when-compiling-with-clang/20201123-154454
git checkout 83df1f983ed4219556020bf30cc819e66bb7e69c
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
drivers/tty/serial/imx.c:359:19: warning: unused function 'imx_uart_is_imx21' [-Wunused-function]
static inline int imx_uart_is_imx21(struct imx_port *sport)
^
drivers/tty/serial/imx.c:364:19: warning: unused function 'imx_uart_is_imx53' [-Wunused-function]
static inline int imx_uart_is_imx53(struct imx_port *sport)
^
drivers/tty/serial/imx.c:369:19: warning: unused function 'imx_uart_is_imx6q' [-Wunused-function]
static inline int imx_uart_is_imx6q(struct imx_port *sport)
^
In file included from drivers/tty/serial/imx.c:11:
In file included from include/linux/module.h:12:
In file included from include/linux/list.h:9:
In file included from include/linux/kernel.h:19:
>> arch/arm/include/asm/div64.h:39:2: error: multiple outputs to hard register: r0
asm( __asmeq("%0", __xh)
^
3 warnings and 1 error generated.
--
In file included from drivers/gpu/drm/tegra/dc.c:7:
In file included from include/linux/clk.h:13:
In file included from include/linux/kernel.h:19:
>> arch/arm/include/asm/div64.h:39:2: error: multiple outputs to hard register: r0
asm( __asmeq("%0", __xh)
^
1 error generated.
vim +39 arch/arm/include/asm/div64.h
^1da177e4c3f415 include/asm-arm/div64.h Linus Torvalds 2005-04-16 32
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 33 static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 34 {
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 35 register unsigned int __base asm("r4") = base;
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 36 register unsigned long long __n asm("r0") = *n;
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 37 register unsigned long long __res asm("r2");
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 38 register unsigned int __rem asm(__xh);
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 @39 asm( __asmeq("%0", __xh)
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 40 __asmeq("%1", "r2")
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 41 __asmeq("%2", "r0")
83df1f983ed4219 arch/arm/include/asm/div64.h Antony Yu 2020-11-23 42 __asmeq("%3", "r0")
83df1f983ed4219 arch/arm/include/asm/div64.h Antony Yu 2020-11-23 43 __asmeq("%4", "r4")
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 44 "bl __do_div64"
83df1f983ed4219 arch/arm/include/asm/div64.h Antony Yu 2020-11-23 45 : "=r" (__rem), "=r" (__res), "=r" (__n)
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 46 : "r" (__n), "r" (__base)
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 47 : "ip", "lr", "cc");
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 48 *n = __res;
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 49 return __rem;
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 50 }
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 51 #define __div64_32 __div64_32
040b323b5012b55 arch/arm/include/asm/div64.h Nicolas Pitre 2015-11-02 52
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 33586 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v2] ARM: fix __div64_32() error when compiling with clang
2020-11-24 21:06 ` Nick Desaulniers
@ 2020-11-30 8:20 ` Antony Yu
0 siblings, 0 replies; 17+ messages in thread
From: Antony Yu @ 2020-11-30 8:20 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Nathan Chancellor, Russell King, Linux ARM, LKML,
clang-built-linux, swpenim
[-- Attachment #1: v2-0001-ARM-fix-__div64_32-error-when-compiling-with-clan.patch --]
[-- Type: text/x-diff, Size: 1326 bytes --]
From 3a4c19e09079324a625941ea3c16fe4b0df2ed86 Mon Sep 17 00:00:00 2001
From: Antony Yu <swpenim@gmail.com>
Date: Mon, 9 Nov 2020 17:31:52 +0800
Subject: [PATCH v2] ARM: fix __div64_32() error when compiling with clang
__do_div64 clobbers the input register r0 in little endian system.
According to the inline assembly document, if an input operand is
modified, it should be tied to a output operand. This patch can
prevent compilers from reusing r0 register after asm statements.
Signed-off-by: Antony Yu <swpenim@gmail.com>
Reported-by: kernel test robot <lkp@intel.com>
---
changed in v2
- add kernel-test-robot tag
- fix build failure on big endian
arch/arm/include/asm/div64.h | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
index 898e9c78a7e7..961a129eb41f 100644
--- a/arch/arm/include/asm/div64.h
+++ b/arch/arm/include/asm/div64.h
@@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
asm( __asmeq("%0", __xh)
__asmeq("%1", "r2")
__asmeq("%2", "r0")
- __asmeq("%3", "r4")
+ __asmeq("%3", "r0")
+ __asmeq("%4", "r4")
"bl __do_div64"
- : "=r" (__rem), "=r" (__res)
+ : "=r" (__rem), "=r" (__res), "=r" (__xl)
: "r" (__n), "r" (__base)
: "ip", "lr", "cc");
*n = __res;
--
2.29.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-23 7:36 [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang Antony Yu
2020-11-23 18:16 ` Nathan Chancellor
2020-11-24 23:16 ` [RESEND,PATCH] " kernel test robot
@ 2020-11-30 10:11 ` Ard Biesheuvel
2020-11-30 10:12 ` Ard Biesheuvel
2 siblings, 1 reply; 17+ messages in thread
From: Ard Biesheuvel @ 2020-11-30 10:11 UTC (permalink / raw)
To: Antony Yu
Cc: Nick Desaulniers, Russell King, Linux Kernel Mailing List,
clang-built-linux, Nathan Chancellor, Linux ARM
On Mon, 23 Nov 2020 at 08:39, Antony Yu <swpenim@gmail.com> wrote:
>
> __do_div64 clobbers the input register r0 in little endian system.
> According to the inline assembly document, if an input operand is
> modified, it should be tied to a output operand. This patch can
> prevent compilers from reusing r0 register after asm statements.
>
> Signed-off-by: Antony Yu <swpenim@gmail.com>
> ---
> arch/arm/include/asm/div64.h | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> index 898e9c78a7e7..809efc51e90f 100644
> --- a/arch/arm/include/asm/div64.h
> +++ b/arch/arm/include/asm/div64.h
> @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> asm( __asmeq("%0", __xh)
> __asmeq("%1", "r2")
> __asmeq("%2", "r0")
> - __asmeq("%3", "r4")
> + __asmeq("%3", "r0")
> + __asmeq("%4", "r4")
> "bl __do_div64"
> - : "=r" (__rem), "=r" (__res)
> + : "=r" (__rem), "=r" (__res), "=r" (__n)
> : "r" (__n), "r" (__base)
> : "ip", "lr", "cc");
> *n = __res;
> --
> 2.23.0
>
Agree that using r0 as an input operand only is incorrect, and not
only on Clang. The compiler might assume that r0 will retain its value
across the asm() block, which is obviously not the case.
However, your patch will likely break big-endian, since in that case,
__xh == r0, and so it will appear twice.
Perhaps it would be better to change the type of __rem to unsigned
long long as well?
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 10:11 ` Ard Biesheuvel
@ 2020-11-30 10:12 ` Ard Biesheuvel
2020-11-30 10:21 ` Russell King - ARM Linux admin
2020-11-30 15:50 ` Nicolas Pitre
0 siblings, 2 replies; 17+ messages in thread
From: Ard Biesheuvel @ 2020-11-30 10:12 UTC (permalink / raw)
To: Antony Yu, Nicolas Pitre
Cc: Nick Desaulniers, Russell King, Linux Kernel Mailing List,
clang-built-linux, Nathan Chancellor, Linux ARM
(+ Nico)
On Mon, 30 Nov 2020 at 11:11, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Mon, 23 Nov 2020 at 08:39, Antony Yu <swpenim@gmail.com> wrote:
> >
> > __do_div64 clobbers the input register r0 in little endian system.
> > According to the inline assembly document, if an input operand is
> > modified, it should be tied to a output operand. This patch can
> > prevent compilers from reusing r0 register after asm statements.
> >
> > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > ---
> > arch/arm/include/asm/div64.h | 5 +++--
> > 1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > index 898e9c78a7e7..809efc51e90f 100644
> > --- a/arch/arm/include/asm/div64.h
> > +++ b/arch/arm/include/asm/div64.h
> > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > asm( __asmeq("%0", __xh)
> > __asmeq("%1", "r2")
> > __asmeq("%2", "r0")
> > - __asmeq("%3", "r4")
> > + __asmeq("%3", "r0")
> > + __asmeq("%4", "r4")
> > "bl __do_div64"
> > - : "=r" (__rem), "=r" (__res)
> > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > : "r" (__n), "r" (__base)
> > : "ip", "lr", "cc");
> > *n = __res;
> > --
> > 2.23.0
> >
>
> Agree that using r0 as an input operand only is incorrect, and not
> only on Clang. The compiler might assume that r0 will retain its value
> across the asm() block, which is obviously not the case.
>
> However, your patch will likely break big-endian, since in that case,
> __xh == r0, and so it will appear twice.
>
> Perhaps it would be better to change the type of __rem to unsigned
> long long as well?
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 10:12 ` Ard Biesheuvel
@ 2020-11-30 10:21 ` Russell King - ARM Linux admin
2020-11-30 10:40 ` Ard Biesheuvel
2020-11-30 15:50 ` Nicolas Pitre
1 sibling, 1 reply; 17+ messages in thread
From: Russell King - ARM Linux admin @ 2020-11-30 10:21 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Antony Yu, Nicolas Pitre, Nick Desaulniers,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
On Mon, Nov 30, 2020 at 11:12:33AM +0100, Ard Biesheuvel wrote:
> (+ Nico)
>
> On Mon, 30 Nov 2020 at 11:11, Ard Biesheuvel <ardb@kernel.org> wrote:
> >
> > On Mon, 23 Nov 2020 at 08:39, Antony Yu <swpenim@gmail.com> wrote:
> > >
> > > __do_div64 clobbers the input register r0 in little endian system.
> > > According to the inline assembly document, if an input operand is
> > > modified, it should be tied to a output operand. This patch can
> > > prevent compilers from reusing r0 register after asm statements.
> > >
> > > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > > ---
> > > arch/arm/include/asm/div64.h | 5 +++--
> > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > > index 898e9c78a7e7..809efc51e90f 100644
> > > --- a/arch/arm/include/asm/div64.h
> > > +++ b/arch/arm/include/asm/div64.h
> > > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > > asm( __asmeq("%0", __xh)
> > > __asmeq("%1", "r2")
> > > __asmeq("%2", "r0")
> > > - __asmeq("%3", "r4")
> > > + __asmeq("%3", "r0")
> > > + __asmeq("%4", "r4")
> > > "bl __do_div64"
> > > - : "=r" (__rem), "=r" (__res)
> > > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > > : "r" (__n), "r" (__base)
> > > : "ip", "lr", "cc");
> > > *n = __res;
> > > --
> > > 2.23.0
> > >
> >
> > Agree that using r0 as an input operand only is incorrect, and not
> > only on Clang. The compiler might assume that r0 will retain its value
> > across the asm() block, which is obviously not the case.
However, you can _not_ have an asm block that names two outputs using
the same physical register - that's why both the original patch and
the posted v2 will fail.
You also can't mark r0 as clobbered because it's used as an operand
and that is not allowed by gcc.
The fact is, we have two register variables occupying the same register,
which are __n and __rem. It doesn't matter which endian-ness __rem is,
r0 will be used for both __n (input) and __rem (output).
If the compiler can't work out that if a physical register used as an
output operand will be written by the assembler, then the compiler is
quite simply buggy.
The code is correct as it stands; Clang is buggy.
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 10:21 ` Russell King - ARM Linux admin
@ 2020-11-30 10:40 ` Ard Biesheuvel
2020-11-30 13:58 ` David Laight
0 siblings, 1 reply; 17+ messages in thread
From: Ard Biesheuvel @ 2020-11-30 10:40 UTC (permalink / raw)
To: Russell King - ARM Linux admin
Cc: Antony Yu, Nicolas Pitre, Nick Desaulniers,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
On Mon, 30 Nov 2020 at 11:21, Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
>
> On Mon, Nov 30, 2020 at 11:12:33AM +0100, Ard Biesheuvel wrote:
> > (+ Nico)
> >
> > On Mon, 30 Nov 2020 at 11:11, Ard Biesheuvel <ardb@kernel.org> wrote:
> > >
> > > On Mon, 23 Nov 2020 at 08:39, Antony Yu <swpenim@gmail.com> wrote:
> > > >
> > > > __do_div64 clobbers the input register r0 in little endian system.
> > > > According to the inline assembly document, if an input operand is
> > > > modified, it should be tied to a output operand. This patch can
> > > > prevent compilers from reusing r0 register after asm statements.
> > > >
> > > > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > > > ---
> > > > arch/arm/include/asm/div64.h | 5 +++--
> > > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > > > index 898e9c78a7e7..809efc51e90f 100644
> > > > --- a/arch/arm/include/asm/div64.h
> > > > +++ b/arch/arm/include/asm/div64.h
> > > > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > > > asm( __asmeq("%0", __xh)
> > > > __asmeq("%1", "r2")
> > > > __asmeq("%2", "r0")
> > > > - __asmeq("%3", "r4")
> > > > + __asmeq("%3", "r0")
> > > > + __asmeq("%4", "r4")
> > > > "bl __do_div64"
> > > > - : "=r" (__rem), "=r" (__res)
> > > > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > > > : "r" (__n), "r" (__base)
> > > > : "ip", "lr", "cc");
> > > > *n = __res;
> > > > --
> > > > 2.23.0
> > > >
> > >
> > > Agree that using r0 as an input operand only is incorrect, and not
> > > only on Clang. The compiler might assume that r0 will retain its value
> > > across the asm() block, which is obviously not the case.
>
> However, you can _not_ have an asm block that names two outputs using
> the same physical register - that's why both the original patch and
> the posted v2 will fail.
>
> You also can't mark r0 as clobbered because it's used as an operand
> and that is not allowed by gcc.
>
> The fact is, we have two register variables occupying the same register,
> which are __n and __rem. It doesn't matter which endian-ness __rem is,
> r0 will be used for both __n (input) and __rem (output).
>
__rem is a 32-bit variable, so in LE mode, only r1 is used for __rem,
not r0. So r0/r1 are used as an input operand pair, and r1 is used as
an output operand.
So I don't think the compiler has to be buggy in order for it to
assume that r0 will still contain the low word of the dividend
afterwards.
And actually, the same applies on BE, but the other way around. So we
should mark __xl as an output register as well, as __xl will assume
the right value depending on the endianness.
I suggest something like the below,
diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
index 898e9c78a7e7..85ff9109595e 100644
--- a/arch/arm/include/asm/div64.h
+++ b/arch/arm/include/asm/div64.h
@@ -36,12 +36,14 @@ static inline uint32_t __div64_32(uint64_t *n,
uint32_t base)
register unsigned long long __n asm("r0") = *n;
register unsigned long long __res asm("r2");
register unsigned int __rem asm(__xh);
+ register unsigned int __dummy asm(__xl);
asm( __asmeq("%0", __xh)
__asmeq("%1", "r2")
- __asmeq("%2", "r0")
- __asmeq("%3", "r4")
+ __asmeq("%2", __xl)
+ __asmeq("%3", "r0")
+ __asmeq("%4", "r4")
"bl __do_div64"
- : "=r" (__rem), "=r" (__res)
+ : "=r" (__rem), "=r" (__res), "=r"(__dummy)
: "r" (__n), "r" (__base)
: "ip", "lr", "cc");
*n = __res;
> If the compiler can't work out that if a physical register used as an
> output operand will be written by the assembler, then the compiler is
> quite simply buggy.
>
> The code is correct as it stands; Clang is buggy.
>
> --
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
^ permalink raw reply related [flat|nested] 17+ messages in thread
* RE: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 10:40 ` Ard Biesheuvel
@ 2020-11-30 13:58 ` David Laight
2020-11-30 14:18 ` Russell King - ARM Linux admin
0 siblings, 1 reply; 17+ messages in thread
From: David Laight @ 2020-11-30 13:58 UTC (permalink / raw)
To: 'Ard Biesheuvel', Russell King - ARM Linux admin
Cc: Antony Yu, Nicolas Pitre, Nick Desaulniers,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
> And actually, the same applies on BE, but the other way around. So we
> should mark __xl as an output register as well, as __xl will assume
> the right value depending on the endianness.
Why not use "+r" to indicate than an 'output' parameter is also
used as an input.
Rather cleaner than specifying the same C variable as both
input and output.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 13:58 ` David Laight
@ 2020-11-30 14:18 ` Russell King - ARM Linux admin
0 siblings, 0 replies; 17+ messages in thread
From: Russell King - ARM Linux admin @ 2020-11-30 14:18 UTC (permalink / raw)
To: David Laight
Cc: 'Ard Biesheuvel',
Antony Yu, Nicolas Pitre, Nick Desaulniers,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
On Mon, Nov 30, 2020 at 01:58:27PM +0000, David Laight wrote:
> > And actually, the same applies on BE, but the other way around. So we
> > should mark __xl as an output register as well, as __xl will assume
> > the right value depending on the endianness.
>
> Why not use "+r" to indicate than an 'output' parameter is also
> used as an input.
>
> Rather cleaner than specifying the same C variable as both
> input and output.
You have an incorrect understanding. "__n" is the input operand in r0.
"__rem" is the output operand in r0/r1.
No single C variable is used as both an input and an output.
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 10:12 ` Ard Biesheuvel
2020-11-30 10:21 ` Russell King - ARM Linux admin
@ 2020-11-30 15:50 ` Nicolas Pitre
2020-11-30 17:18 ` Ard Biesheuvel
1 sibling, 1 reply; 17+ messages in thread
From: Nicolas Pitre @ 2020-11-30 15:50 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Antony Yu, Nick Desaulniers, Russell King,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
On Mon, 30 Nov 2020, Ard Biesheuvel wrote:
> (+ Nico)
>
> On Mon, 30 Nov 2020 at 11:11, Ard Biesheuvel <ardb@kernel.org> wrote:
> >
> > On Mon, 23 Nov 2020 at 08:39, Antony Yu <swpenim@gmail.com> wrote:
> > >
> > > __do_div64 clobbers the input register r0 in little endian system.
> > > According to the inline assembly document, if an input operand is
> > > modified, it should be tied to a output operand. This patch can
> > > prevent compilers from reusing r0 register after asm statements.
> > >
> > > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > > ---
> > > arch/arm/include/asm/div64.h | 5 +++--
> > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > > index 898e9c78a7e7..809efc51e90f 100644
> > > --- a/arch/arm/include/asm/div64.h
> > > +++ b/arch/arm/include/asm/div64.h
> > > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > > asm( __asmeq("%0", __xh)
> > > __asmeq("%1", "r2")
> > > __asmeq("%2", "r0")
> > > - __asmeq("%3", "r4")
> > > + __asmeq("%3", "r0")
> > > + __asmeq("%4", "r4")
> > > "bl __do_div64"
> > > - : "=r" (__rem), "=r" (__res)
> > > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > > : "r" (__n), "r" (__base)
> > > : "ip", "lr", "cc");
> > > *n = __res;
> > > --
> > > 2.23.0
> > >
> >
> > Agree that using r0 as an input operand only is incorrect, and not
> > only on Clang. The compiler might assume that r0 will retain its value
> > across the asm() block, which is obviously not the case.
You're right.
This was done like that most likely to work around some stupid code
generation with "__n >> 32" while using gcc from about 20 years ago. IOW
I don't exactly remember why I did it like that, but it is certainly
flawed.
Here's my version of the fix which should be correct. Warning: this
is completely untested, but should in theory produce the same code on
modern gcc.
diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
index 898e9c78a7..595e538f5b 100644
--- a/arch/arm/include/asm/div64.h
+++ b/arch/arm/include/asm/div64.h
@@ -21,29 +21,20 @@
* assembly implementation with completely non standard calling convention
* for arguments and results (beware).
*/
-
-#ifdef __ARMEB__
-#define __xh "r0"
-#define __xl "r1"
-#else
-#define __xl "r0"
-#define __xh "r1"
-#endif
-
static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
{
register unsigned int __base asm("r4") = base;
register unsigned long long __n asm("r0") = *n;
register unsigned long long __res asm("r2");
- register unsigned int __rem asm(__xh);
- asm( __asmeq("%0", __xh)
+ unsigned int __rem;
+ asm( __asmeq("%0", "r0")
__asmeq("%1", "r2")
- __asmeq("%2", "r0")
- __asmeq("%3", "r4")
+ __asmeq("%2", "r4")
"bl __do_div64"
- : "=r" (__rem), "=r" (__res)
- : "r" (__n), "r" (__base)
+ : "+r" (__n), "=r" (__res)
+ : "r" (__base)
: "ip", "lr", "cc");
+ __rem = __n >> 32;
*n = __res;
return __rem;
}
I'll submit it if someone confirms it works.
Nicolas
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 15:50 ` Nicolas Pitre
@ 2020-11-30 17:18 ` Ard Biesheuvel
2020-11-30 17:52 ` Nicolas Pitre
0 siblings, 1 reply; 17+ messages in thread
From: Ard Biesheuvel @ 2020-11-30 17:18 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Antony Yu, Nick Desaulniers, Russell King,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
On Mon, 30 Nov 2020 at 16:51, Nicolas Pitre <nico@fluxnic.net> wrote:
>
> On Mon, 30 Nov 2020, Ard Biesheuvel wrote:
>
> > (+ Nico)
> >
> > On Mon, 30 Nov 2020 at 11:11, Ard Biesheuvel <ardb@kernel.org> wrote:
> > >
> > > On Mon, 23 Nov 2020 at 08:39, Antony Yu <swpenim@gmail.com> wrote:
> > > >
> > > > __do_div64 clobbers the input register r0 in little endian system.
> > > > According to the inline assembly document, if an input operand is
> > > > modified, it should be tied to a output operand. This patch can
> > > > prevent compilers from reusing r0 register after asm statements.
> > > >
> > > > Signed-off-by: Antony Yu <swpenim@gmail.com>
> > > > ---
> > > > arch/arm/include/asm/div64.h | 5 +++--
> > > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > > > index 898e9c78a7e7..809efc51e90f 100644
> > > > --- a/arch/arm/include/asm/div64.h
> > > > +++ b/arch/arm/include/asm/div64.h
> > > > @@ -39,9 +39,10 @@ static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > > > asm( __asmeq("%0", __xh)
> > > > __asmeq("%1", "r2")
> > > > __asmeq("%2", "r0")
> > > > - __asmeq("%3", "r4")
> > > > + __asmeq("%3", "r0")
> > > > + __asmeq("%4", "r4")
> > > > "bl __do_div64"
> > > > - : "=r" (__rem), "=r" (__res)
> > > > + : "=r" (__rem), "=r" (__res), "=r" (__n)
> > > > : "r" (__n), "r" (__base)
> > > > : "ip", "lr", "cc");
> > > > *n = __res;
> > > > --
> > > > 2.23.0
> > > >
> > >
> > > Agree that using r0 as an input operand only is incorrect, and not
> > > only on Clang. The compiler might assume that r0 will retain its value
> > > across the asm() block, which is obviously not the case.
>
> You're right.
>
> This was done like that most likely to work around some stupid code
> generation with "__n >> 32" while using gcc from about 20 years ago. IOW
> I don't exactly remember why I did it like that, but it is certainly
> flawed.
>
> Here's my version of the fix which should be correct. Warning: this
> is completely untested, but should in theory produce the same code on
> modern gcc.
>
> diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> index 898e9c78a7..595e538f5b 100644
> --- a/arch/arm/include/asm/div64.h
> +++ b/arch/arm/include/asm/div64.h
> @@ -21,29 +21,20 @@
> * assembly implementation with completely non standard calling convention
> * for arguments and results (beware).
> */
> -
> -#ifdef __ARMEB__
> -#define __xh "r0"
> -#define __xl "r1"
> -#else
> -#define __xl "r0"
> -#define __xh "r1"
> -#endif
> -
> static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> {
> register unsigned int __base asm("r4") = base;
> register unsigned long long __n asm("r0") = *n;
> register unsigned long long __res asm("r2");
> - register unsigned int __rem asm(__xh);
> - asm( __asmeq("%0", __xh)
> + unsigned int __rem;
> + asm( __asmeq("%0", "r0")
> __asmeq("%1", "r2")
> - __asmeq("%2", "r0")
> - __asmeq("%3", "r4")
> + __asmeq("%2", "r4")
> "bl __do_div64"
> - : "=r" (__rem), "=r" (__res)
> - : "r" (__n), "r" (__base)
> + : "+r" (__n), "=r" (__res)
> + : "r" (__base)
> : "ip", "lr", "cc");
> + __rem = __n >> 32;
This treats {r0, r1} as a {low, high} pair, regardless of endianness,
and so it puts the value of r0 into r1. Doesn't that mean the shift
should only be done on little endian?
> *n = __res;
> return __rem;
> }
>
> I'll submit it if someone confirms it works.
>
>
> Nicolas
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 17:18 ` Ard Biesheuvel
@ 2020-11-30 17:52 ` Nicolas Pitre
2020-11-30 18:08 ` Ard Biesheuvel
0 siblings, 1 reply; 17+ messages in thread
From: Nicolas Pitre @ 2020-11-30 17:52 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Antony Yu, Nick Desaulniers, Russell King,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
On Mon, 30 Nov 2020, Ard Biesheuvel wrote:
> On Mon, 30 Nov 2020 at 16:51, Nicolas Pitre <nico@fluxnic.net> wrote:
>
> > Here's my version of the fix which should be correct. Warning: this
> > is completely untested, but should in theory produce the same code on
> > modern gcc.
> >
> > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > index 898e9c78a7..595e538f5b 100644
> > --- a/arch/arm/include/asm/div64.h
> > +++ b/arch/arm/include/asm/div64.h
> > @@ -21,29 +21,20 @@
> > * assembly implementation with completely non standard calling convention
> > * for arguments and results (beware).
> > */
> > -
> > -#ifdef __ARMEB__
> > -#define __xh "r0"
> > -#define __xl "r1"
> > -#else
> > -#define __xl "r0"
> > -#define __xh "r1"
> > -#endif
> > -
> > static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > {
> > register unsigned int __base asm("r4") = base;
> > register unsigned long long __n asm("r0") = *n;
> > register unsigned long long __res asm("r2");
> > - register unsigned int __rem asm(__xh);
> > - asm( __asmeq("%0", __xh)
> > + unsigned int __rem;
> > + asm( __asmeq("%0", "r0")
> > __asmeq("%1", "r2")
> > - __asmeq("%2", "r0")
> > - __asmeq("%3", "r4")
> > + __asmeq("%2", "r4")
> > "bl __do_div64"
> > - : "=r" (__rem), "=r" (__res)
> > - : "r" (__n), "r" (__base)
> > + : "+r" (__n), "=r" (__res)
> > + : "r" (__base)
> > : "ip", "lr", "cc");
> > + __rem = __n >> 32;
>
> This treats {r0, r1} as a {low, high} pair, regardless of endianness,
> and so it puts the value of r0 into r1. Doesn't that mean the shift
> should only be done on little endian?
Not quite. r0-r1 = low-high is for little endian. Then "__n >> 32" is
actually translated into "mov r0, r1" to move it into __rem and returned
through r0.
On big endial it is r0-r1 = high-low. Here "__n >> 32" picks r0 and
moves it to __rem which is returned through r0 so no extra instruction
needed.
Of course the function is inlined so r0 can be anything, or optimized
away if__rem is not used.
Nicolas
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang
2020-11-30 17:52 ` Nicolas Pitre
@ 2020-11-30 18:08 ` Ard Biesheuvel
0 siblings, 0 replies; 17+ messages in thread
From: Ard Biesheuvel @ 2020-11-30 18:08 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Antony Yu, Nick Desaulniers, Russell King,
Linux Kernel Mailing List, clang-built-linux, Nathan Chancellor,
Linux ARM
On Mon, 30 Nov 2020 at 18:52, Nicolas Pitre <nico@fluxnic.net> wrote:
>
> On Mon, 30 Nov 2020, Ard Biesheuvel wrote:
>
> > On Mon, 30 Nov 2020 at 16:51, Nicolas Pitre <nico@fluxnic.net> wrote:
> >
> > > Here's my version of the fix which should be correct. Warning: this
> > > is completely untested, but should in theory produce the same code on
> > > modern gcc.
> > >
> > > diff --git a/arch/arm/include/asm/div64.h b/arch/arm/include/asm/div64.h
> > > index 898e9c78a7..595e538f5b 100644
> > > --- a/arch/arm/include/asm/div64.h
> > > +++ b/arch/arm/include/asm/div64.h
> > > @@ -21,29 +21,20 @@
> > > * assembly implementation with completely non standard calling convention
> > > * for arguments and results (beware).
> > > */
> > > -
> > > -#ifdef __ARMEB__
> > > -#define __xh "r0"
> > > -#define __xl "r1"
> > > -#else
> > > -#define __xl "r0"
> > > -#define __xh "r1"
> > > -#endif
> > > -
> > > static inline uint32_t __div64_32(uint64_t *n, uint32_t base)
> > > {
> > > register unsigned int __base asm("r4") = base;
> > > register unsigned long long __n asm("r0") = *n;
> > > register unsigned long long __res asm("r2");
> > > - register unsigned int __rem asm(__xh);
> > > - asm( __asmeq("%0", __xh)
> > > + unsigned int __rem;
> > > + asm( __asmeq("%0", "r0")
> > > __asmeq("%1", "r2")
> > > - __asmeq("%2", "r0")
> > > - __asmeq("%3", "r4")
> > > + __asmeq("%2", "r4")
> > > "bl __do_div64"
> > > - : "=r" (__rem), "=r" (__res)
> > > - : "r" (__n), "r" (__base)
> > > + : "+r" (__n), "=r" (__res)
> > > + : "r" (__base)
> > > : "ip", "lr", "cc");
> > > + __rem = __n >> 32;
> >
> > This treats {r0, r1} as a {low, high} pair, regardless of endianness,
> > and so it puts the value of r0 into r1. Doesn't that mean the shift
> > should only be done on little endian?
>
> Not quite. r0-r1 = low-high is for little endian. Then "__n >> 32" is
> actually translated into "mov r0, r1" to move it into __rem and returned
> through r0.
>
> On big endial it is r0-r1 = high-low. Here "__n >> 32" picks r0 and
> moves it to __rem which is returned through r0 so no extra instruction
> needed.
>
> Of course the function is inlined so r0 can be anything, or optimized
> away if__rem is not used.
>
OK, you're right. I got myself confused there, but a quick test with
GCC confirms your explanation:
$ arm-linux-gnueabihf-gcc -mbig-endian -O2 -S -o - \
-xc - <<<"long f(long long l) { return l >> 32; }"
just produces
bx lr
whereas removing the -mbig-endian gives
mov r0, r1
bx lr
I tested the change and it builds and runs fine (although I am not
sure how much coverage this code gets on an ordinary boot):
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Ard Biesheuvel <ardb@kernel.org>
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2020-11-30 18:09 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-23 7:36 [RESEND,PATCH] ARM: fix __div64_32() error when compiling with clang Antony Yu
2020-11-23 18:16 ` Nathan Chancellor
2020-11-24 7:42 ` Antony Yu
2020-11-24 10:14 ` Antony Yu
2020-11-24 21:06 ` Nick Desaulniers
2020-11-30 8:20 ` [PATCH v2] " Antony Yu
2020-11-24 23:16 ` [RESEND,PATCH] " kernel test robot
2020-11-30 10:11 ` Ard Biesheuvel
2020-11-30 10:12 ` Ard Biesheuvel
2020-11-30 10:21 ` Russell King - ARM Linux admin
2020-11-30 10:40 ` Ard Biesheuvel
2020-11-30 13:58 ` David Laight
2020-11-30 14:18 ` Russell King - ARM Linux admin
2020-11-30 15:50 ` Nicolas Pitre
2020-11-30 17:18 ` Ard Biesheuvel
2020-11-30 17:52 ` Nicolas Pitre
2020-11-30 18:08 ` Ard Biesheuvel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).