* Draft 3 of bpf(2) man page for review
@ 2015-07-22 18:43 Michael Kerrisk (man-pages)
2015-07-22 19:22 ` Alexei Starovoitov
0 siblings, 1 reply; 10+ messages in thread
From: Michael Kerrisk (man-pages) @ 2015-07-22 18:43 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann
Cc: mtk.manpages, linux-man, linux-kernel, Silvan Jegen, Walter Harms
Hello Alexei, Daniel, et al.
Please find below another draft of the bpf(2) man page.
It has gone through some fairly substantial editing since
the last draft.
There still remain quite a number of FIXMEs. Alexei, I am mindful
that you'd like to get a version of this page released soon. With
that in mind, I've marked some of the FIXMEs as "FIXME!!". I'd
like to get most of those FIXMEs resolved before an initial release
of the page. The other FIXMEs could be resolved incrementally later.
Cheers,
Michael
.\" Copyright (C) 2015 Alexei Starovoitov <ast@kernel.org>
.\" and Copyright (C) 2015 Michael Kerrisk <mtk.manpages@gmail.com>
.\"
.\" %%%LICENSE_START(VERBATIM)
.\" Permission is granted to make and distribute verbatim copies of this
.\" manual provided the copyright notice and this permission notice are
.\" preserved on all copies.
.\"
.\" Permission is granted to copy and distribute modified versions of this
.\" manual under the conditions for verbatim copying, provided that the
.\" entire resulting derived work is distributed under the terms of a
.\" permission notice identical to this one.
.\"
.\" Since the Linux kernel and libraries are constantly changing, this
.\" manual page may be incorrect or out-of-date. The author(s) assume no
.\" responsibility for errors or omissions, or for damages resulting from
.\" the use of the information contained herein. The author(s) may not
.\" have taken the same level of care in the production of this manual,
.\" which is licensed free of charge, as they might when working
.\" professionally.
.\"
.\" Formatted or processed versions of this manual, if unaccompanied by
.\" the source, must acknowledge the copyright and authors of this work.
.\" %%%LICENSE_END
.\"
.TH BPF 2 2015-03-10 "Linux" "Linux Programmer's Manual"
.SH NAME
bpf - perform a command on an extended eBPF map or program
.SH SYNOPSIS
.nf
.B #include <linux/bpf.h>
.sp
.BI "int bpf(int cmd, union bpf_attr *attr, unsigned int size);
.SH DESCRIPTION
The
.BR bpf ()
system call performs a range of operations related to extended
Berkeley Packet Filters.
Extended BPF (or eBPF) is similar to
the original ("classic") BPF (cBPF) used to filter network packets.
For both cBPF and eBPF programs,
the kernel statically analyzes the programs before loading them,
in order to ensure that they cannot harm the running system.
.P
eBPF extends cBPF in multiple ways, including the ability to call
a fixed set of in-kernel helper functions
.\" See 'enum bpf_func_id' in include/uapi/linux/bpf.h
(via the
.B BPF_CALL
opcode extension provided by eBPF)
and access shared data structures such as eBPF maps.
.SS Extended BPF Design/Architecture
.\"
.\" FIXME In the following line, what does "different data types" mean?
.\" Are the values in a map not just blobs?
.\" Daniel Borkman commented:
.\" Sort of, currently, these blobs can have different sizes of keys
.\" and values (you can even have structs as keys). For the map itself
.\" they are treated as blob internally. However, recently, bpf tail call
.\" got added where you can lookup another program from an array map and
.\" call into it. Here, that particular type of map can only have entries
.\" of type of eBPF program fd. I think, if needed, adding a paragraph to
.\" the tail call could be done as follow-up after we have an initial man
.\" page in the tree included.
.\"
BPF maps are a generic data structure for storage of different data types.
A user process can create multiple maps (with key/value-pairs being
opaque bytes of data) and access them via file descriptors.
eBPF programs can access maps from inside the kernel in parallel.
.\"
.\" FIXME!! What does the previous sentence mean?
.\"
.\" Isn't "from inside the kernel" redundant? (I mean: all eBPF programs
.\" are running inside the kernel, right?)
.\" And what does "in parallel" mean?
.\" Would a simpler version of this sentence be correct? As in:
.\" "Different eBPF programs can access the same maps in parallel."
.\" ?
.\" (Actually, the page already says something like that lower down.)
.\"
It's up to the user process and eBPF program to decide what they store
inside maps.
.P
eBPF programs are similar to kernel modules.
They are loaded by the user
process and automatically unloaded when the process exits.
.\"
.\" FIXME Daniel Borkmann commented about the preceding sentence:
.\"
.\" Generally that's true. Btw, in 4.1 kernel, tc(8) also got support for
.\" eBPF classifier and actions, and here it's slightly different: in tc,
.\" we load the programs, maps etc, and push down the eBPF program fd in
.\" order to let the kernel hold reference on the program itself.
.\"
.\" Thus, there, the program fd that the application owns is gone when the
.\" application terminates, but the eBPF program itself still lives on
.\" inside the kernel.
.\"
.\" Probably something should be said about this in this man page.
.\"
Each program is a set of instructions that is safe to run until
its completion.
An in-kernel verifier statically determines that the eBPF program
terminates and is safe to execute.
During verification, the kernel increments reference counts for each of
the maps that the eBPF program uses,
so that the selected maps cannot be removed until the program is unloaded.
eBPF programs can be attached to different events.
These events can be the arrival of network packets, tracing
events, classification event by qdisc (for eBPF programs attached to a
.BR tc (8)
classifier), and other types that may be added in the future.
A new event triggers execution of the eBPF program, which
may store information about the event in eBPF maps.
Beyond storing data, eBPF programs may call a fixed set of
in-kernel helper functions.
The same eBPF program can be attached to multiple events and different
eBPF programs can access the same map:
.in +4n
.nf
tracing tracing tracing packet packet
event A event B event C on eth0 on eth1
| | | | |
| | | | |
--> tracing <-- tracing socket socket
prog_1 prog_2 prog_3 prog_4
| | | |
|--- -----| |-------| map_3
map_1 map_2
.fi
.in
.SS Arguments
The operation to be performed by the
.BR bpf ()
system call is determined by the
.IR cmd
argument.
Each operation takes an accompanying argument,
provided via
.IR attr ,
which is a pointer to a union of type
.IR bpf_attr
(see below).
The
.I size
argument is the size of the union pointed to by
.IR attr .
The value provided in
.IR cmd
is one of the following:
.TP
.B BPF_MAP_CREATE
Create a map with and return a file descriptor that refers to the map.
.TP
.B BPF_MAP_LOOKUP_ELEM
Look up an element by key in a specified map and return its value.
.TP
.B BPF_MAP_UPDATE_ELEM
Create or update an element (key/value pair) in a specified map.
.TP
.B BPF_MAP_DELETE_ELEM
Look up and delete an element by key in a specified map.
.TP
.B BPF_MAP_GET_NEXT_KEY
Look up an element by key in a specified map and return the key
of the next element.
.TP
.B BPF_PROG_LOAD
Verify and load an eBPF program,
returning a new file descriptor associated with the program.
.P
The
.I bpf_attr
union consists of various anonymous structures that are used by different
.BR bpf ()
commands:
.in +4n
.nf
union bpf_attr {
struct { /* Used by BPF_MAP_CREATE */
__u32 map_type;
__u32 key_size; /* size of key in bytes */
__u32 value_size; /* size of value in bytes */
__u32 max_entries; /* maximum number of entries
in a map */
};
struct { /* Used by BPF_MAP_*_ELEM and BPF_MAP_GET_NEXT_KEY
commands */
__u32 map_fd;
__aligned_u64 key;
union {
__aligned_u64 value;
__aligned_u64 next_key;
};
__u64 flags;
};
struct { /* Used by BPF_PROG_LOAD */
__u32 prog_type;
__u32 insn_cnt;
__aligned_u64 insns; /* 'const struct bpf_insn *' */
__aligned_u64 license; /* 'const char *' */
__u32 log_level; /* verbosity level of verifier */
__u32 log_size; /* size of user buffer */
__aligned_u64 log_buf; /* user supplied 'char *'
buffer */
__u32 kern_version;
/* checked when prog_type=kprobe
(since Linux 4.1) */
.\" commit 2541517c32be2531e0da59dfd7efc1ce844644f5
};
} __attribute__((aligned(8)));
.fi
.in
.SS eBPF maps
Maps are a generic data structure for storage of different types of data.
They allow sharing of data between eBPF kernel programs,
and also between kernel and user-space applications.
Each map type has the following attributes:
.PD 0
.IP * 3
type
.IP *
maximum number of elements
.IP *
key size in bytes
.IP *
value size in bytes
.PD
.PP
The following wrapper functions demonstrate how various
.BR bpf ()
commands can be used to access the maps.
The functions use the
.IR cmd
argument to invoke different operations.
.TP
.B BPF_MAP_CREATE
The
.B BPF_MAP_CREATE
command creates a new map,
returning a new file descriptor that refers to the map.
.in +4n
.nf
int
bpf_create_map(enum bpf_map_type map_type, int key_size,
int value_size, int max_entries)
{
union bpf_attr attr = {
.map_type = map_type,
.key_size = key_size,
.value_size = value_size,
.max_entries = max_entries
};
return bpf(BPF_MAP_CREATE, &attr, sizeof(attr));
}
.fi
.in
The new map has the type specified by
.IR map_type ,
and attributes as specified in
.IR key_size ,
.IR value_size ,
and
.IR max_entries .
.\" FIXME!! In the next sentence, what does "process-local" mean?
On success, this operation returns a process-local file descriptor.
On error, \-1 is returned and
.I errno
is set to
.BR EINVAL ,
.BR EPERM ,
or
.BR ENOMEM .
The attributes
.I key_size
and
.I value_size
will be used by the verifier during program loading to check that the program
is calling
.BR bpf_map_*_elem ()
helper functions with a correctly initialized
.I key
and to check that the program doesn't access the map element
.I value
beyond the specified
.IR value_size .
For example, when a map is created with a
.IR key_size
of 8 and the eBPF program calls
.in +4n
.nf
bpf_map_lookup_elem(map_fd, fp - 4)
.fi
.in
the program will be rejected,
since the in-kernel helper function
bpf_map_lookup_elem(map_fd, void *key)
expects to read 8 bytes from
.I key
pointer, but
.IR "fp\ -\ 4"
.\" FIXME!! I'm lost! What is 'fp' in this context?
starting address will cause out-of-bounds stack access.
Similarly, when a map is created with a
.I value_size
of 1 and the eBPF program contains
.in +4n
.nf
value = bpf_map_lookup_elem(...);
*(u32 *) value = 1;
.fi
.in
the program will be rejected, since it accesses the
.I value
pointer beyond the specified 1 byte
.I value_size
limit.
Currently, the following values are supported for
.IR map_type :
.in +4n
.nf
enum bpf_map_type {
BPF_MAP_TYPE_UNSPEC, /* Reserve 0 as invalid map type */
BPF_MAP_TYPE_HASH,
BPF_MAP_TYPE_ARRAY,
BPF_MAP_TYPE_PROG_ARRAY,
};
.fi
.in
.I map_type
selects one of the available map implementations in the kernel.
.\" FIXME We need an explanation of why one might choose each of
.\" these map implementations
For all map types,
eBPF programs access maps with the same
.BR bpf_map_lookup_elem ()
and
.BR bpf_map_update_elem ()
helper functions.
Further details of the various map types are given below.
.TP
.B BPF_MAP_LOOKUP_ELEM
The
.B BPF_MAP_LOOKUP_ELEM
command looks up an element with a given
.I key
in the map referred to by the file descriptor
.IR fd .
.in +4n
.nf
int
bpf_lookup_elem(int fd, void *key, void *value)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
.value = ptr_to_u64(value),
};
return bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
}
.fi
.in
If an element is found,
the operation returns zero and stores the element's value into
.IR value ,
which must point to a buffer of
.I value_size
bytes.
If no element is found, the operation returns \-1 and sets
.I errno
to
.BR ENOENT .
.TP
.B BPF_MAP_UPDATE_ELEM
The
.B BPF_MAP_UPDATE_ELEM
command
creates or updates an element with a given
.I key/value
in the map referred to by the file descriptor
.IR fd .
.in +4n
.nf
int
bpf_update_elem(int fd, void *key, void *value, __u64 flags)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
.value = ptr_to_u64(value),
.flags = flags,
};
return bpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr));
}
.fi
.in
The
.I flags
argument should be specified as one of the following:
.RS
.TP
.B BPF_ANY
Create a new element or update an existing element.
.TP
.B BPF_NOEXIST
Create a new element only if it did not exist.
.TP
.B BPF_EXIST
Update an existing element.
.RE
.IP
On success, the operation returns zero.
On error, \-1 is returned and
.I errno
is set to
.BR EINVAL ,
.BR EPERM ,
.BR ENOMEM ,
or
.BR E2BIG .
.B E2BIG
indicates that the number of elements in the map reached the
.I max_entries
limit specified at map creation time.
.B EEXIST
will be returned if
.I flags
specifies
.B BPF_NOEXIST
and the element with
.I key
already exists in the map.
.B ENOENT
will be returned if
.I flags
specifies
.B BPF_EXIST
and the element with
.I key
doesn't exist in the map.
.TP
.B BPF_MAP_DELETE_ELEM
The
.B BPF_MAP_DELETE_ELEM
command
deleted the element whose key is
.I key
from the map referred to by the file descriptor
.IR fd .
.in +4n
.nf
int
bpf_delete_elem(int fd, void *key)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
};
return bpf(BPF_MAP_DELETE_ELEM, &attr, sizeof(attr));
}
.fi
.in
On success, zero is returned.
If the element is not found, \-1 is returned and
.I errno
is set to
.BR ENOENT .
.TP
.B BPF_MAP_GET_NEXT_KEY
The
.B BPF_MAP_GET_NEXT_KEY
command looks up an element by
.I key
in the map referred to by the file descriptor
.IR fd
and sets the
.I next_key
pointer to the key of the next element.
.nf
.in +4n
int
bpf_get_next_key(int fd, void *key, void *next_key)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
.next_key = ptr_to_u64(next_key),
};
return bpf(BPF_MAP_GET_NEXT_KEY, &attr, sizeof(attr));
}
.fi
.in
If
.I key
is found, the operation returns zero and sets the
.I next_key
pointer to the key of the next element.
If
.I key
is not found, the operation returns zero and sets the
.I next_key
pointer to the key of the first element.
If
.I key
is the last element, \-1 is returned and
.I errno
is set to
.BR ENOENT .
Other possible
.I errno
values are
.BR ENOMEM ,
.BR EFAULT ,
.BR EPERM ,
and
.BR EINVAL .
This method can be used to iterate over all elements in the map.
.TP
.B close(map_fd)
Delete the map referred to by the file descriptor
.IR map_fd .
When the user-space program that created a map exits, all maps will
be deleted automatically (but see NOTES).
.\"
.SS eBPF map types
The following map types are supported:
.TP
.B BPF_MAP_TYPE_HASH
.\" commit 0f8e4bd8a1fc8c4185f1630061d0a1f2d197a475
.\" FIXME!! Please review the following list of points, which draws
.\" heavily from the commit message, but reworks the text significantly
.\" and so may have introduced errors.
Hash-table maps have the following characteristics:
.RS
.IP * 3
Maps are created and destroyed by user-space programs.
Both user-space and eBPF programs
can perform lookuo, update, and delete operations.
.IP *
The kernel takes care of allocating and freeing key/value pairs.
.IP *
The
.BR map_update_elem ()
helper with fail to insert new element when the
.I max_entries
limit is reached.
(This ensures that eBPF programs cannot exhaust memory.)
.IP *
.BR map_update_elem ()
replaces existing elements atomically.
.RE
.IP
Hash-table maps are
optimized for speed of lookup.
.TP
.B BPF_MAP_TYPE_ARRAY
.\" commit 28fbcfa08d8ed7c5a50d41a0433aad222835e8e3
.\" FIXME!! Please review the following list of points, which draws
.\" heavily from the commit message, but reworks the text significantly
.\" and so may have introduced errors.
Array maps have the following characteristics:
.RS
.IP * 3
Optimized for fastest possible lookup.
In the future ithe verifier/JIT compiler
may recognize lookup() operations that employ a constant key
and optimize it into constant pointer.
It is possible to optimize a non-constant
key into direct pointer arithmetic as well, since pointers and
.I value_size
are constant for the life of the eBPF program.
In other words,
.BR array_map_lookup_elem ()
may be 'inlined' by the verifier/JIT compiler
while preserving concurrent access to this map from user space.
.IP *
All array elements pre-allocated and zero initialized at init time
.IP *
The key is an array index, and must be exactly four bytes.
.IP *
.BR map_delete_elem ()
fails with the error
.BR EINVAL ,
since elements cannot be deleted.
.IP *
.BR map_update_elem ()
replaces elements in an non-atomic fashion;
for atomic updates, a hash-table map should be used instead.
.RE
.IP
Among the uses for array maps are the following:
.RS
.IP * 3
As "global" eBPF variables: an array of 1 element whose key is (index) 0
and where the value is a collection of 'global' variables which
eBPF programs can use to keep state between events.
.IP *
Aggregation of tracing events into a fixed set of buckets.
.RE
.TP
.BR BPF_MAP_TYPE_PROG_ARRAY " (since Linux 4.2)"
.\" FIXME: we need documentation of BPF_MAP_TYPE_PROG_ARRAY
[To be completed]
.\"
.SS eBPF programs
The
.B BPF_PROG_LOAD
command is used to load an eBPF program into the kernel.
The return value for this command is a new file descriptor associated
with this eBPF program.
.in +4n
.nf
char bpf_log_buf[LOG_BUF_SIZE];
int
bpf_prog_load(enum bpf_prog_type prog_type,
const struct bpf_insn *insns, int insn_cnt,
const char *license)
{
union bpf_attr attr = {
.prog_type = prog_type,
.insns = ptr_to_u64(insns),
.insn_cnt = insn_cnt,
.license = ptr_to_u64(license),
.log_buf = ptr_to_u64(bpf_log_buf),
.log_size = LOG_BUF_SIZE,
.log_level = 1,
};
return bpf(BPF_PROG_LOAD, &attr, sizeof(attr));
}
.fi
.in
.I prog_type
is one of the available program types:
.in +4n
.nf
enum bpf_prog_type {
BPF_PROG_TYPE_UNSPEC, /* Reserve 0 as invalid
program type */
BPF_PROG_TYPE_SOCKET_FILTER,
BPF_PROG_TYPE_KPROBE,
BPF_PROG_TYPE_SCHED_CLS,
BPF_PROG_TYPE_SCHED_ACT,
};
.fi
.in
For further details of eBPF program types, see below.
The remaining fields of
.I bpf_attr
are set as follows:
.IP * 3
.I insns
is an array of
.I "struct bpf_insn"
instructions.
.IP *
.I insn_cnt
is the number of instructions in the program referred to by
.IR insns .
.IP *
.I license
is a license string, which must be GPL compatible to call helper functions
marked
.IR gpl_only .
.IP *
.I log_buf
is a pointer to a caller-allocated buffer in which the in-kernel
verifier can store the verification log.
This log is a multi-line string that can be checked by
the program author in order to understand how the verifier came to
the conclusion that the BPF program is unsafe.
The format of the output can change at any time as the verifier evolves.
.IP *
.I log_size
size of the buffer pointed to by
.IR log_bug .
If the size of the buffer is not large enough to store all
verifier messages, \-1 is returned and
.I errno
is set to
.BR ENOSPC .
.IP *
.I log_level
verbosity level of the verifier.
A value of zero means that the verifier will
not provide a log.
.P
Applying
.BR close (2)
to the file descriptor returned by
.B BPF_PROG_LOAD
will unload the eBPF program (but see NOTES).
Maps are accessible from eBPF programs and are used to exchange data between
eBPF programs and between eBPF programs and user-space programs.
For example,
eBPF programs can process various events (like kprobe, packets) and
store their data into a map,
and user-space programs can then fetch data from the map.
Conversely, user-space programs can use a map as a configuration mechanism,
populating the map with values checked by the eBPF program,
which then modifies its behavior on the fly according to those values.
.SS eBPF program types
By picking
.IR prog_type ,
the program author selects a set of helper functions that can be called from
the eBPF program and the corresponding format of
.I struct bpf_context
(which is the data blob passed into the eBPF program as the first argument).
For example, programs loaded with a
.I prog_type
of
.B BPF_PROG_TYPE_SOCKET_FILTER
may call the
.BR bpf_map_lookup_elem ()
helper,
whereas some other program types may not be able to employ this helper.
The set of functions available to eBPF programs of a given type may increase
in the future.
The following program types are supported:
.TP
.BR BPF_PROG_TYPE_SOCKET_FILTER " (since Linux 3.19)"
Currently, the set of functions for
.B BPF_PROG_TYPE_SOCKET_FILTER
is:
.in +4n
.nf
bpf_map_lookup_elem(map_fd, void *key)
/* look up key in a map_fd */
bpf_map_update_elem(map_fd, void *key, void *value)
/* update key/value */
bpf_map_delete_elem(map_fd, void *key)
/* delete key in a map_fd */
.fi
.in
.\" FIXME The following paragraph needs amending. Alexei commented:
.\"
.\" Actually now in case of SOCKET_FILTER, SCHED_CLS, SCHED_ACT
.\" the program can now access skb fields.
.\" See 'struct __sk_buff' and commit 9bac3d6d548e5
.\"
.\" Do we want some text here to explain how the program access __sk_buff?
The
.I bpf_context
argument is a pointer to a
.IR "struct sk_buff" .
Programs cannot access the fields of
.I sk_buff
directly.
.\"
.TP
.BR BPF_PROG_TYPE_KPROBE " (since Linux 4.1)
.\" commit 2541517c32be2531e0da59dfd7efc1ce844644f5
[To be documented]
.\" FIXME Document this program type
.\" Describe allowed helper functions for this program type
.\" Describe bpf_context for this program type
.\" FIXME We need text here to describe 'kern_version'
.TP
.BR BPF_PROG_TYPE_SCHED_CLS " (since Linux 4.1)
.\" commit 96be4325f443dbbfeb37d2a157675ac0736531a1
.\" commit e2e9b6541dd4b31848079da80fe2253daaafb549
[To be documented]
.\" FIXME Document this program type
.\" Describe allowed helper functions for this program type
.\" Describe bpf_context for this program type
.TP
.BR BPF_PROG_TYPE_SCHED_ACT " (since Linux 4.1)
.\" commit 94caee8c312d96522bcdae88791aaa9ebcd5f22c
.\" commit a8cb5f556b567974d75ea29c15181c445c541b1f
[To be documented]
.\" FIXME Document this program type
.\" Describe allowed helper functions for this program type
.\" Describe bpf_context for this program type
.SS Events
Once a program is loaded, it can be attached to an event.
Various kernel subsystems have different ways to do so.
Since Linux 3.19,
.\" commit 89aa075832b0da4402acebd698d0411dcc82d03e
the following call will attach the program
.I prog_fd
to the socket
.IR sockfd ,
which was created by an earlier call to
.BR socket (2):
.in +4n
.nf
setsockopt(sockfd, SOL_SOCKET, SO_ATTACH_BPF,
&prog_fd, sizeof(prog_fd));
.fi
.in
Since Linux 4.1,
.\" commit 2541517c32be2531e0da59dfd7efc1ce844644f5
the following call may be used to attach
the eBPF program referred to by the file descriptor
.I prog_fd
to a perf event file descriptor,
.IR event_fd ,
that was created by a previous call to
.BR perf_event_open (2):
.in +4n
.nf
ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd);
.fi
.in
.\"
.\"
.SH EXAMPLES
.nf
/* bpf+sockets example:
* 1. create array map of 256 elements
* 2. load program that counts number of packets received
* r0 = skb->data[ETH_HLEN + offsetof(struct iphdr, protocol)]
* map[r0]++
* 3. attach prog_fd to raw socket via setsockopt()
* 4. print number of received TCP/UDP packets every second
*/
int
main(int argc, char **argv)
{
int sock, map_fd, prog_fd, key;
long long value = 0, tcp_cnt, udp_cnt;
map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(key),
sizeof(value), 256);
if (map_fd < 0) {
printf("failed to create map '%s'\\n", strerror(errno));
/* likely not run as root */
return 1;
}
struct bpf_insn prog[] = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = r1 */
BPF_LD_ABS(BPF_B, ETH_HLEN + offsetof(struct iphdr, protocol)),
/* r0 = ip->proto */
BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_0, -4),
/* *(u32 *)(fp - 4) = r0 */
BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), /* r2 = fp */
BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), /* r2 = r2 - 4 */
BPF_LD_MAP_FD(BPF_REG_1, map_fd), /* r1 = map_fd */
BPF_CALL_FUNC(BPF_FUNC_map_lookup_elem),
/* r0 = map_lookup(r1, r2) */
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
/* if (r0 == 0) goto pc+2 */
BPF_MOV64_IMM(BPF_REG_1, 1), /* r1 = 1 */
BPF_XADD(BPF_DW, BPF_REG_0, BPF_REG_1, 0, 0),
/* lock *(u64 *) r0 += r1 */
.\" == atomic64_add
BPF_MOV64_IMM(BPF_REG_0, 0), /* r0 = 0 */
BPF_EXIT_INSN(), /* return r0 */
};
prog_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, prog,
sizeof(prog), "GPL");
sock = open_raw_sock("lo");
assert(setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, &prog_fd,
sizeof(prog_fd)) == 0);
for (;;) {
key = IPPROTO_TCP;
assert(bpf_lookup_elem(map_fd, &key, &tcp_cnt) == 0);
key = IPPROTO_UDP
assert(bpf_lookup_elem(map_fd, &key, &udp_cnt) == 0);
printf("TCP %lld UDP %lld packets\n", tcp_cnt, udp_cnt);
sleep(1);
}
return 0;
}
.fi
Some complete working code can be found in the
.IR samples/bpf
directory in the kernel source tree.
.SH RETURN VALUE
For a successful call, the return value depends on the operation:
.TP
.B BPF_MAP_CREATE
The new file descriptor associated with the eBPF map.
.TP
.B BPF_PROG_LOAD
The new file descriptor associated with the eBPF program.
.TP
All other commands
Zero.
.PP
On error, \-1 is returned, and
.I errno
is set appropriately.
.SH ERRORS
.TP
.B EPERM
The call was made without sufficient privilege
(without the
.B CAP_SYS_ADMIN
capability).
.TP
.B ENOMEM
Cannot allocate sufficient memory.
.TP
.B EBADF
.I fd
is not an open file descriptor
.TP
.B EFAULT
One of the pointers
.RI ( key
or
.I value
or
.I log_buf
or
.IR insns )
is outside the accessible address space.
.TP
.B EINVAL
The value specified in
.I cmd
is not recognized by this kernel.
.TP
.B EINVAL
For
.BR BPF_MAP_CREATE ,
either
.I map_type
or attributes are invalid.
.TP
.B EINVAL
For
.BR BPF_MAP_*_ELEM
commands,
some of the fields of
.I "union bpf_attr"
that are not used by this command
are not set to zero.
.TP
.B EINVAL
For
.BR BPF_PROG_LOAD,
indicates an attempt to load an invalid program.
BPF programs can be deemed
einvalid due to unrecognized instructions, the use of reserved fields, jumps
out of range, infinite loops or calls of unknown functions.
.TP
.BR EACCES
For
.BR BPF_PROG_LOAD,
even though all program instructions are valid, the program has been
rejected because it was deemed unsafe.
This may be because it may have
accessed a disallowed memory region or an uninitialized stack/register or
because the function constraints don't match the actual types or because
there was a misaligned memory access.
In this case, it is recommended to call
.BR bpf ()
again with
.I log_level = 1
and examine
.I log_buf
for the specific reason provided by the verifier.
.TP
.BR ENOENT
For
.B BPF_MAP_LOOKUP_ELEM
or
.BR BPF_MAP_DELETE_ELEM ,
indicates that the element with the given
.I key
was not found.
.TP
.BR E2BIG
The BPF program is too large or a map reached the
.I max_entries
limit (maximum number of elements).
.SH VERSIONS
The
.BR bpf ()
system call first appeared in Linux 3.18.
.SH CONFORMING TO
The
.BR bpf ()
system call is Linux-specific.
.SH NOTES
In the current implementation, all
.BR bpf ()
commands require the caller to have the
.B CAP_SYS_ADMIN
capability.
.\" FIXME!! Alexei, is the following correct?
eBPF objects (maps and programs) can be shared between processes.
For example, after
.BR fork (2),
the child inherits file descriptors referring to the same eBPF objects.
In addition, file descriptors referring to eBPF objects can be
transferred over UNIX domain sockets.
File descriptors referring to eBPF objects can be duplicated
in the usual way, using
.BR dup (2)
and similar calls.
An eBPF object is deallocated only after all file descriptors
referring to the object have been closed.
eBPF programs can be written in a restricted C that is compiled (using the
.B clang
compiler) into eBPF bytecode and executed on the in-kernel virtual machine or
just-in-time compiled into native code.
(Various features are omitted from this restricted C, such as loops,
global variables, variadic functions, floating-point numbers,
and passing structures as function arguments.)
Some examples can be found in the
.I samples/bpf/*_kern.c
files in the kernel source tree.
.\" There are also examples for the tc classifier, in the iproute2
.\" project, in examples/bpf
.SH SEE ALSO
.BR seccomp (2),
.BR socket (7),
.BR tc (8),
.BR tc-bpf (8)
Both classic and extended BPF are explained in the kernel source file
.IR Documentation/networking/filter.txt .
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-22 18:43 Draft 3 of bpf(2) man page for review Michael Kerrisk (man-pages)
@ 2015-07-22 19:22 ` Alexei Starovoitov
2015-07-22 20:10 ` Michael Kerrisk (man-pages)
0 siblings, 1 reply; 10+ messages in thread
From: Alexei Starovoitov @ 2015-07-22 19:22 UTC (permalink / raw)
To: Michael Kerrisk (man-pages), Daniel Borkmann
Cc: linux-man, linux-kernel, Silvan Jegen, Walter Harms
On 7/22/15 11:43 AM, Michael Kerrisk (man-pages) wrote:
> .TH BPF 2 2015-03-10 "Linux" "Linux Programmer's Manual"
should the date be updated ?
> BPF maps are a generic data structure for storage of different data types.
> A user process can create multiple maps (with key/value-pairs being
> opaque bytes of data) and access them via file descriptors.
> eBPF programs can access maps from inside the kernel in parallel.
> .\"
> .\" FIXME!! What does the previous sentence mean?
> .\"
> .\" Isn't "from inside the kernel" redundant? (I mean: all eBPF programs
> .\" are running inside the kernel, right?)
99.9% of the time. yes. all eBPF programs are running inside the kernel,
though recently I've seen two versions of 'user space eBPF' where
kernel interpreter/x64_jit were ported to user space.
If you think 'from kernel' is redundant, just drop it.
> .\" And what does "in parallel" mean?
> .\" Would a simpler version of this sentence be correct? As in:
> .\" "Different eBPF programs can access the same maps in parallel."
yes. different eBPF programs and user space processes can access the
same maps in parallel.
> The new map has the type specified by
> .IR map_type ,
> and attributes as specified in
> .IR key_size ,
> .IR value_size ,
> and
> .IR max_entries .
> .\" FIXME!! In the next sentence, what does "process-local" mean?
> On success, this operation returns a process-local file descriptor.
Just drop this unnecessary qualifier. Just 'returns a file descriptor'
> .in +4n
> .nf
> bpf_map_lookup_elem(map_fd, fp - 4)
> .fi
> .in
>
> the program will be rejected,
> since the in-kernel helper function
>
> bpf_map_lookup_elem(map_fd, void *key)
>
> expects to read 8 bytes from
> .I key
> pointer, but
> .IR "fp\ -\ 4"
> .\" FIXME!! I'm lost! What is 'fp' in this context?
it refers to 2nd argument of 'bpf_map_lookup_elem(map_fd, fp - 4)'
fp = top of the stack.
fp - 4 = pointer to 4 bytes below top of the stack.
So 8 byte access from there will be out of bounds.
> The following map types are supported:
> .TP
> .B BPF_MAP_TYPE_HASH
> .\" commit 0f8e4bd8a1fc8c4185f1630061d0a1f2d197a475
> .\" FIXME!! Please review the following list of points, which draws
> .\" heavily from the commit message, but reworks the text significantly
> .\" and so may have introduced errors.
> Hash-table maps have the following characteristics:
> .RS
> .IP * 3
> Maps are created and destroyed by user-space programs.
> Both user-space and eBPF programs
> can perform lookuo, update, and delete operations.
typo 'lookup'
> .IP *
> The kernel takes care of allocating and freeing key/value pairs.
> .IP *
> The
> .BR map_update_elem ()
> helper with fail to insert new element when the
> .I max_entries
> limit is reached.
> (This ensures that eBPF programs cannot exhaust memory.)
> .IP *
> .BR map_update_elem ()
> replaces existing elements atomically.
> .RE
> .IP
> Hash-table maps are
> optimized for speed of lookup.
> .TP
> .B BPF_MAP_TYPE_ARRAY
> .\" commit 28fbcfa08d8ed7c5a50d41a0433aad222835e8e3
> .\" FIXME!! Please review the following list of points, which draws
> .\" heavily from the commit message, but reworks the text significantly
> .\" and so may have introduced errors.
> Array maps have the following characteristics:
> .RS
> .IP * 3
> Optimized for fastest possible lookup.
> In the future ithe verifier/JIT compiler
typo 'the'
> may recognize lookup() operations that employ a constant key
> and optimize it into constant pointer.
> It is possible to optimize a non-constant
> key into direct pointer arithmetic as well, since pointers and
> .I value_size
> are constant for the life of the eBPF program.
> In other words,
> .BR array_map_lookup_elem ()
> may be 'inlined' by the verifier/JIT compiler
> while preserving concurrent access to this map from user space.
> .IP *
> All array elements pre-allocated and zero initialized at init time
> .IP *
> The key is an array index, and must be exactly four bytes.
> .IP *
> .BR map_delete_elem ()
> fails with the error
> .BR EINVAL ,
> since elements cannot be deleted.
> .IP *
> .BR map_update_elem ()
> replaces elements in an non-atomic fashion;
> for atomic updates, a hash-table map should be used instead.
the description of hash and array maps looks good.
> .\" FIXME The following paragraph needs amending. Alexei commented:
> .\"
> .\" Actually now in case of SOCKET_FILTER, SCHED_CLS, SCHED_ACT
> .\" the program can now access skb fields.
> .\" See 'struct __sk_buff' and commit 9bac3d6d548e5
> .\"
> .\" Do we want some text here to explain how the program access __sk_buff?
I think commit 9bac3d6d548e5 tried to explain it, but translating
that to english would be nice :)
> .\" FIXME!! Alexei, is the following correct?
> eBPF objects (maps and programs) can be shared between processes.
> For example, after
> .BR fork (2),
> the child inherits file descriptors referring to the same eBPF objects.
> In addition, file descriptors referring to eBPF objects can be
> transferred over UNIX domain sockets.
> File descriptors referring to eBPF objects can be duplicated
> in the usual way, using
> .BR dup (2)
> and similar calls.
> An eBPF object is deallocated only after all file descriptors
> referring to the object have been closed.
yes. all correct.
> eBPF programs can be written in a restricted C that is compiled (using the
> .B clang
> compiler) into eBPF bytecode and executed on the in-kernel virtual machine or
> just-in-time compiled into native code.
> (Various features are omitted from this restricted C, such as loops,
> global variables, variadic functions, floating-point numbers,
> and passing structures as function arguments.)
> Some examples can be found in the
> .I samples/bpf/*_kern.c
> files in the kernel source tree.
thanks. whole thing looks good.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-22 19:22 ` Alexei Starovoitov
@ 2015-07-22 20:10 ` Michael Kerrisk (man-pages)
2015-07-22 22:12 ` Alexei Starovoitov
2015-07-23 9:31 ` Daniel Borkmann
0 siblings, 2 replies; 10+ messages in thread
From: Michael Kerrisk (man-pages) @ 2015-07-22 20:10 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann
Cc: mtk.manpages, linux-man, linux-kernel, Silvan Jegen, Walter Harms
On 07/22/2015 09:22 PM, Alexei Starovoitov wrote:
> On 7/22/15 11:43 AM, Michael Kerrisk (man-pages) wrote:
>> .TH BPF 2 2015-03-10 "Linux" "Linux Programmer's Manual"
>
> should the date be updated ?
It'll get updated later, by scripts.
>> BPF maps are a generic data structure for storage of different data types.
>> A user process can create multiple maps (with key/value-pairs being
>> opaque bytes of data) and access them via file descriptors.
>> eBPF programs can access maps from inside the kernel in parallel.
>> .\"
>> .\" FIXME!! What does the previous sentence mean?
>> .\"
>> .\" Isn't "from inside the kernel" redundant? (I mean: all eBPF programs
>> .\" are running inside the kernel, right?)
>
> 99.9% of the time. yes. all eBPF programs are running inside the kernel,
> though recently I've seen two versions of 'user space eBPF' where
> kernel interpreter/x64_jit were ported to user space.
> If you think 'from kernel' is redundant, just drop it.
Okay. Done.
>> .\" And what does "in parallel" mean?
>> .\" Would a simpler version of this sentence be correct? As in:
>> .\" "Different eBPF programs can access the same maps in parallel."
>
> yes. different eBPF programs and user space processes can access the
> same maps in parallel.
Okay.
>> The new map has the type specified by
>> .IR map_type ,
>> and attributes as specified in
>> .IR key_size ,
>> .IR value_size ,
>> and
>> .IR max_entries .
>> .\" FIXME!! In the next sentence, what does "process-local" mean?
>> On success, this operation returns a process-local file descriptor.
>
> Just drop this unnecessary qualifier. Just 'returns a file descriptor'
Done.
>> .in +4n
>> .nf
>> bpf_map_lookup_elem(map_fd, fp - 4)
>> .fi
>> .in
>>
>> the program will be rejected,
>> since the in-kernel helper function
>>
>> bpf_map_lookup_elem(map_fd, void *key)
>>
>> expects to read 8 bytes from
>> .I key
>> pointer, but
>> .IR "fp\ -\ 4"
>> .\" FIXME!! I'm lost! What is 'fp' in this context?
>
> it refers to 2nd argument of 'bpf_map_lookup_elem(map_fd, fp - 4)'
> fp = top of the stack.
> fp - 4 = pointer to 4 bytes below top of the stack.
> So 8 byte access from there will be out of bounds.
Okay. I added some words mentioning that 'fp' is top of stack.
>> The following map types are supported:
>> .TP
>> .B BPF_MAP_TYPE_HASH
>> .\" commit 0f8e4bd8a1fc8c4185f1630061d0a1f2d197a475
>> .\" FIXME!! Please review the following list of points, which draws
>> .\" heavily from the commit message, but reworks the text significantly
>> .\" and so may have introduced errors.
>> Hash-table maps have the following characteristics:
>> .RS
>> .IP * 3
>> Maps are created and destroyed by user-space programs.
>> Both user-space and eBPF programs
>> can perform lookuo, update, and delete operations.
>
> typo 'lookup'
Thanks, fixed.
>> .IP *
>> The kernel takes care of allocating and freeing key/value pairs.
>> .IP *
>> The
>> .BR map_update_elem ()
>> helper with fail to insert new element when the
>> .I max_entries
>> limit is reached.
>> (This ensures that eBPF programs cannot exhaust memory.)
>> .IP *
>> .BR map_update_elem ()
>> replaces existing elements atomically.
>> .RE
>> .IP
>> Hash-table maps are
>> optimized for speed of lookup.
>> .TP
>> .B BPF_MAP_TYPE_ARRAY
>> .\" commit 28fbcfa08d8ed7c5a50d41a0433aad222835e8e3
>> .\" FIXME!! Please review the following list of points, which draws
>> .\" heavily from the commit message, but reworks the text significantly
>> .\" and so may have introduced errors.
>> Array maps have the following characteristics:
>> .RS
>> .IP * 3
>> Optimized for fastest possible lookup.
>> In the future ithe verifier/JIT compiler
>
> typo 'the'
Fixed.
>> may recognize lookup() operations that employ a constant key
>> and optimize it into constant pointer.
>> It is possible to optimize a non-constant
>> key into direct pointer arithmetic as well, since pointers and
>> .I value_size
>> are constant for the life of the eBPF program.
>> In other words,
>> .BR array_map_lookup_elem ()
>> may be 'inlined' by the verifier/JIT compiler
>> while preserving concurrent access to this map from user space.
>> .IP *
>> All array elements pre-allocated and zero initialized at init time
>> .IP *
>> The key is an array index, and must be exactly four bytes.
>> .IP *
>> .BR map_delete_elem ()
>> fails with the error
>> .BR EINVAL ,
>> since elements cannot be deleted.
>> .IP *
>> .BR map_update_elem ()
>> replaces elements in an non-atomic fashion;
>> for atomic updates, a hash-table map should be used instead.
>
> the description of hash and array maps looks good.
Okay. Thanks for checking.
>> .\" FIXME The following paragraph needs amending. Alexei commented:
>> .\"
>> .\" Actually now in case of SOCKET_FILTER, SCHED_CLS, SCHED_ACT
>> .\" the program can now access skb fields.
>> .\" See 'struct __sk_buff' and commit 9bac3d6d548e5
>> .\"
>> .\" Do we want some text here to explain how the program access __sk_buff?
>
> I think commit 9bac3d6d548e5 tried to explain it, but translating
> that to english would be nice :)
Yes, but my C-to-English translator failed.
>> .\" FIXME!! Alexei, is the following correct?
>> eBPF objects (maps and programs) can be shared between processes.
>> For example, after
>> .BR fork (2),
>> the child inherits file descriptors referring to the same eBPF objects.
>> In addition, file descriptors referring to eBPF objects can be
>> transferred over UNIX domain sockets.
>> File descriptors referring to eBPF objects can be duplicated
>> in the usual way, using
>> .BR dup (2)
>> and similar calls.
>> An eBPF object is deallocated only after all file descriptors
>> referring to the object have been closed.
>
> yes. all correct.
Thanks.
>> eBPF programs can be written in a restricted C that is compiled (using the
>> .B clang
>> compiler) into eBPF bytecode and executed on the in-kernel virtual machine or
>> just-in-time compiled into native code.
>> (Various features are omitted from this restricted C, such as loops,
>> global variables, variadic functions, floating-point numbers,
>> and passing structures as function arguments.)
>> Some examples can be found in the
>> .I samples/bpf/*_kern.c
>> files in the kernel source tree.
>
> thanks. whole thing looks good.
Thanks.
Below is the current rendered version of the man page.
Cheers,
Michael
NAME
bpf - perform a command on an extended eBPF map or program
SYNOPSIS
#include <linux/bpf.h>
int bpf(int cmd, union bpf_attr *attr, unsigned int size);
DESCRIPTION
The bpf() system call performs a range of operations related to
extended Berkeley Packet Filters. Extended BPF (or eBPF) is sim‐
ilar to the original ("classic") BPF (cBPF) used to filter net‐
work packets. For both cBPF and eBPF programs, the kernel stati‐
cally analyzes the programs before loading them, in order to
ensure that they cannot harm the running system.
eBPF extends cBPF in multiple ways, including the ability to call
a fixed set of in-kernel helper functions (via the BPF_CALL
opcode extension provided by eBPF) and access shared data struc‐
tures such as eBPF maps.
Extended BPF Design/Architecture
BPF maps are a generic data structure for storage of different
data types. A user process can create multiple maps (with
key/value-pairs being opaque bytes of data) and access them via
file descriptors. Differnt eBPF programs can access the same
maps in parallel. It's up to the user process and eBPF program
to decide what they store inside maps.
eBPF programs are similar to kernel modules. They are loaded by
the user process and automatically unloaded when the process
exits. Each program is a set of instructions that is safe to run
until its completion. An in-kernel verifier statically deter‐
mines that the eBPF program terminates and is safe to execute.
During verification, the kernel increments reference counts for
each of the maps that the eBPF program uses, so that the selected
maps cannot be removed until the program is unloaded.
eBPF programs can be attached to different events. These events
can be the arrival of network packets, tracing events, classifi‐
cation event by qdisc (for eBPF programs attached to a tc(8)
classifier), and other types that may be added in the future. A
new event triggers execution of the eBPF program, which may store
information about the event in eBPF maps. Beyond storing data,
eBPF programs may call a fixed set of in-kernel helper functions.
The same eBPF program can be attached to multiple events and dif‐
ferent eBPF programs can access the same map:
tracing tracing tracing packet packet
event A event B event C on eth0 on eth1
| | | | |
| | | | |
--> tracing <-- tracing socket socket
prog_1 prog_2 prog_3 prog_4
| | | |
|--- -----| |-------| map_3
map_1 map_2
Arguments
The operation to be performed by the bpf() system call is deter‐
mined by the cmd argument. Each operation takes an accompanying
argument, provided via attr, which is a pointer to a union of
type bpf_attr (see below). The size argument is the size of the
union pointed to by attr.
The value provided in cmd is one of the following:
BPF_MAP_CREATE
Create a map with and return a file descriptor that refers
to the map.
BPF_MAP_LOOKUP_ELEM
Look up an element by key in a specified map and return
its value.
BPF_MAP_UPDATE_ELEM
Create or update an element (key/value pair) in a speci‐
fied map.
BPF_MAP_DELETE_ELEM
Look up and delete an element by key in a specified map.
BPF_MAP_GET_NEXT_KEY
Look up an element by key in a specified map and return
the key of the next element.
BPF_PROG_LOAD
Verify and load an eBPF program, returning a new file
descriptor associated with the program.
The bpf_attr union consists of various anonymous structures that
are used by different bpf() commands:
union bpf_attr {
struct { /* Used by BPF_MAP_CREATE */
__u32 map_type;
__u32 key_size; /* size of key in bytes */
__u32 value_size; /* size of value in bytes */
__u32 max_entries; /* maximum number of entries
in a map */
};
struct { /* Used by BPF_MAP_*_ELEM and BPF_MAP_GET_NEXT_KEY
commands */
__u32 map_fd;
__aligned_u64 key;
union {
__aligned_u64 value;
__aligned_u64 next_key;
};
__u64 flags;
};
struct { /* Used by BPF_PROG_LOAD */
__u32 prog_type;
__u32 insn_cnt;
__aligned_u64 insns; /* 'const struct bpf_insn *' */
__aligned_u64 license; /* 'const char *' */
__u32 log_level; /* verbosity level of verifier */
__u32 log_size; /* size of user buffer */
__aligned_u64 log_buf; /* user supplied 'char *'
buffer */
__u32 kern_version;
/* checked when prog_type=kprobe
(since Linux 4.1) */
};
} __attribute__((aligned(8)));
eBPF maps
Maps are a generic data structure for storage of different types
of data. They allow sharing of data between eBPF kernel pro‐
grams, and also between kernel and user-space applications.
Each map type has the following attributes:
* type
* maximum number of elements
* key size in bytes
* value size in bytes
The following wrapper functions demonstrate how various bpf()
commands can be used to access the maps. The functions use the
cmd argument to invoke different operations.
BPF_MAP_CREATE
The BPF_MAP_CREATE command creates a new map, returning a
new file descriptor that refers to the map.
int
bpf_create_map(enum bpf_map_type map_type, int key_size,
int value_size, int max_entries)
{
union bpf_attr attr = {
.map_type = map_type,
.key_size = key_size,
.value_size = value_size,
.max_entries = max_entries
};
return bpf(BPF_MAP_CREATE, &attr, sizeof(attr));
}
The new map has the type specified by map_type, and
attributes as specified in key_size, value_size, and
max_entries. On success, this operation returns a file
descriptor. On error, -1 is returned and errno is set to
EINVAL, EPERM, or ENOMEM.
The attributes key_size and value_size will be used by the
verifier during program loading to check that the program
is calling bpf_map_*_elem() helper functions with a cor‐
rectly initialized key and to check that the program
doesn't access the map element value beyond the specified
value_size. For example, when a map is created with a
key_size of 8 and the eBPF program calls
bpf_map_lookup_elem(map_fd, fp - 4)
the program will be rejected, since the in-kernel helper
function
bpf_map_lookup_elem(map_fd, void *key)
expects to read 8 bytes from the location pointed to by
key, but the fp - 4 (where fp is the top of the stack)
starting address will cause out-of-bounds stack access.
Similarly, when a map is created with a value_size of 1
and the eBPF program contains
value = bpf_map_lookup_elem(...);
*(u32 *) value = 1;
the program will be rejected, since it accesses the value
pointer beyond the specified 1 byte value_size limit.
Currently, the following values are supported for
map_type:
enum bpf_map_type {
BPF_MAP_TYPE_UNSPEC, /* Reserve 0 as invalid map type */
BPF_MAP_TYPE_HASH,
BPF_MAP_TYPE_ARRAY,
BPF_MAP_TYPE_PROG_ARRAY,
};
map_type selects one of the available map implementations
in the kernel. For all map types, eBPF programs access
maps with the same bpf_map_lookup_elem() and
bpf_map_update_elem() helper functions. Further details
of the various map types are given below.
BPF_MAP_LOOKUP_ELEM
The BPF_MAP_LOOKUP_ELEM command looks up an element with a
given key in the map referred to by the file descriptor
fd.
int
bpf_lookup_elem(int fd, void *key, void *value)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
.value = ptr_to_u64(value),
};
return bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
}
If an element is found, the operation returns zero and
stores the element's value into value, which must point to
a buffer of value_size bytes.
If no element is found, the operation returns -1 and sets
errno to ENOENT.
BPF_MAP_UPDATE_ELEM
The BPF_MAP_UPDATE_ELEM command creates or updates an ele‐
ment with a given key/value in the map referred to by the
file descriptor fd.
int
bpf_update_elem(int fd, void *key, void *value, __u64 flags)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
.value = ptr_to_u64(value),
.flags = flags,
};
return bpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr));
}
The flags argument should be specified as one of the fol‐
lowing:
BPF_ANY
Create a new element or update an existing element.
BPF_NOEXIST
Create a new element only if it did not exist.
BPF_EXIST
Update an existing element.
On success, the operation returns zero. On error, -1 is
returned and errno is set to EINVAL, EPERM, ENOMEM, or
E2BIG. E2BIG indicates that the number of elements in the
map reached the max_entries limit specified at map cre‐
ation time. EEXIST will be returned if flags specifies
BPF_NOEXIST and the element with key already exists in the
map. ENOENT will be returned if flags specifies BPF_EXIST
and the element with key doesn't exist in the map.
BPF_MAP_DELETE_ELEM
The BPF_MAP_DELETE_ELEM command deleted the element whose
key is key from the map referred to by the file descriptor
fd.
int
bpf_delete_elem(int fd, void *key)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
};
return bpf(BPF_MAP_DELETE_ELEM, &attr, sizeof(attr));
}
On success, zero is returned. If the element is not
found, -1 is returned and errno is set to ENOENT.
BPF_MAP_GET_NEXT_KEY
The BPF_MAP_GET_NEXT_KEY command looks up an element by
key in the map referred to by the file descriptor fd and
sets the next_key pointer to the key of the next element.
int
bpf_get_next_key(int fd, void *key, void *next_key)
{
union bpf_attr attr = {
.map_fd = fd,
.key = ptr_to_u64(key),
.next_key = ptr_to_u64(next_key),
};
return bpf(BPF_MAP_GET_NEXT_KEY, &attr, sizeof(attr));
}
If key is found, the operation returns zero and sets the
next_key pointer to the key of the next element. If key
is not found, the operation returns zero and sets the
next_key pointer to the key of the first element. If key
is the last element, -1 is returned and errno is set to
ENOENT. Other possible errno values are ENOMEM, EFAULT,
EPERM, and EINVAL. This method can be used to iterate
over all elements in the map.
close(map_fd)
Delete the map referred to by the file descriptor map_fd.
When the user-space program that created a map exits, all
maps will be deleted automatically (but see NOTES).
eBPF map types
The following map types are supported:
BPF_MAP_TYPE_HASH
Hash-table maps have the following characteristics:
* Maps are created and destroyed by user-space programs.
Both user-space and eBPF programs can perform lookup,
update, and delete operations.
* The kernel takes care of allocating and freeing
key/value pairs.
* The map_update_elem() helper with fail to insert new
element when the max_entries limit is reached. (This
ensures that eBPF programs cannot exhaust memory.)
* map_update_elem() replaces existing elements atomi‐
cally.
Hash-table maps are optimized for speed of lookup.
BPF_MAP_TYPE_ARRAY
Array maps have the following characteristics:
* Optimized for fastest possible lookup. In the future
the verifier/JIT compiler may recognize lookup() opera‐
tions that employ a constant key and optimize it into
constant pointer. It is possible to optimize a non-
constant key into direct pointer arithmetic as well,
since pointers and value_size are constant for the life
of the eBPF program. In other words,
array_map_lookup_elem() may be 'inlined' by the veri‐
fier/JIT compiler while preserving concurrent access to
this map from user space.
* All array elements pre-allocated and zero initialized
at init time
* The key is an array index, and must be exactly four
bytes.
* map_delete_elem() fails with the error EINVAL, since
elements cannot be deleted.
* map_update_elem() replaces elements in an non-atomic
fashion; for atomic updates, a hash-table map should be
used instead.
Among the uses for array maps are the following:
* As "global" eBPF variables: an array of 1 element whose
key is (index) 0 and where the value is a collection of
'global' variables which eBPF programs can use to keep
state between events.
* Aggregation of tracing events into a fixed set of buck‐
ets.
BPF_MAP_TYPE_PROG_ARRAY (since Linux 4.2)
[To be completed]
eBPF programs
The BPF_PROG_LOAD command is used to load an eBPF program into
the kernel. The return value for this command is a new file
descriptor associated with this eBPF program.
char bpf_log_buf[LOG_BUF_SIZE];
int
bpf_prog_load(enum bpf_prog_type prog_type,
const struct bpf_insn *insns, int insn_cnt,
const char *license)
{
union bpf_attr attr = {
.prog_type = prog_type,
.insns = ptr_to_u64(insns),
.insn_cnt = insn_cnt,
.license = ptr_to_u64(license),
.log_buf = ptr_to_u64(bpf_log_buf),
.log_size = LOG_BUF_SIZE,
.log_level = 1,
};
return bpf(BPF_PROG_LOAD, &attr, sizeof(attr));
}
prog_type is one of the available program types:
enum bpf_prog_type {
BPF_PROG_TYPE_UNSPEC, /* Reserve 0 as invalid
program type */
BPF_PROG_TYPE_SOCKET_FILTER,
BPF_PROG_TYPE_KPROBE,
BPF_PROG_TYPE_SCHED_CLS,
BPF_PROG_TYPE_SCHED_ACT,
};
For further details of eBPF program types, see below.
The remaining fields of bpf_attr are set as follows:
* insns is an array of struct bpf_insn instructions.
* insn_cnt is the number of instructions in the program referred
to by insns.
* license is a license string, which must be GPL compatible to
call helper functions marked gpl_only.
* log_buf is a pointer to a caller-allocated buffer in which the
in-kernel verifier can store the verification log. This log
is a multi-line string that can be checked by the program
author in order to understand how the verifier came to the
conclusion that the BPF program is unsafe. The format of the
output can change at any time as the verifier evolves.
* log_size size of the buffer pointed to by log_bug. If the
size of the buffer is not large enough to store all verifier
messages, -1 is returned and errno is set to ENOSPC.
* log_level verbosity level of the verifier. A value of zero
means that the verifier will not provide a log.
Applying close(2) to the file descriptor returned by
BPF_PROG_LOAD will unload the eBPF program (but see NOTES).
Maps are accessible from eBPF programs and are used to exchange
data between eBPF programs and between eBPF programs and user-
space programs. For example, eBPF programs can process various
events (like kprobe, packets) and store their data into a map,
and user-space programs can then fetch data from the map. Con‐
versely, user-space programs can use a map as a configuration
mechanism, populating the map with values checked by the eBPF
program, which then modifies its behavior on the fly according to
those values.
eBPF program types
By picking prog_type, the program author selects a set of helper
functions that can be called from the eBPF program and the corre‐
sponding format of struct bpf_context (which is the data blob
passed into the eBPF program as the first argument). For exam‐
ple, programs loaded with a prog_type of
BPF_PROG_TYPE_SOCKET_FILTER may call the bpf_map_lookup_elem()
helper, whereas some other program types may not be able to
employ this helper. The set of functions available to eBPF pro‐
grams of a given type may increase in the future.
The following program types are supported:
BPF_PROG_TYPE_SOCKET_FILTER (since Linux 3.19)
Currently, the set of functions for
BPF_PROG_TYPE_SOCKET_FILTER is:
bpf_map_lookup_elem(map_fd, void *key)
/* look up key in a map_fd */
bpf_map_update_elem(map_fd, void *key, void *value)
/* update key/value */
bpf_map_delete_elem(map_fd, void *key)
/* delete key in a map_fd */
The bpf_context argument is a pointer to a struct sk_buff.
Programs cannot access the fields of sk_buff directly.
BPF_PROG_TYPE_KPROBE (since Linux 4.1)
[To be documented]
BPF_PROG_TYPE_SCHED_CLS (since Linux 4.1)
[To be documented]
BPF_PROG_TYPE_SCHED_ACT (since Linux 4.1)
[To be documented]
Events
Once a program is loaded, it can be attached to an event. Vari‐
ous kernel subsystems have different ways to do so.
Since Linux 3.19, the following call will attach the program
prog_fd to the socket sockfd, which was created by an earlier
call to socket(2):
setsockopt(sockfd, SOL_SOCKET, SO_ATTACH_BPF,
&prog_fd, sizeof(prog_fd));
Since Linux 4.1, the following call may be used to attach the
eBPF program referred to by the file descriptor prog_fd to a perf
event file descriptor, event_fd, that was created by a previous
call to perf_event_open(2):
ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd);
EXAMPLES
/* bpf+sockets example:
* 1. create array map of 256 elements
* 2. load program that counts number of packets received
* r0 = skb->data[ETH_HLEN + offsetof(struct iphdr, protocol)]
* map[r0]++
* 3. attach prog_fd to raw socket via setsockopt()
* 4. print number of received TCP/UDP packets every second
*/
int
main(int argc, char **argv)
{
int sock, map_fd, prog_fd, key;
long long value = 0, tcp_cnt, udp_cnt;
map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(key),
sizeof(value), 256);
if (map_fd < 0) {
printf("failed to create map '%s'\n", strerror(errno));
/* likely not run as root */
return 1;
}
struct bpf_insn prog[] = {
BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = r1 */
BPF_LD_ABS(BPF_B, ETH_HLEN + offsetof(struct iphdr, protocol)),
/* r0 = ip->proto */
BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_0, -4),
/* *(u32 *)(fp - 4) = r0 */
BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), /* r2 = fp */
BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), /* r2 = r2 - 4 */
BPF_LD_MAP_FD(BPF_REG_1, map_fd), /* r1 = map_fd */
BPF_CALL_FUNC(BPF_FUNC_map_lookup_elem),
/* r0 = map_lookup(r1, r2) */
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
/* if (r0 == 0) goto pc+2 */
BPF_MOV64_IMM(BPF_REG_1, 1), /* r1 = 1 */
BPF_XADD(BPF_DW, BPF_REG_0, BPF_REG_1, 0, 0),
/* lock *(u64 *) r0 += r1 */
BPF_MOV64_IMM(BPF_REG_0, 0), /* r0 = 0 */
BPF_EXIT_INSN(), /* return r0 */
};
prog_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, prog,
sizeof(prog), "GPL");
sock = open_raw_sock("lo");
assert(setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, &prog_fd,
sizeof(prog_fd)) == 0);
for (;;) {
key = IPPROTO_TCP;
assert(bpf_lookup_elem(map_fd, &key, &tcp_cnt) == 0);
key = IPPROTO_UDP
assert(bpf_lookup_elem(map_fd, &key, &udp_cnt) == 0);
printf("TCP %lld UDP %lld packets0, tcp_cnt, udp_cnt);
sleep(1);
}
return 0;
}
Some complete working code can be found in the samples/bpf direc‐
tory in the kernel source tree.
RETURN VALUE
For a successful call, the return value depends on the operation:
BPF_MAP_CREATE
The new file descriptor associated with the eBPF map.
BPF_PROG_LOAD
The new file descriptor associated with the eBPF program.
All other commands
Zero.
On error, -1 is returned, and errno is set appropriately.
ERRORS
EPERM The call was made without sufficient privilege (without
the CAP_SYS_ADMIN capability).
ENOMEM Cannot allocate sufficient memory.
EBADF fd is not an open file descriptor
EFAULT One of the pointers (key or value or log_buf or insns) is
outside the accessible address space.
EINVAL The value specified in cmd is not recognized by this ker‐
nel.
EINVAL For BPF_MAP_CREATE, either map_type or attributes are
invalid.
EINVAL For BPF_MAP_*_ELEM commands, some of the fields of union
bpf_attr that are not used by this command are not set to
zero.
EINVAL For BPF_PROG_LOAD, indicates an attempt to load an invalid
program. BPF programs can be deemed einvalid due to
unrecognized instructions, the use of reserved fields,
jumps out of range, infinite loops or calls of unknown
functions.
EACCES For BPF_PROG_LOAD, even though all program instructions
are valid, the program has been rejected because it was
deemed unsafe. This may be because it may have accessed a
disallowed memory region or an uninitialized stack/regis‐
ter or because the function constraints don't match the
actual types or because there was a misaligned memory
access. In this case, it is recommended to call bpf()
again with log_level = 1 and examine log_buf for the spe‐
cific reason provided by the verifier.
ENOENT For BPF_MAP_LOOKUP_ELEM or BPF_MAP_DELETE_ELEM, indicates
that the element with the given key was not found.
E2BIG The BPF program is too large or a map reached the
max_entries limit (maximum number of elements).
VERSIONS
The bpf() system call first appeared in Linux 3.18.
CONFORMING TO
The bpf() system call is Linux-specific.
NOTES
In the current implementation, all bpf() commands require the
caller to have the CAP_SYS_ADMIN capability.
eBPF objects (maps and programs) can be shared between processes.
For example, after fork(2), the child inherits file descriptors
referring to the same eBPF objects. In addition, file descrip‐
tors referring to eBPF objects can be transferred over UNIX
domain sockets. File descriptors referring to eBPF objects can
be duplicated in the usual way, using dup(2) and similar calls.
An eBPF object is deallocated only after all file descriptors
referring to the object have been closed.
eBPF programs can be written in a restricted C that is compiled
(using the clang compiler) into eBPF bytecode and executed on the
in-kernel virtual machine or just-in-time compiled into native
code. (Various features are omitted from this restricted C, such
as loops, global variables, variadic functions, floating-point
numbers, and passing structures as function arguments.) Some
examples can be found in the samples/bpf/*_kern.c files in the
kernel source tree.
SEE ALSO
seccomp(2), socket(7), tc(8), tc-bpf(8)
Both classic and extended BPF are explained in the kernel source
file Documentation/networking/filter.txt.
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-22 20:10 ` Michael Kerrisk (man-pages)
@ 2015-07-22 22:12 ` Alexei Starovoitov
2015-07-23 9:58 ` Michael Kerrisk (man-pages)
2015-07-23 9:31 ` Daniel Borkmann
1 sibling, 1 reply; 10+ messages in thread
From: Alexei Starovoitov @ 2015-07-22 22:12 UTC (permalink / raw)
To: Michael Kerrisk (man-pages), Daniel Borkmann
Cc: linux-man, linux-kernel, Silvan Jegen, Walter Harms
On 7/22/15 1:10 PM, Michael Kerrisk (man-pages) wrote:
> BPF maps are a generic data structure for storage of different
> data types. A user process can create multiple maps (with
> key/value-pairs being opaque bytes of data) and access them via
> file descriptors. Differnt eBPF programs can access the same
> maps in parallel. It's up to the user process and eBPF program
> to decide what they store inside maps.
typo in 'Different'
> eBPF program types
> By picking prog_type, the program author selects a set of helper
> functions that can be called from the eBPF program and the corre‐
> sponding format of struct bpf_context (which is the data blob
> passed into the eBPF program as the first argument). For exam‐
> ple, programs loaded with a prog_type of
> BPF_PROG_TYPE_SOCKET_FILTER may call the bpf_map_lookup_elem()
> helper, whereas some other program types may not be able to
> employ this helper. The set of functions available to eBPF pro‐
> grams of a given type may increase in the future.
overall it's all correct, but today bpf_map_lookup_elem() is allowed
for all program types. May be change that to:
"BPF_PROG_TYPE_KPROBE may call the bpf_probe_read() helper"
since only type_kprobe programs can call it.
type_sockets/type_sched_* cannot.
> The bpf_context argument is a pointer to a struct sk_buff.
> Programs cannot access the fields of sk_buff directly.
Probably drop last sentence and replace 'sk_buff' with '__sk_buff'
in the first? Or for the first version we can drop both sentences.
The rest looks great. Thank you much!
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-22 20:10 ` Michael Kerrisk (man-pages)
2015-07-22 22:12 ` Alexei Starovoitov
@ 2015-07-23 9:31 ` Daniel Borkmann
2015-07-23 11:23 ` Michael Kerrisk (man-pages)
1 sibling, 1 reply; 10+ messages in thread
From: Daniel Borkmann @ 2015-07-23 9:31 UTC (permalink / raw)
To: Michael Kerrisk (man-pages), Alexei Starovoitov
Cc: linux-man, linux-kernel, Silvan Jegen, Walter Harms
Hi Michael,
looks good already, a couple of comments inline, on top of Alexei's feedback:
On 07/22/2015 10:10 PM, Michael Kerrisk (man-pages) wrote:
...
> NAME
> bpf - perform a command on an extended eBPF map or program
'extended eBPF' should perhaps just say 'eBPF' or 'extended BPF' (the
'e' itself stands for 'extended')
> SYNOPSIS
> #include <linux/bpf.h>
>
> int bpf(int cmd, union bpf_attr *attr, unsigned int size);
>
> DESCRIPTION
> The bpf() system call performs a range of operations related to
> extended Berkeley Packet Filters. Extended BPF (or eBPF) is sim‐
> ilar to the original ("classic") BPF (cBPF) used to filter net‐
> work packets. For both cBPF and eBPF programs, the kernel stati‐
> cally analyzes the programs before loading them, in order to
> ensure that they cannot harm the running system.
>
> eBPF extends cBPF in multiple ways, including the ability to call
> a fixed set of in-kernel helper functions (via the BPF_CALL
> opcode extension provided by eBPF) and access shared data struc‐
> tures such as eBPF maps.
>
> Extended BPF Design/Architecture
> BPF maps are a generic data structure for storage of different
Maybe s/BPF/eBPF/ as we introduced its definition above and used 'eBPF maps'
just in the previous sentence. (I would from the onwards just use either eBPF
or cBPF, makes it probably more clear).
> data types. A user process can create multiple maps (with
> key/value-pairs being opaque bytes of data) and access them via
> file descriptors. Differnt eBPF programs can access the same
> maps in parallel. It's up to the user process and eBPF program
> to decide what they store inside maps.
>
> eBPF programs are similar to kernel modules. They are loaded by
> the user process and automatically unloaded when the process
> exits. Each program is a set of instructions that is safe to run
The 1st and 2nd sentence in that order/combination may sounds a bit weird.
Maybe I would just drop the first sentence? I would argue that there might
be a few similarities, but more differences overall. So I guess we'd either
need to elaborate on the 1st sentence or just leave it out (could perhaps
be a FIXME comment to later on introduce a new section that elaborates on
both?).
> until its completion. An in-kernel verifier statically deter‐
> mines that the eBPF program terminates and is safe to execute.
> During verification, the kernel increments reference counts for
> each of the maps that the eBPF program uses, so that the selected
> maps cannot be removed until the program is unloaded.
s/selected/attached/ ? Btw, a user obviously can close() the map fds if he
wants to, but ultimatively they're freed when the program unloads.
> eBPF programs can be attached to different events. These events
> can be the arrival of network packets, tracing events, classifi‐
> cation event by qdisc (for eBPF programs attached to a tc(8)
> classifier), and other types that may be added in the future. A
Maybe: classification events by network queuing disciplines
> new event triggers execution of the eBPF program, which may store
> information about the event in eBPF maps. Beyond storing data,
> eBPF programs may call a fixed set of in-kernel helper functions.
I think this was mentioned before, but ok.
> The same eBPF program can be attached to multiple events and dif‐
> ferent eBPF programs can access the same map:
>
> tracing tracing tracing packet packet
> event A event B event C on eth0 on eth1
> | | | | |
> | | | | |
> --> tracing <-- tracing socket socket
> prog_1 prog_2 prog_3 prog_4
> | | | |
> |--- -----| |-------| map_3
> map_1 map_2
Maybe prog_4 example could also be: s/socket/tc ingress classifier/ ;)
> Arguments
> The operation to be performed by the bpf() system call is deter‐
> mined by the cmd argument. Each operation takes an accompanying
> argument, provided via attr, which is a pointer to a union of
> type bpf_attr (see below). The size argument is the size of the
> union pointed to by attr.
>
> The value provided in cmd is one of the following:
>
> BPF_MAP_CREATE
> Create a map with and return a file descriptor that refers
> to the map.
'Create a map with and'
> BPF_MAP_LOOKUP_ELEM
> Look up an element by key in a specified map and return
> its value.
>
> BPF_MAP_UPDATE_ELEM
> Create or update an element (key/value pair) in a speci‐
> fied map.
>
> BPF_MAP_DELETE_ELEM
> Look up and delete an element by key in a specified map.
>
> BPF_MAP_GET_NEXT_KEY
> Look up an element by key in a specified map and return
> the key of the next element.
>
> BPF_PROG_LOAD
> Verify and load an eBPF program, returning a new file
> descriptor associated with the program.
>
> The bpf_attr union consists of various anonymous structures that
> are used by different bpf() commands:
>
> union bpf_attr {
> struct { /* Used by BPF_MAP_CREATE */
> __u32 map_type;
> __u32 key_size; /* size of key in bytes */
> __u32 value_size; /* size of value in bytes */
> __u32 max_entries; /* maximum number of entries
> in a map */
> };
>
> struct { /* Used by BPF_MAP_*_ELEM and BPF_MAP_GET_NEXT_KEY
> commands */
> __u32 map_fd;
> __aligned_u64 key;
> union {
> __aligned_u64 value;
> __aligned_u64 next_key;
> };
> __u64 flags;
> };
>
> struct { /* Used by BPF_PROG_LOAD */
> __u32 prog_type;
> __u32 insn_cnt;
> __aligned_u64 insns; /* 'const struct bpf_insn *' */
> __aligned_u64 license; /* 'const char *' */
> __u32 log_level; /* verbosity level of verifier */
> __u32 log_size; /* size of user buffer */
> __aligned_u64 log_buf; /* user supplied 'char *'
> buffer */
> __u32 kern_version;
> /* checked when prog_type=kprobe
> (since Linux 4.1) */
> };
> } __attribute__((aligned(8)));
>
> eBPF maps
> Maps are a generic data structure for storage of different types
> of data. They allow sharing of data between eBPF kernel pro‐
> grams, and also between kernel and user-space applications.
>
> Each map type has the following attributes:
>
> * type
> * maximum number of elements
> * key size in bytes
> * value size in bytes
>
> The following wrapper functions demonstrate how various bpf()
> commands can be used to access the maps. The functions use the
> cmd argument to invoke different operations.
>
> BPF_MAP_CREATE
> The BPF_MAP_CREATE command creates a new map, returning a
> new file descriptor that refers to the map.
>
> int
> bpf_create_map(enum bpf_map_type map_type, int key_size,
> int value_size, int max_entries)
key_size, value_size and max_entries could rather be 'unsigned int' in
this API example.
> {
> union bpf_attr attr = {
> .map_type = map_type,
> .key_size = key_size,
> .value_size = value_size,
> .max_entries = max_entries
> };
>
> return bpf(BPF_MAP_CREATE, &attr, sizeof(attr));
> }
>
> The new map has the type specified by map_type, and
> attributes as specified in key_size, value_size, and
> max_entries. On success, this operation returns a file
> descriptor. On error, -1 is returned and errno is set to
> EINVAL, EPERM, or ENOMEM.
>
> The attributes key_size and value_size will be used by the
attribute's?
> verifier during program loading to check that the program
> is calling bpf_map_*_elem() helper functions with a cor‐
> rectly initialized key and to check that the program
> doesn't access the map element value beyond the specified
> value_size. For example, when a map is created with a
> key_size of 8 and the eBPF program calls
>
> bpf_map_lookup_elem(map_fd, fp - 4)
>
> the program will be rejected, since the in-kernel helper
> function
>
> bpf_map_lookup_elem(map_fd, void *key)
>
> expects to read 8 bytes from the location pointed to by
> key, but the fp - 4 (where fp is the top of the stack)
> starting address will cause out-of-bounds stack access.
>
> Similarly, when a map is created with a value_size of 1
> and the eBPF program contains
>
> value = bpf_map_lookup_elem(...);
> *(u32 *) value = 1;
>
> the program will be rejected, since it accesses the value
> pointer beyond the specified 1 byte value_size limit.
>
> Currently, the following values are supported for
> map_type:
>
> enum bpf_map_type {
> BPF_MAP_TYPE_UNSPEC, /* Reserve 0 as invalid map type */
> BPF_MAP_TYPE_HASH,
> BPF_MAP_TYPE_ARRAY,
> BPF_MAP_TYPE_PROG_ARRAY,
> };
>
> map_type selects one of the available map implementations
> in the kernel. For all map types, eBPF programs access
> maps with the same bpf_map_lookup_elem() and
> bpf_map_update_elem() helper functions. Further details
> of the various map types are given below.
>
> BPF_MAP_LOOKUP_ELEM
> The BPF_MAP_LOOKUP_ELEM command looks up an element with a
> given key in the map referred to by the file descriptor
> fd.
>
> int
> bpf_lookup_elem(int fd, void *key, void *value)
It's just an API example implementation, and we cast the const away
in ptr_to_u64() [which is not provided here, that's ok], but it documents
the API itself better for those who implement it. I did the same in
iproute2's tc/tc_bpf.c:
const void *key
> {
> union bpf_attr attr = {
> .map_fd = fd,
> .key = ptr_to_u64(key),
> .value = ptr_to_u64(value),
> };
>
> return bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
> }
>
> If an element is found, the operation returns zero and
> stores the element's value into value, which must point to
> a buffer of value_size bytes.
>
> If no element is found, the operation returns -1 and sets
> errno to ENOENT.
>
> BPF_MAP_UPDATE_ELEM
> The BPF_MAP_UPDATE_ELEM command creates or updates an ele‐
> ment with a given key/value in the map referred to by the
> file descriptor fd.
>
> int
> bpf_update_elem(int fd, void *key, void *value, __u64 flags)
> {
const void *key, const void *value, uint64_t flags
The type __u64 is kernel internal, so if there's no strict reason to use it,
we should just use what's provided by stdint.h.
> union bpf_attr attr = {
> .map_fd = fd,
> .key = ptr_to_u64(key),
> .value = ptr_to_u64(value),
> .flags = flags,
> };
>
> return bpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr));
> }
>
> The flags argument should be specified as one of the fol‐
> lowing:
>
> BPF_ANY
> Create a new element or update an existing element.
>
> BPF_NOEXIST
> Create a new element only if it did not exist.
>
> BPF_EXIST
> Update an existing element.
>
> On success, the operation returns zero. On error, -1 is
> returned and errno is set to EINVAL, EPERM, ENOMEM, or
> E2BIG. E2BIG indicates that the number of elements in the
> map reached the max_entries limit specified at map cre‐
> ation time. EEXIST will be returned if flags specifies
> BPF_NOEXIST and the element with key already exists in the
> map. ENOENT will be returned if flags specifies BPF_EXIST
> and the element with key doesn't exist in the map.
>
> BPF_MAP_DELETE_ELEM
> The BPF_MAP_DELETE_ELEM command deleted the element whose
> key is key from the map referred to by the file descriptor
> fd.
>
> int
> bpf_delete_elem(int fd, void *key)
const void *key
> {
> union bpf_attr attr = {
> .map_fd = fd,
> .key = ptr_to_u64(key),
> };
>
> return bpf(BPF_MAP_DELETE_ELEM, &attr, sizeof(attr));
> }
>
> On success, zero is returned. If the element is not
> found, -1 is returned and errno is set to ENOENT.
>
> BPF_MAP_GET_NEXT_KEY
> The BPF_MAP_GET_NEXT_KEY command looks up an element by
> key in the map referred to by the file descriptor fd and
> sets the next_key pointer to the key of the next element.
>
> int
> bpf_get_next_key(int fd, void *key, void *next_key)
> {
const void *key
> union bpf_attr attr = {
> .map_fd = fd,
> .key = ptr_to_u64(key),
> .next_key = ptr_to_u64(next_key),
> };
>
> return bpf(BPF_MAP_GET_NEXT_KEY, &attr, sizeof(attr));
> }
>
> If key is found, the operation returns zero and sets the
> next_key pointer to the key of the next element. If key
> is not found, the operation returns zero and sets the
> next_key pointer to the key of the first element. If key
> is the last element, -1 is returned and errno is set to
> ENOENT. Other possible errno values are ENOMEM, EFAULT,
> EPERM, and EINVAL. This method can be used to iterate
> over all elements in the map.
>
> close(map_fd)
> Delete the map referred to by the file descriptor map_fd.
> When the user-space program that created a map exits, all
> maps will be deleted automatically (but see NOTES).
>
> eBPF map types
> The following map types are supported:
>
> BPF_MAP_TYPE_HASH
> Hash-table maps have the following characteristics:
>
> * Maps are created and destroyed by user-space programs.
> Both user-space and eBPF programs can perform lookup,
> update, and delete operations.
>
> * The kernel takes care of allocating and freeing
> key/value pairs.
>
> * The map_update_elem() helper with fail to insert new
> element when the max_entries limit is reached. (This
> ensures that eBPF programs cannot exhaust memory.)
>
> * map_update_elem() replaces existing elements atomi‐
> cally.
>
> Hash-table maps are optimized for speed of lookup.
>
> BPF_MAP_TYPE_ARRAY
> Array maps have the following characteristics:
>
> * Optimized for fastest possible lookup. In the future
> the verifier/JIT compiler may recognize lookup() opera‐
> tions that employ a constant key and optimize it into
> constant pointer. It is possible to optimize a non-
> constant key into direct pointer arithmetic as well,
> since pointers and value_size are constant for the life
> of the eBPF program. In other words,
> array_map_lookup_elem() may be 'inlined' by the veri‐
> fier/JIT compiler while preserving concurrent access to
> this map from user space.
>
> * All array elements pre-allocated and zero initialized
> at init time
>
> * The key is an array index, and must be exactly four
> bytes.
>
> * map_delete_elem() fails with the error EINVAL, since
> elements cannot be deleted.
>
> * map_update_elem() replaces elements in an non-atomic
> fashion; for atomic updates, a hash-table map should be
> used instead.
This point here is most important, i.e. to not have false user expecations.
Maybe it's also worth mentioning that when you have a value_size of sizeof(long),
you can however use __sync_fetch_and_add() atomic builtin from the LLVM backend.
> Among the uses for array maps are the following:
>
> * As "global" eBPF variables: an array of 1 element whose
> key is (index) 0 and where the value is a collection of
> 'global' variables which eBPF programs can use to keep
> state between events.
>
> * Aggregation of tracing events into a fixed set of buck‐
> ets.
>
> BPF_MAP_TYPE_PROG_ARRAY (since Linux 4.2)
> [To be completed]
>
> eBPF programs
> The BPF_PROG_LOAD command is used to load an eBPF program into
> the kernel. The return value for this command is a new file
> descriptor associated with this eBPF program.
>
> char bpf_log_buf[LOG_BUF_SIZE];
>
> int
> bpf_prog_load(enum bpf_prog_type prog_type,
> const struct bpf_insn *insns, int insn_cnt,
> const char *license)
Maybe:
int bpf_prog_load(enum bpf_prog_type type, const struct bpf_insn *insns,
unsigned int num_insns, const char *license)
[ The double prog_type is redundant. ]
> {
> union bpf_attr attr = {
> .prog_type = prog_type,
> .insns = ptr_to_u64(insns),
> .insn_cnt = insn_cnt,
> .license = ptr_to_u64(license),
> .log_buf = ptr_to_u64(bpf_log_buf),
> .log_size = LOG_BUF_SIZE,
> .log_level = 1,
> };
Would be nice to have this indented properly, I mean that all should
be aligned with tab before '='. That would make it much easier to read.
Also for all other code examples in this man-page (I forgot to mention
it for the above).
>
> return bpf(BPF_PROG_LOAD, &attr, sizeof(attr));
> }
>
> prog_type is one of the available program types:
>
> enum bpf_prog_type {
> BPF_PROG_TYPE_UNSPEC, /* Reserve 0 as invalid
> program type */
A pity that these *_UNSPEC types (also for the map) had to make it
into the uapi. :(
> BPF_PROG_TYPE_SOCKET_FILTER,
> BPF_PROG_TYPE_KPROBE,
> BPF_PROG_TYPE_SCHED_CLS,
> BPF_PROG_TYPE_SCHED_ACT,
> };
>
> For further details of eBPF program types, see below.
>
> The remaining fields of bpf_attr are set as follows:
>
> * insns is an array of struct bpf_insn instructions.
>
> * insn_cnt is the number of instructions in the program referred
> to by insns.
>
> * license is a license string, which must be GPL compatible to
> call helper functions marked gpl_only.
Not strictly. So here, the same rules apply as with kernel modules. I.e. what
the kernel checks for are the following license strings:
static inline int license_is_gpl_compatible(const char *license)
{
return (strcmp(license, "GPL") == 0
|| strcmp(license, "GPL v2") == 0
|| strcmp(license, "GPL and additional rights") == 0
|| strcmp(license, "Dual BSD/GPL") == 0
|| strcmp(license, "Dual MIT/GPL") == 0
|| strcmp(license, "Dual MPL/GPL") == 0);
}
With any of them, the eBPF program is declared GPL compatible. Maybe of interest
for those that want to use dual licensing of some sort.
> * log_buf is a pointer to a caller-allocated buffer in which the
> in-kernel verifier can store the verification log. This log
> is a multi-line string that can be checked by the program
> author in order to understand how the verifier came to the
> conclusion that the BPF program is unsafe. The format of the
> output can change at any time as the verifier evolves.
>
> * log_size size of the buffer pointed to by log_bug. If the
> size of the buffer is not large enough to store all verifier
> messages, -1 is returned and errno is set to ENOSPC.
>
> * log_level verbosity level of the verifier. A value of zero
> means that the verifier will not provide a log.
Note that the log buffer is optional as mentioned here log_level = 0. The
above example code of bpf_prog_load() suggests that it always needs to be
provided.
I once ran indeed into an issue where the program itself was correct, but
it got rejected by the kernel, because my log buffer size was too small, so
in tc, we now have it larger as bpf_log_buf[65536] ...
> Applying close(2) to the file descriptor returned by
> BPF_PROG_LOAD will unload the eBPF program (but see NOTES).
>
> Maps are accessible from eBPF programs and are used to exchange
> data between eBPF programs and between eBPF programs and user-
> space programs. For example, eBPF programs can process various
> events (like kprobe, packets) and store their data into a map,
> and user-space programs can then fetch data from the map. Con‐
> versely, user-space programs can use a map as a configuration
> mechanism, populating the map with values checked by the eBPF
> program, which then modifies its behavior on the fly according to
> those values.
>
> eBPF program types
> By picking prog_type, the program author selects a set of helper
> functions that can be called from the eBPF program and the corre‐
> sponding format of struct bpf_context (which is the data blob
> passed into the eBPF program as the first argument). For exam‐
I had to read this twice. ;) Maybe this needs to be reworded slightly.
It just means that depending on the program type that the author selects,
you might end up with a different subset of helper functions, and a
different program input/context. For example tracing does not have the
exact same helpers as socket filters (it might have some that can be used
by both). Also, the eBPF program input (context) for socket filters is a
network packet, wheras for tracing you operate on a set of registers.
> ple, programs loaded with a prog_type of
> BPF_PROG_TYPE_SOCKET_FILTER may call the bpf_map_lookup_elem()
> helper, whereas some other program types may not be able to
> employ this helper. The set of functions available to eBPF pro‐
> grams of a given type may increase in the future.
>
> The following program types are supported:
>
> BPF_PROG_TYPE_SOCKET_FILTER (since Linux 3.19)
> Currently, the set of functions for
> BPF_PROG_TYPE_SOCKET_FILTER is:
>
> bpf_map_lookup_elem(map_fd, void *key)
> /* look up key in a map_fd */
> bpf_map_update_elem(map_fd, void *key, void *value)
> /* update key/value */
> bpf_map_delete_elem(map_fd, void *key)
> /* delete key in a map_fd */
>
> The bpf_context argument is a pointer to a struct sk_buff.
> Programs cannot access the fields of sk_buff directly.
>
> BPF_PROG_TYPE_KPROBE (since Linux 4.1)
> [To be documented]
>
> BPF_PROG_TYPE_SCHED_CLS (since Linux 4.1)
> [To be documented]
>
> BPF_PROG_TYPE_SCHED_ACT (since Linux 4.1)
> [To be documented]
>
> Events
> Once a program is loaded, it can be attached to an event. Vari‐
> ous kernel subsystems have different ways to do so.
>
> Since Linux 3.19, the following call will attach the program
> prog_fd to the socket sockfd, which was created by an earlier
> call to socket(2):
>
> setsockopt(sockfd, SOL_SOCKET, SO_ATTACH_BPF,
> &prog_fd, sizeof(prog_fd));
>
> Since Linux 4.1, the following call may be used to attach the
> eBPF program referred to by the file descriptor prog_fd to a perf
> event file descriptor, event_fd, that was created by a previous
> call to perf_event_open(2):
>
> ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd);
>
> EXAMPLES
> /* bpf+sockets example:
> * 1. create array map of 256 elements
> * 2. load program that counts number of packets received
> * r0 = skb->data[ETH_HLEN + offsetof(struct iphdr, protocol)]
> * map[r0]++
> * 3. attach prog_fd to raw socket via setsockopt()
> * 4. print number of received TCP/UDP packets every second
> */
> int
> main(int argc, char **argv)
> {
> int sock, map_fd, prog_fd, key;
> long long value = 0, tcp_cnt, udp_cnt;
>
> map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(key),
> sizeof(value), 256);
> if (map_fd < 0) {
> printf("failed to create map '%s'\n", strerror(errno));
> /* likely not run as root */
> return 1;
> }
>
> struct bpf_insn prog[] = {
> BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = r1 */
> BPF_LD_ABS(BPF_B, ETH_HLEN + offsetof(struct iphdr, protocol)),
> /* r0 = ip->proto */
> BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_0, -4),
> /* *(u32 *)(fp - 4) = r0 */
> BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), /* r2 = fp */
> BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), /* r2 = r2 - 4 */
> BPF_LD_MAP_FD(BPF_REG_1, map_fd), /* r1 = map_fd */
> BPF_CALL_FUNC(BPF_FUNC_map_lookup_elem),
> /* r0 = map_lookup(r1, r2) */
> BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
> /* if (r0 == 0) goto pc+2 */
> BPF_MOV64_IMM(BPF_REG_1, 1), /* r1 = 1 */
> BPF_XADD(BPF_DW, BPF_REG_0, BPF_REG_1, 0, 0),
> /* lock *(u64 *) r0 += r1 */
> BPF_MOV64_IMM(BPF_REG_0, 0), /* r0 = 0 */
> BPF_EXIT_INSN(), /* return r0 */
> };
>
> prog_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, prog,
> sizeof(prog), "GPL");
>
> sock = open_raw_sock("lo");
>
> assert(setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, &prog_fd,
> sizeof(prog_fd)) == 0);
>
> for (;;) {
> key = IPPROTO_TCP;
> assert(bpf_lookup_elem(map_fd, &key, &tcp_cnt) == 0);
> key = IPPROTO_UDP
> assert(bpf_lookup_elem(map_fd, &key, &udp_cnt) == 0);
> printf("TCP %lld UDP %lld packets0, tcp_cnt, udp_cnt);
> sleep(1);
> }
>
> return 0;
> }
>
> Some complete working code can be found in the samples/bpf direc‐
> tory in the kernel source tree.
>
> RETURN VALUE
> For a successful call, the return value depends on the operation:
>
> BPF_MAP_CREATE
> The new file descriptor associated with the eBPF map.
>
> BPF_PROG_LOAD
> The new file descriptor associated with the eBPF program.
>
> All other commands
> Zero.
>
> On error, -1 is returned, and errno is set appropriately.
>
> ERRORS
> EPERM The call was made without sufficient privilege (without
> the CAP_SYS_ADMIN capability).
>
> ENOMEM Cannot allocate sufficient memory.
>
> EBADF fd is not an open file descriptor
>
> EFAULT One of the pointers (key or value or log_buf or insns) is
> outside the accessible address space.
>
> EINVAL The value specified in cmd is not recognized by this ker‐
> nel.
>
> EINVAL For BPF_MAP_CREATE, either map_type or attributes are
> invalid.
>
> EINVAL For BPF_MAP_*_ELEM commands, some of the fields of union
> bpf_attr that are not used by this command are not set to
> zero.
>
> EINVAL For BPF_PROG_LOAD, indicates an attempt to load an invalid
> program. BPF programs can be deemed einvalid due to
> unrecognized instructions, the use of reserved fields,
> jumps out of range, infinite loops or calls of unknown
> functions.
>
> EACCES For BPF_PROG_LOAD, even though all program instructions
> are valid, the program has been rejected because it was
> deemed unsafe. This may be because it may have accessed a
> disallowed memory region or an uninitialized stack/regis‐
> ter or because the function constraints don't match the
> actual types or because there was a misaligned memory
> access. In this case, it is recommended to call bpf()
> again with log_level = 1 and examine log_buf for the spe‐
> cific reason provided by the verifier.
>
> ENOENT For BPF_MAP_LOOKUP_ELEM or BPF_MAP_DELETE_ELEM, indicates
> that the element with the given key was not found.
>
> E2BIG The BPF program is too large or a map reached the
> max_entries limit (maximum number of elements).
>
> VERSIONS
> The bpf() system call first appeared in Linux 3.18.
>
> CONFORMING TO
> The bpf() system call is Linux-specific.
>
> NOTES
> In the current implementation, all bpf() commands require the
> caller to have the CAP_SYS_ADMIN capability.
>
> eBPF objects (maps and programs) can be shared between processes.
> For example, after fork(2), the child inherits file descriptors
> referring to the same eBPF objects. In addition, file descrip‐
> tors referring to eBPF objects can be transferred over UNIX
> domain sockets. File descriptors referring to eBPF objects can
> be duplicated in the usual way, using dup(2) and similar calls.
> An eBPF object is deallocated only after all file descriptors
> referring to the object have been closed.
>
> eBPF programs can be written in a restricted C that is compiled
> (using the clang compiler) into eBPF bytecode and executed on the
> in-kernel virtual machine or just-in-time compiled into native
> code. (Various features are omitted from this restricted C, such
> as loops, global variables, variadic functions, floating-point
> numbers, and passing structures as function arguments.) Some
> examples can be found in the samples/bpf/*_kern.c files in the
> kernel source tree.
I would also make a note about the JIT compiler here, i.e. that it's disabled
by default, and can be enabled via:
* Normal mode: echo 1 > /proc/sys/net/core/bpf_jit_enable
* Debugging mode: echo 2 > /proc/sys/net/core/bpf_jit_enable
[opcodes dumped in hex into the kernel log, which can then be disassembled
with tools/net/bpf_jit_disasm.c from the kernel tree]
When enabled, after a eBPF program gets loaded, it's transparently compiled /
translated inside the kernel into machine opcodes for better performance,
currently on x86_64, arm64 and s390.
> SEE ALSO
> seccomp(2), socket(7), tc(8), tc-bpf(8)
>
> Both classic and extended BPF are explained in the kernel source
> file Documentation/networking/filter.txt.
>
Thanks for all the work!
Cheers,
Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-22 22:12 ` Alexei Starovoitov
@ 2015-07-23 9:58 ` Michael Kerrisk (man-pages)
0 siblings, 0 replies; 10+ messages in thread
From: Michael Kerrisk (man-pages) @ 2015-07-23 9:58 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann
Cc: mtk.manpages, linux-man, linux-kernel, Silvan Jegen, Walter Harms
On 07/23/2015 12:12 AM, Alexei Starovoitov wrote:
> On 7/22/15 1:10 PM, Michael Kerrisk (man-pages) wrote:
>> BPF maps are a generic data structure for storage of different
>> data types. A user process can create multiple maps (with
>> key/value-pairs being opaque bytes of data) and access them via
>> file descriptors. Differnt eBPF programs can access the same
>> maps in parallel. It's up to the user process and eBPF program
>> to decide what they store inside maps.
>
> typo in 'Different'
Fixed.
>> eBPF program types
>> By picking prog_type, the program author selects a set of helper
>> functions that can be called from the eBPF program and the corre‐
>> sponding format of struct bpf_context (which is the data blob
>> passed into the eBPF program as the first argument). For exam‐
>> ple, programs loaded with a prog_type of
>> BPF_PROG_TYPE_SOCKET_FILTER may call the bpf_map_lookup_elem()
>> helper, whereas some other program types may not be able to
>> employ this helper. The set of functions available to eBPF pro‐
>> grams of a given type may increase in the future.
>
> overall it's all correct, but today bpf_map_lookup_elem() is allowed
> for all program types. May be change that to:
> "BPF_PROG_TYPE_KPROBE may call the bpf_probe_read() helper"
> since only type_kprobe programs can call it.
> type_sockets/type_sched_* cannot.
Changed, as you suggest.
>> The bpf_context argument is a pointer to a struct sk_buff.
>> Programs cannot access the fields of sk_buff directly.
>
> Probably drop last sentence and replace 'sk_buff' with '__sk_buff'
> in the first? Or for the first version we can drop both sentences.
I did the former.
> The rest looks great. Thank you much!
Thanks,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-23 9:31 ` Daniel Borkmann
@ 2015-07-23 11:23 ` Michael Kerrisk (man-pages)
2015-07-23 12:47 ` Daniel Borkmann
0 siblings, 1 reply; 10+ messages in thread
From: Michael Kerrisk (man-pages) @ 2015-07-23 11:23 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: mtk.manpages, linux-man, linux-kernel, Silvan Jegen, Walter Harms
Hi Daniel,
On 07/23/2015 11:31 AM, Daniel Borkmann wrote:
> Hi Michael,
>
> looks good already, a couple of comments inline, on top of Alexei's feedback:
>
> On 07/22/2015 10:10 PM, Michael Kerrisk (man-pages) wrote:
> ...
>> NAME
>> bpf - perform a command on an extended eBPF map or program
>
> 'extended eBPF' should perhaps just say 'eBPF' or 'extended BPF' (the
> 'e' itself stands for 'extended')
D'oh! Fixed.
>> SYNOPSIS
>> #include <linux/bpf.h>
>>
>> int bpf(int cmd, union bpf_attr *attr, unsigned int size);
>>
>> DESCRIPTION
>> The bpf() system call performs a range of operations related to
>> extended Berkeley Packet Filters. Extended BPF (or eBPF) is sim‐
>> ilar to the original ("classic") BPF (cBPF) used to filter net‐
>> work packets. For both cBPF and eBPF programs, the kernel stati‐
>> cally analyzes the programs before loading them, in order to
>> ensure that they cannot harm the running system.
>>
>> eBPF extends cBPF in multiple ways, including the ability to call
>> a fixed set of in-kernel helper functions (via the BPF_CALL
>> opcode extension provided by eBPF) and access shared data struc‐
>> tures such as eBPF maps.
>>
>> Extended BPF Design/Architecture
>> BPF maps are a generic data structure for storage of different
>
> Maybe s/BPF/eBPF/ as we introduced its definition above and used 'eBPF maps'
> just in the previous sentence.
Done.
> (I would from the onwards just use either eBPF
> or cBPF, makes it probably more clear).
Agreed. (I fixed a few other cases cases.)
>> data types. A user process can create multiple maps (with
>> key/value-pairs being opaque bytes of data) and access them via
>> file descriptors. Differnt eBPF programs can access the same
>> maps in parallel. It's up to the user process and eBPF program
>> to decide what they store inside maps.
>>
>> eBPF programs are similar to kernel modules. They are loaded by
>> the user process and automatically unloaded when the process
>> exits. Each program is a set of instructions that is safe to run
>
> The 1st and 2nd sentence in that order/combination may sounds a bit weird.
> Maybe I would just drop the first sentence? I would argue that there might
> be a few similarities, but more differences overall. So I guess we'd either
> need to elaborate on the 1st sentence or just leave it out (could perhaps
> be a FIXME comment to later on introduce a new section that elaborates on
> both?).
I was also not quite happy with that first sentence. I've dropped it.
>> until its completion. An in-kernel verifier statically deter‐
>> mines that the eBPF program terminates and is safe to execute.
>> During verification, the kernel increments reference counts for
>> each of the maps that the eBPF program uses, so that the selected
>> maps cannot be removed until the program is unloaded.
>
> s/selected/attached/ ?
Done.
> Btw, a user obviously can close() the map fds if he
> wants to, but ultimatively they're freed when the program unloads.
Okay. (Not sure if you meant that something should be added to the page.)
>> eBPF programs can be attached to different events. These events
>> can be the arrival of network packets, tracing events, classifi‐
>> cation event by qdisc (for eBPF programs attached to a tc(8)
>> classifier), and other types that may be added in the future. A
>
> Maybe: classification events by network queuing disciplines
Yes, better. Done.
>> new event triggers execution of the eBPF program, which may store
>> information about the event in eBPF maps. Beyond storing data,
>> eBPF programs may call a fixed set of in-kernel helper functions.
>
> I think this was mentioned before, but ok.
>
>> The same eBPF program can be attached to multiple events and dif‐
>> ferent eBPF programs can access the same map:
>>
>> tracing tracing tracing packet packet
>> event A event B event C on eth0 on eth1
>> | | | | |
>> | | | | |
>> --> tracing <-- tracing socket socket
>> prog_1 prog_2 prog_3 prog_4
>> | | | |
>> |--- -----| |-------| map_3
>> map_1 map_2
>
> Maybe prog_4 example could also be: s/socket/tc ingress classifier/ ;)
Done.
>> Arguments
>> The operation to be performed by the bpf() system call is deter‐
>> mined by the cmd argument. Each operation takes an accompanying
>> argument, provided via attr, which is a pointer to a union of
>> type bpf_attr (see below). The size argument is the size of the
>> union pointed to by attr.
>>
>> The value provided in cmd is one of the following:
>>
>> BPF_MAP_CREATE
>> Create a map with and return a file descriptor that refers
>> to the map.
>
> 'Create a map with and'
Fixed.
>> BPF_MAP_LOOKUP_ELEM
>> Look up an element by key in a specified map and return
>> its value.
>>
>> BPF_MAP_UPDATE_ELEM
>> Create or update an element (key/value pair) in a speci‐
>> fied map.
>>
>> BPF_MAP_DELETE_ELEM
>> Look up and delete an element by key in a specified map.
>>
>> BPF_MAP_GET_NEXT_KEY
>> Look up an element by key in a specified map and return
>> the key of the next element.
>>
>> BPF_PROG_LOAD
>> Verify and load an eBPF program, returning a new file
>> descriptor associated with the program.
>>
>> The bpf_attr union consists of various anonymous structures that
>> are used by different bpf() commands:
>>
>> union bpf_attr {
>> struct { /* Used by BPF_MAP_CREATE */
>> __u32 map_type;
>> __u32 key_size; /* size of key in bytes */
>> __u32 value_size; /* size of value in bytes */
>> __u32 max_entries; /* maximum number of entries
>> in a map */
>> };
>>
>> struct { /* Used by BPF_MAP_*_ELEM and BPF_MAP_GET_NEXT_KEY
>> commands */
>> __u32 map_fd;
>> __aligned_u64 key;
>> union {
>> __aligned_u64 value;
>> __aligned_u64 next_key;
>> };
>> __u64 flags;
>> };
>>
>> struct { /* Used by BPF_PROG_LOAD */
>> __u32 prog_type;
>> __u32 insn_cnt;
>> __aligned_u64 insns; /* 'const struct bpf_insn *' */
>> __aligned_u64 license; /* 'const char *' */
>> __u32 log_level; /* verbosity level of verifier */
>> __u32 log_size; /* size of user buffer */
>> __aligned_u64 log_buf; /* user supplied 'char *'
>> buffer */
>> __u32 kern_version;
>> /* checked when prog_type=kprobe
>> (since Linux 4.1) */
>> };
>> } __attribute__((aligned(8)));
>>
>> eBPF maps
>> Maps are a generic data structure for storage of different types
>> of data. They allow sharing of data between eBPF kernel pro‐
>> grams, and also between kernel and user-space applications.
>>
>> Each map type has the following attributes:
>>
>> * type
>> * maximum number of elements
>> * key size in bytes
>> * value size in bytes
>>
>> The following wrapper functions demonstrate how various bpf()
>> commands can be used to access the maps. The functions use the
>> cmd argument to invoke different operations.
>>
>> BPF_MAP_CREATE
>> The BPF_MAP_CREATE command creates a new map, returning a
>> new file descriptor that refers to the map.
>>
>> int
>> bpf_create_map(enum bpf_map_type map_type, int key_size,
>> int value_size, int max_entries)
>
> key_size, value_size and max_entries could rather be 'unsigned int' in
> this API example.
Done. (This also should be fixed in the kernel source file
samples/bpf/libbpf.c. Same remark probably applies to some of your
other suggestions below.)
>> {
>> union bpf_attr attr = {
>> .map_type = map_type,
>> .key_size = key_size,
>> .value_size = value_size,
>> .max_entries = max_entries
>> };
>>
>> return bpf(BPF_MAP_CREATE, &attr, sizeof(attr));
>> }
>>
>> The new map has the type specified by map_type, and
>> attributes as specified in key_size, value_size, and
>> max_entries. On success, this operation returns a file
>> descriptor. On error, -1 is returned and errno is set to
>> EINVAL, EPERM, or ENOMEM.
>>
>> The attributes key_size and value_size will be used by the
>
> attribute's?
Nope. But I changed this to "The key_size and value_size attributes will be",
which may read clearer.
>> verifier during program loading to check that the program
>> is calling bpf_map_*_elem() helper functions with a cor‐
>> rectly initialized key and to check that the program
>> doesn't access the map element value beyond the specified
>> value_size. For example, when a map is created with a
>> key_size of 8 and the eBPF program calls
>>
>> bpf_map_lookup_elem(map_fd, fp - 4)
>>
>> the program will be rejected, since the in-kernel helper
>> function
>>
>> bpf_map_lookup_elem(map_fd, void *key)
>>
>> expects to read 8 bytes from the location pointed to by
>> key, but the fp - 4 (where fp is the top of the stack)
>> starting address will cause out-of-bounds stack access.
>>
>> Similarly, when a map is created with a value_size of 1
>> and the eBPF program contains
>>
>> value = bpf_map_lookup_elem(...);
>> *(u32 *) value = 1;
>>
>> the program will be rejected, since it accesses the value
>> pointer beyond the specified 1 byte value_size limit.
>>
>> Currently, the following values are supported for
>> map_type:
>>
>> enum bpf_map_type {
>> BPF_MAP_TYPE_UNSPEC, /* Reserve 0 as invalid map type */
>> BPF_MAP_TYPE_HASH,
>> BPF_MAP_TYPE_ARRAY,
>> BPF_MAP_TYPE_PROG_ARRAY,
>> };
>>
>> map_type selects one of the available map implementations
>> in the kernel. For all map types, eBPF programs access
>> maps with the same bpf_map_lookup_elem() and
>> bpf_map_update_elem() helper functions. Further details
>> of the various map types are given below.
>>
>> BPF_MAP_LOOKUP_ELEM
>> The BPF_MAP_LOOKUP_ELEM command looks up an element with a
>> given key in the map referred to by the file descriptor
>> fd.
>>
>> int
>> bpf_lookup_elem(int fd, void *key, void *value)
>
> It's just an API example implementation, and we cast the const away
> in ptr_to_u64() [which is not provided here, that's ok], but it documents
> the API itself better for those who implement it. I did the same in
> iproute2's tc/tc_bpf.c:
>
> const void *key
Done.
>> union bpf_attr attr = {
>> .map_fd = fd,
>> .key = ptr_to_u64(key),
>> .value = ptr_to_u64(value),
>> };
>>
>> return bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
>> }
>>
>> If an element is found, the operation returns zero and
>> stores the element's value into value, which must point to
>> a buffer of value_size bytes.
>>
>> If no element is found, the operation returns -1 and sets
>> errno to ENOENT.
>>
>> BPF_MAP_UPDATE_ELEM
>> The BPF_MAP_UPDATE_ELEM command creates or updates an ele‐
>> ment with a given key/value in the map referred to by the
>> file descriptor fd.
>>
>> int
>> bpf_update_elem(int fd, void *key, void *value, __u64 flags)
>> {
>
> const void *key, const void *value, uint64_t flags
Done.
> The type __u64 is kernel internal, so if there's no strict reason to use it,
> we should just use what's provided by stdint.h.
Agreed. Done. (By the way, what about all the __u32 and __u64 elements in the
bpf_attr union?)
>> union bpf_attr attr = {
>> .map_fd = fd,
>> .key = ptr_to_u64(key),
>> .value = ptr_to_u64(value),
>> .flags = flags,
>> };
>>
>> return bpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr));
>> }
>>
>> The flags argument should be specified as one of the fol‐
>> lowing:
>>
>> BPF_ANY
>> Create a new element or update an existing element.
>>
>> BPF_NOEXIST
>> Create a new element only if it did not exist.
>>
>> BPF_EXIST
>> Update an existing element.
>>
>> On success, the operation returns zero. On error, -1 is
>> returned and errno is set to EINVAL, EPERM, ENOMEM, or
>> E2BIG. E2BIG indicates that the number of elements in the
>> map reached the max_entries limit specified at map cre‐
>> ation time. EEXIST will be returned if flags specifies
>> BPF_NOEXIST and the element with key already exists in the
>> map. ENOENT will be returned if flags specifies BPF_EXIST
>> and the element with key doesn't exist in the map.
>>
>> BPF_MAP_DELETE_ELEM
>> The BPF_MAP_DELETE_ELEM command deleted the element whose
>> key is key from the map referred to by the file descriptor
>> fd.
>>
>> int
>> bpf_delete_elem(int fd, void *key)
>
> const void *key
Done.
>> {
>> union bpf_attr attr = {
>> .map_fd = fd,
>> .key = ptr_to_u64(key),
>> };
>>
>> return bpf(BPF_MAP_DELETE_ELEM, &attr, sizeof(attr));
>> }
>>
>> On success, zero is returned. If the element is not
>> found, -1 is returned and errno is set to ENOENT.
>>
>> BPF_MAP_GET_NEXT_KEY
>> The BPF_MAP_GET_NEXT_KEY command looks up an element by
>> key in the map referred to by the file descriptor fd and
>> sets the next_key pointer to the key of the next element.
>>
>> int
>> bpf_get_next_key(int fd, void *key, void *next_key)
>> {
>
> const void *key
Done.
>> union bpf_attr attr = {
>> .map_fd = fd,
>> .key = ptr_to_u64(key),
>> .next_key = ptr_to_u64(next_key),
>> };
>>
>> return bpf(BPF_MAP_GET_NEXT_KEY, &attr, sizeof(attr));
>> }
>>
>> If key is found, the operation returns zero and sets the
>> next_key pointer to the key of the next element. If key
>> is not found, the operation returns zero and sets the
>> next_key pointer to the key of the first element. If key
>> is the last element, -1 is returned and errno is set to
>> ENOENT. Other possible errno values are ENOMEM, EFAULT,
>> EPERM, and EINVAL. This method can be used to iterate
>> over all elements in the map.
>>
>> close(map_fd)
>> Delete the map referred to by the file descriptor map_fd.
>> When the user-space program that created a map exits, all
>> maps will be deleted automatically (but see NOTES).
>>
>> eBPF map types
>> The following map types are supported:
>>
>> BPF_MAP_TYPE_HASH
>> Hash-table maps have the following characteristics:
>>
>> * Maps are created and destroyed by user-space programs.
>> Both user-space and eBPF programs can perform lookup,
>> update, and delete operations.
>>
>> * The kernel takes care of allocating and freeing
>> key/value pairs.
>>
>> * The map_update_elem() helper with fail to insert new
>> element when the max_entries limit is reached. (This
>> ensures that eBPF programs cannot exhaust memory.)
>>
>> * map_update_elem() replaces existing elements atomi‐
>> cally.
>>
>> Hash-table maps are optimized for speed of lookup.
>>
>> BPF_MAP_TYPE_ARRAY
>> Array maps have the following characteristics:
>>
>> * Optimized for fastest possible lookup. In the future
>> the verifier/JIT compiler may recognize lookup() opera‐
>> tions that employ a constant key and optimize it into
>> constant pointer. It is possible to optimize a non-
>> constant key into direct pointer arithmetic as well,
>> since pointers and value_size are constant for the life
>> of the eBPF program. In other words,
>> array_map_lookup_elem() may be 'inlined' by the veri‐
>> fier/JIT compiler while preserving concurrent access to
>> this map from user space.
>>
>> * All array elements pre-allocated and zero initialized
>> at init time
>>
>> * The key is an array index, and must be exactly four
>> bytes.
>>
>> * map_delete_elem() fails with the error EINVAL, since
>> elements cannot be deleted.
>>
>> * map_update_elem() replaces elements in an non-atomic
>> fashion; for atomic updates, a hash-table map should be
>> used instead.
>
> This point here is most important, i.e. to not have false user expecations.
> Maybe it's also worth mentioning that when you have a value_size of sizeof(long),
> you can however use __sync_fetch_and_add() atomic builtin from the LLVM backend.
I think I'll leave out that detail for the moment.
>> Among the uses for array maps are the following:
>>
>> * As "global" eBPF variables: an array of 1 element whose
>> key is (index) 0 and where the value is a collection of
>> 'global' variables which eBPF programs can use to keep
>> state between events.
>>
>> * Aggregation of tracing events into a fixed set of buck‐
>> ets.
>>
>> BPF_MAP_TYPE_PROG_ARRAY (since Linux 4.2)
>> [To be completed]
>>
>> eBPF programs
>> The BPF_PROG_LOAD command is used to load an eBPF program into
>> the kernel. The return value for this command is a new file
>> descriptor associated with this eBPF program.
>>
>> char bpf_log_buf[LOG_BUF_SIZE];
>>
>> int
>> bpf_prog_load(enum bpf_prog_type prog_type,
>> const struct bpf_insn *insns, int insn_cnt,
>> const char *license)
>
> Maybe:
>
> int bpf_prog_load(enum bpf_prog_type type, const struct bpf_insn *insns,
> unsigned int num_insns, const char *license)
>
> [ The double prog_type is redundant. ]
Done.
>> {
>> union bpf_attr attr = {
>> .prog_type = prog_type,
>> .insns = ptr_to_u64(insns),
>> .insn_cnt = insn_cnt,
>> .license = ptr_to_u64(license),
>> .log_buf = ptr_to_u64(bpf_log_buf),
>> .log_size = LOG_BUF_SIZE,
>> .log_level = 1,
>> };
>
> Would be nice to have this indented properly, I mean that all should
> be aligned with tab before '='. That would make it much easier to read.
> Also for all other code examples in this man-page (I forgot to mention
> it for the above).
Done (for all examples).
>>
>> return bpf(BPF_PROG_LOAD, &attr, sizeof(attr));
>> }
>>
>> prog_type is one of the available program types:
>>
>> enum bpf_prog_type {
>> BPF_PROG_TYPE_UNSPEC, /* Reserve 0 as invalid
>> program type */
>
> A pity that these *_UNSPEC types (also for the map) had to make it
> into the uapi. :(
Yes, it seemed odd to me.
>> BPF_PROG_TYPE_SOCKET_FILTER,
>> BPF_PROG_TYPE_KPROBE,
>> BPF_PROG_TYPE_SCHED_CLS,
>> BPF_PROG_TYPE_SCHED_ACT,
>> };
>>
>> For further details of eBPF program types, see below.
>>
>> The remaining fields of bpf_attr are set as follows:
>>
>> * insns is an array of struct bpf_insn instructions.
>>
>> * insn_cnt is the number of instructions in the program referred
>> to by insns.
>>
>> * license is a license string, which must be GPL compatible to
>> call helper functions marked gpl_only.
>
> Not strictly. So here, the same rules apply as with kernel modules. I.e. what
> the kernel checks for are the following license strings:
>
> static inline int license_is_gpl_compatible(const char *license)
> {
> return (strcmp(license, "GPL") == 0
> || strcmp(license, "GPL v2") == 0
> || strcmp(license, "GPL and additional rights") == 0
> || strcmp(license, "Dual BSD/GPL") == 0
> || strcmp(license, "Dual MIT/GPL") == 0
> || strcmp(license, "Dual MPL/GPL") == 0);
> }
>
> With any of them, the eBPF program is declared GPL compatible. Maybe of interest
> for those that want to use dual licensing of some sort.
So, I'm a little unclear here. What text do you suggest for the page?
>> * log_buf is a pointer to a caller-allocated buffer in which the
>> in-kernel verifier can store the verification log. This log
>> is a multi-line string that can be checked by the program
>> author in order to understand how the verifier came to the
>> conclusion that the BPF program is unsafe. The format of the
>> output can change at any time as the verifier evolves.
>>
>> * log_size size of the buffer pointed to by log_bug. If the
>> size of the buffer is not large enough to store all verifier
>> messages, -1 is returned and errno is set to ENOSPC.
>>
>> * log_level verbosity level of the verifier. A value of zero
>> means that the verifier will not provide a log.
>
> Note that the log buffer is optional as mentioned here log_level = 0. The
> above example code of bpf_prog_load() suggests that it always needs to be
> provided.
>
> I once ran indeed into an issue where the program itself was correct, but
> it got rejected by the kernel, because my log buffer size was too small, so
> in tc, we now have it larger as bpf_log_buf[65536] ...
So, I'm not clear. Do you mean that some piece of text here in the page
should be changed? If so, could elaborate?
>> Applying close(2) to the file descriptor returned by
>> BPF_PROG_LOAD will unload the eBPF program (but see NOTES).
>>
>> Maps are accessible from eBPF programs and are used to exchange
>> data between eBPF programs and between eBPF programs and user-
>> space programs. For example, eBPF programs can process various
>> events (like kprobe, packets) and store their data into a map,
>> and user-space programs can then fetch data from the map. Con‐
>> versely, user-space programs can use a map as a configuration
>> mechanism, populating the map with values checked by the eBPF
>> program, which then modifies its behavior on the fly according to
>> those values.
>>
>> eBPF program types
>> By picking prog_type, the program author selects a set of helper
>> functions that can be called from the eBPF program and the corre‐
>> sponding format of struct bpf_context (which is the data blob
>> passed into the eBPF program as the first argument). For exam‐
>
> I had to read this twice. ;) Maybe this needs to be reworded slightly.
>
> It just means that depending on the program type that the author selects,
> you might end up with a different subset of helper functions, and a
> different program input/context. For example tracing does not have the
> exact same helpers as socket filters (it might have some that can be used
> by both). Also, the eBPF program input (context) for socket filters is a
> network packet, wheras for tracing you operate on a set of registers.
Changed. Now we have:
eBPF program types
The eBPF program type (prog_type) determines the subset of a ker‐
nel helper functions that the program may call. The program type
also determines dthe program input (context)—the format of struct
bpf_context (which is the data blob passed into the eBPF program
as the first argument).
For example, a tracing program does not have the exact same sub‐
set of helper functions as a socket filter program (though they
may have some helpers in common). Similarly, the input (context)
for a tracing program is a set of register values, while for a
socket filter it is a network packet.
The set of functions available to eBPF programs of a given type
may increase in the future.
>> ple, programs loaded with a prog_type of
>> BPF_PROG_TYPE_SOCKET_FILTER may call the bpf_map_lookup_elem()
>> helper, whereas some other program types may not be able to
>> employ this helper. The set of functions available to eBPF pro‐
>> grams of a given type may increase in the future.
>>
>> The following program types are supported:
>>
>> BPF_PROG_TYPE_SOCKET_FILTER (since Linux 3.19)
>> Currently, the set of functions for
>> BPF_PROG_TYPE_SOCKET_FILTER is:
>>
>> bpf_map_lookup_elem(map_fd, void *key)
>> /* look up key in a map_fd */
>> bpf_map_update_elem(map_fd, void *key, void *value)
>> /* update key/value */
>> bpf_map_delete_elem(map_fd, void *key)
>> /* delete key in a map_fd */
>>
>> The bpf_context argument is a pointer to a struct sk_buff.
>> Programs cannot access the fields of sk_buff directly.
>>
>> BPF_PROG_TYPE_KPROBE (since Linux 4.1)
>> [To be documented]
>>
>> BPF_PROG_TYPE_SCHED_CLS (since Linux 4.1)
>> [To be documented]
>>
>> BPF_PROG_TYPE_SCHED_ACT (since Linux 4.1)
>> [To be documented]
>>
>> Events
>> Once a program is loaded, it can be attached to an event. Vari‐
>> ous kernel subsystems have different ways to do so.
>>
>> Since Linux 3.19, the following call will attach the program
>> prog_fd to the socket sockfd, which was created by an earlier
>> call to socket(2):
>>
>> setsockopt(sockfd, SOL_SOCKET, SO_ATTACH_BPF,
>> &prog_fd, sizeof(prog_fd));
>>
>> Since Linux 4.1, the following call may be used to attach the
>> eBPF program referred to by the file descriptor prog_fd to a perf
>> event file descriptor, event_fd, that was created by a previous
>> call to perf_event_open(2):
>>
>> ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd);
>>
>> EXAMPLES
>> /* bpf+sockets example:
>> * 1. create array map of 256 elements
>> * 2. load program that counts number of packets received
>> * r0 = skb->data[ETH_HLEN + offsetof(struct iphdr, protocol)]
>> * map[r0]++
>> * 3. attach prog_fd to raw socket via setsockopt()
>> * 4. print number of received TCP/UDP packets every second
>> */
>> int
>> main(int argc, char **argv)
>> {
>> int sock, map_fd, prog_fd, key;
>> long long value = 0, tcp_cnt, udp_cnt;
>>
>> map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(key),
>> sizeof(value), 256);
>> if (map_fd < 0) {
>> printf("failed to create map '%s'\n", strerror(errno));
>> /* likely not run as root */
>> return 1;
>> }
>>
>> struct bpf_insn prog[] = {
>> BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = r1 */
>> BPF_LD_ABS(BPF_B, ETH_HLEN + offsetof(struct iphdr, protocol)),
>> /* r0 = ip->proto */
>> BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_0, -4),
>> /* *(u32 *)(fp - 4) = r0 */
>> BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), /* r2 = fp */
>> BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), /* r2 = r2 - 4 */
>> BPF_LD_MAP_FD(BPF_REG_1, map_fd), /* r1 = map_fd */
>> BPF_CALL_FUNC(BPF_FUNC_map_lookup_elem),
>> /* r0 = map_lookup(r1, r2) */
>> BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
>> /* if (r0 == 0) goto pc+2 */
>> BPF_MOV64_IMM(BPF_REG_1, 1), /* r1 = 1 */
>> BPF_XADD(BPF_DW, BPF_REG_0, BPF_REG_1, 0, 0),
>> /* lock *(u64 *) r0 += r1 */
>> BPF_MOV64_IMM(BPF_REG_0, 0), /* r0 = 0 */
>> BPF_EXIT_INSN(), /* return r0 */
>> };
>>
>> prog_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, prog,
>> sizeof(prog), "GPL");
>>
>> sock = open_raw_sock("lo");
>>
>> assert(setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, &prog_fd,
>> sizeof(prog_fd)) == 0);
>>
>> for (;;) {
>> key = IPPROTO_TCP;
>> assert(bpf_lookup_elem(map_fd, &key, &tcp_cnt) == 0);
>> key = IPPROTO_UDP
>> assert(bpf_lookup_elem(map_fd, &key, &udp_cnt) == 0);
>> printf("TCP %lld UDP %lld packets0, tcp_cnt, udp_cnt);
>> sleep(1);
>> }
>>
>> return 0;
>> }
>>
>> Some complete working code can be found in the samples/bpf direc‐
>> tory in the kernel source tree.
>>
>> RETURN VALUE
>> For a successful call, the return value depends on the operation:
>>
>> BPF_MAP_CREATE
>> The new file descriptor associated with the eBPF map.
>>
>> BPF_PROG_LOAD
>> The new file descriptor associated with the eBPF program.
>>
>> All other commands
>> Zero.
>>
>> On error, -1 is returned, and errno is set appropriately.
>>
>> ERRORS
>> EPERM The call was made without sufficient privilege (without
>> the CAP_SYS_ADMIN capability).
>>
>> ENOMEM Cannot allocate sufficient memory.
>>
>> EBADF fd is not an open file descriptor
>>
>> EFAULT One of the pointers (key or value or log_buf or insns) is
>> outside the accessible address space.
>>
>> EINVAL The value specified in cmd is not recognized by this ker‐
>> nel.
>>
>> EINVAL For BPF_MAP_CREATE, either map_type or attributes are
>> invalid.
>>
>> EINVAL For BPF_MAP_*_ELEM commands, some of the fields of union
>> bpf_attr that are not used by this command are not set to
>> zero.
>>
>> EINVAL For BPF_PROG_LOAD, indicates an attempt to load an invalid
>> program. BPF programs can be deemed einvalid due to
>> unrecognized instructions, the use of reserved fields,
>> jumps out of range, infinite loops or calls of unknown
>> functions.
>>
>> EACCES For BPF_PROG_LOAD, even though all program instructions
>> are valid, the program has been rejected because it was
>> deemed unsafe. This may be because it may have accessed a
>> disallowed memory region or an uninitialized stack/regis‐
>> ter or because the function constraints don't match the
>> actual types or because there was a misaligned memory
>> access. In this case, it is recommended to call bpf()
>> again with log_level = 1 and examine log_buf for the spe‐
>> cific reason provided by the verifier.
>>
>> ENOENT For BPF_MAP_LOOKUP_ELEM or BPF_MAP_DELETE_ELEM, indicates
>> that the element with the given key was not found.
>>
>> E2BIG The BPF program is too large or a map reached the
>> max_entries limit (maximum number of elements).
>>
>> VERSIONS
>> The bpf() system call first appeared in Linux 3.18.
>>
>> CONFORMING TO
>> The bpf() system call is Linux-specific.
>>
>> NOTES
>> In the current implementation, all bpf() commands require the
>> caller to have the CAP_SYS_ADMIN capability.
>>
>> eBPF objects (maps and programs) can be shared between processes.
>> For example, after fork(2), the child inherits file descriptors
>> referring to the same eBPF objects. In addition, file descrip‐
>> tors referring to eBPF objects can be transferred over UNIX
>> domain sockets. File descriptors referring to eBPF objects can
>> be duplicated in the usual way, using dup(2) and similar calls.
>> An eBPF object is deallocated only after all file descriptors
>> referring to the object have been closed.
>>
>> eBPF programs can be written in a restricted C that is compiled
>> (using the clang compiler) into eBPF bytecode and executed on the
>> in-kernel virtual machine or just-in-time compiled into native
>> code. (Various features are omitted from this restricted C, such
>> as loops, global variables, variadic functions, floating-point
>> numbers, and passing structures as function arguments.) Some
>> examples can be found in the samples/bpf/*_kern.c files in the
>> kernel source tree.
>
> I would also make a note about the JIT compiler here, i.e. that it's disabled
> by default, and can be enabled via:
>
> * Normal mode: echo 1 > /proc/sys/net/core/bpf_jit_enable
>
> * Debugging mode: echo 2 > /proc/sys/net/core/bpf_jit_enable
> [opcodes dumped in hex into the kernel log, which can then be disassembled
Here, I assume you mean thet the generated (native) opcodes are dumpeed, right?
> with tools/net/bpf_jit_disasm.c from the kernel tree]
>
> When enabled, after a eBPF program gets loaded, it's transparently compiled /
> translated inside the kernel into machine opcodes for better performance,
> currently on x86_64, arm64 and s390.
According to Documentation/networking/filter.txt the JIT compiler supports
many more architectures:
The Linux kernel has a built-in BPF JIT compiler for x86_64,
SPARC, PowerPC, ARM, ARM64, MIPS and s390 and can be enabled
through CONFIG_BPF_JIT.
Or am I misunderstanding something?
I added the following:
The kernel contains a just-in-time (JIT) compiler that translates
eBPF bytecode into native machine code for better performance.
The JIT compiler is disabled by default, but its operation can be
controlled by writing one of the following values to
/proc/sys/net/core/bpf_jit_enable:
0 Disable JIT compilation (default).
1 Normal compilation.
2 Debugging mode. The generated opcodes are dumped in hexadeci‐
mal into the kernel log. These opcodes can then be disassem‐
bled using the program tools/net/bpf_jit_disasm.c provided in
the kernel source tree.
>> SEE ALSO
>> seccomp(2), socket(7), tc(8), tc-bpf(8)
>>
>> Both classic and extended BPF are explained in the kernel source
>> file Documentation/networking/filter.txt.
>>
>
> Thanks for all the work!
You're welcome. Thanks for the help!
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-23 11:23 ` Michael Kerrisk (man-pages)
@ 2015-07-23 12:47 ` Daniel Borkmann
2015-07-23 13:36 ` Michael Kerrisk (man-pages)
0 siblings, 1 reply; 10+ messages in thread
From: Daniel Borkmann @ 2015-07-23 12:47 UTC (permalink / raw)
To: Michael Kerrisk (man-pages), Alexei Starovoitov
Cc: linux-man, linux-kernel, Silvan Jegen, Walter Harms
On 07/23/2015 01:23 PM, Michael Kerrisk (man-pages) wrote:
...
>> Btw, a user obviously can close() the map fds if he
>> wants to, but ultimatively they're freed when the program unloads.
>
> Okay. (Not sure if you meant that something should be added to the page.)
I think not necessary.
[...]
>>> The attributes key_size and value_size will be used by the
>>
>> attribute's?
>
> Nope. But I changed this to "The key_size and value_size attributes will be",
> which may read clearer.
Sorry, true, I was a bit confused. :)
[...]
>> The type __u64 is kernel internal, so if there's no strict reason to use it,
>> we should just use what's provided by stdint.h.
>
> Agreed. Done. (By the way, what about all the __u32 and __u64 elements in the
> bpf_attr union?)
I wouldn't change the bpf_attr from the uapi.
Just the provided example code here, I presume people might copy from here when
they build their own library and in userspace uint64_t seems to be more natural.
[...]
>>> * map_update_elem() replaces elements in an non-atomic
>>> fashion; for atomic updates, a hash-table map should be
>>> used instead.
>>
>> This point here is most important, i.e. to not have false user expecations.
>> Maybe it's also worth mentioning that when you have a value_size of sizeof(long),
>> you can however use __sync_fetch_and_add() atomic builtin from the LLVM backend.
>
> I think I'll leave out that detail for the moment.
Ok, I guess we could revisit/clarify that at a later point in time. I'd add
a TODO comment to the source or the like, as this also is related to the 2nd
below use case (aggregation/accounting), where an array is typically used.
>>> Among the uses for array maps are the following:
>>>
>>> * As "global" eBPF variables: an array of 1 element whose
>>> key is (index) 0 and where the value is a collection of
>>> 'global' variables which eBPF programs can use to keep
>>> state between events.
>>>
>>> * Aggregation of tracing events into a fixed set of buck‐
>>> ets.
[...]
>>> * license is a license string, which must be GPL compatible to
>>> call helper functions marked gpl_only.
>>
>> Not strictly. So here, the same rules apply as with kernel modules. I.e. what
>> the kernel checks for are the following license strings:
>>
>> static inline int license_is_gpl_compatible(const char *license)
>> {
>> return (strcmp(license, "GPL") == 0
>> || strcmp(license, "GPL v2") == 0
>> || strcmp(license, "GPL and additional rights") == 0
>> || strcmp(license, "Dual BSD/GPL") == 0
>> || strcmp(license, "Dual MIT/GPL") == 0
>> || strcmp(license, "Dual MPL/GPL") == 0);
>> }
>>
>> With any of them, the eBPF program is declared GPL compatible. Maybe of interest
>> for those that want to use dual licensing of some sort.
>
> So, I'm a little unclear here. What text do you suggest for the page?
Maybe we should mention in addition that the same licensing rules apply as
in case with kernel modules, so also dual licenses could be used.
>>> * log_buf is a pointer to a caller-allocated buffer in which the
>>> in-kernel verifier can store the verification log. This log
>>> is a multi-line string that can be checked by the program
>>> author in order to understand how the verifier came to the
>>> conclusion that the BPF program is unsafe. The format of the
>>> output can change at any time as the verifier evolves.
>>>
>>> * log_size size of the buffer pointed to by log_bug. If the
>>> size of the buffer is not large enough to store all verifier
>>> messages, -1 is returned and errno is set to ENOSPC.
>>>
>>> * log_level verbosity level of the verifier. A value of zero
>>> means that the verifier will not provide a log.
>>
>> Note that the log buffer is optional as mentioned here log_level = 0. The
>> above example code of bpf_prog_load() suggests that it always needs to be
>> provided.
>>
>> I once ran indeed into an issue where the program itself was correct, but
>> it got rejected by the kernel, because my log buffer size was too small, so
>> in tc, we now have it larger as bpf_log_buf[65536] ...
>
> So, I'm not clear. Do you mean that some piece of text here in the page
> should be changed? If so, could elaborate?
I'd maybe only mention in addition that in log_level=0 case, we also must not
provide a log_buf and log_size, otherwise we get EINVAL.
[...]
>> I had to read this twice. ;) Maybe this needs to be reworded slightly.
>>
>> It just means that depending on the program type that the author selects,
>> you might end up with a different subset of helper functions, and a
>> different program input/context. For example tracing does not have the
>> exact same helpers as socket filters (it might have some that can be used
>> by both). Also, the eBPF program input (context) for socket filters is a
>> network packet, wheras for tracing you operate on a set of registers.
>
> Changed. Now we have:
>
> eBPF program types
> The eBPF program type (prog_type) determines the subset of a ker‐
> nel helper functions that the program may call. The program type
s/a//
> also determines dthe program input (context)—the format of struct
s/dthe/the/
> bpf_context (which is the data blob passed into the eBPF program
> as the first argument).
>
> For example, a tracing program does not have the exact same sub‐
> set of helper functions as a socket filter program (though they
> may have some helpers in common). Similarly, the input (context)
> for a tracing program is a set of register values, while for a
> socket filter it is a network packet.
>
> The set of functions available to eBPF programs of a given type
> may increase in the future.
That's fine with me.
[...]
>> I would also make a note about the JIT compiler here, i.e. that it's disabled
>> by default, and can be enabled via:
>>
>> * Normal mode: echo 1 > /proc/sys/net/core/bpf_jit_enable
>>
>> * Debugging mode: echo 2 > /proc/sys/net/core/bpf_jit_enable
>> [opcodes dumped in hex into the kernel log, which can then be disassembled
>
> Here, I assume you mean thet the generated (native) opcodes are dumpeed, right?
Yes.
>> with tools/net/bpf_jit_disasm.c from the kernel tree]
>>
>> When enabled, after a eBPF program gets loaded, it's transparently compiled /
>> translated inside the kernel into machine opcodes for better performance,
>> currently on x86_64, arm64 and s390.
>
> According to Documentation/networking/filter.txt the JIT compiler supports
> many more architectures:
>
> The Linux kernel has a built-in BPF JIT compiler for x86_64,
> SPARC, PowerPC, ARM, ARM64, MIPS and s390 and can be enabled
> through CONFIG_BPF_JIT.
>
> Or am I misunderstanding something?
The others only work for cBPF and have not (yet) be converted over to eBPF.
For the three mentioned above, the kernel internally migrates cBPF into eBPF
instructions and then JITs the eBPF result eventually.
> I added the following:
>
> The kernel contains a just-in-time (JIT) compiler that translates
> eBPF bytecode into native machine code for better performance.
> The JIT compiler is disabled by default, but its operation can be
> controlled by writing one of the following values to
> /proc/sys/net/core/bpf_jit_enable:
>
> 0 Disable JIT compilation (default).
>
> 1 Normal compilation.
>
> 2 Debugging mode. The generated opcodes are dumped in hexadeci‐
> mal into the kernel log. These opcodes can then be disassem‐
> bled using the program tools/net/bpf_jit_disasm.c provided in
> the kernel source tree.
>
>>> SEE ALSO
>>> seccomp(2), socket(7), tc(8), tc-bpf(8)
>>>
>>> Both classic and extended BPF are explained in the kernel source
>>> file Documentation/networking/filter.txt.
>>>
>>
Rest looks good for an initial version!
Thanks,
Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-23 12:47 ` Daniel Borkmann
@ 2015-07-23 13:36 ` Michael Kerrisk (man-pages)
2015-07-23 13:39 ` Daniel Borkmann
0 siblings, 1 reply; 10+ messages in thread
From: Michael Kerrisk (man-pages) @ 2015-07-23 13:36 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: mtk.manpages, linux-man, linux-kernel, Silvan Jegen, Walter Harms
Hi Daniel,
On 07/23/2015 02:47 PM, Daniel Borkmann wrote:
> On 07/23/2015 01:23 PM, Michael Kerrisk (man-pages) wrote:
> ...
>>> Btw, a user obviously can close() the map fds if he
>>> wants to, but ultimatively they're freed when the program unloads.
>>
>> Okay. (Not sure if you meant that something should be added to the page.)
>
> I think not necessary.
Okay.
> [...]
>>>> The attributes key_size and value_size will be used by the
>>>
>>> attribute's?
>>
>> Nope. But I changed this to "The key_size and value_size attributes will be",
>> which may read clearer.
>
> Sorry, true, I was a bit confused. :)
NP.
> [...]
>>> The type __u64 is kernel internal, so if there's no strict reason to use it,
>>> we should just use what's provided by stdint.h.
>>
>> Agreed. Done. (By the way, what about all the __u32 and __u64 elements in the
>> bpf_attr union?)
>
> I wouldn't change the bpf_attr from the uapi.
Okay.
> Just the provided example code here, I presume people might copy from here when
> they build their own library and in userspace uint64_t seems to be more natural.
Yup.
> [...]
>>>> * map_update_elem() replaces elements in an non-atomic
>>>> fashion; for atomic updates, a hash-table map should be
>>>> used instead.
>>>
>>> This point here is most important, i.e. to not have false user expecations.
>>> Maybe it's also worth mentioning that when you have a value_size of sizeof(long),
>>> you can however use __sync_fetch_and_add() atomic builtin from the LLVM backend.
>>
>> I think I'll leave out that detail for the moment.
>
> Ok, I guess we could revisit/clarify that at a later point in time. I'd add
> a TODO comment to the source or the like, as this also is related to the 2nd
> below use case (aggregation/accounting), where an array is typically used.
Okay. FIXME added.
>>>> Among the uses for array maps are the following:
>>>>
>>>> * As "global" eBPF variables: an array of 1 element whose
>>>> key is (index) 0 and where the value is a collection of
>>>> 'global' variables which eBPF programs can use to keep
>>>> state between events.
>>>>
>>>> * Aggregation of tracing events into a fixed set of buck‐
>>>> ets.
>
> [...]
>>>> * license is a license string, which must be GPL compatible to
>>>> call helper functions marked gpl_only.
>>>
>>> Not strictly. So here, the same rules apply as with kernel modules. I.e. what
>>> the kernel checks for are the following license strings:
>>>
>>> static inline int license_is_gpl_compatible(const char *license)
>>> {
>>> return (strcmp(license, "GPL") == 0
>>> || strcmp(license, "GPL v2") == 0
>>> || strcmp(license, "GPL and additional rights") == 0
>>> || strcmp(license, "Dual BSD/GPL") == 0
>>> || strcmp(license, "Dual MIT/GPL") == 0
>>> || strcmp(license, "Dual MPL/GPL") == 0);
>>> }
>>>
>>> With any of them, the eBPF program is declared GPL compatible. Maybe of interest
>>> for those that want to use dual licensing of some sort.
>>
>> So, I'm a little unclear here. What text do you suggest for the page?
>
> Maybe we should mention in addition that the same licensing rules apply as
> in case with kernel modules, so also dual licenses could be used.
Done.
>>>> * log_buf is a pointer to a caller-allocated buffer in which the
>>>> in-kernel verifier can store the verification log. This log
>>>> is a multi-line string that can be checked by the program
>>>> author in order to understand how the verifier came to the
>>>> conclusion that the BPF program is unsafe. The format of the
>>>> output can change at any time as the verifier evolves.
>>>>
>>>> * log_size size of the buffer pointed to by log_bug. If the
>>>> size of the buffer is not large enough to store all verifier
>>>> messages, -1 is returned and errno is set to ENOSPC.
>>>>
>>>> * log_level verbosity level of the verifier. A value of zero
>>>> means that the verifier will not provide a log.
>>>
>>> Note that the log buffer is optional as mentioned here log_level = 0. The
>>> above example code of bpf_prog_load() suggests that it always needs to be
>>> provided.
>>>
>>> I once ran indeed into an issue where the program itself was correct, but
>>> it got rejected by the kernel, because my log buffer size was too small, so
>>> in tc, we now have it larger as bpf_log_buf[65536] ...
>>
>> So, I'm not clear. Do you mean that some piece of text here in the page
>> should be changed? If so, could elaborate?
>
> I'd maybe only mention in addition that in log_level=0 case, we also must not
> provide a log_buf and log_size, otherwise we get EINVAL.
I changed the text to:
* log_level verbosity level of the verifier. A value of zero
means that the verifier will not provide a log; in this case,
log_buf must be a NULL pointer, and log_size must be zero.
> [...]
>>> I had to read this twice. ;) Maybe this needs to be reworded slightly.
>>>
>>> It just means that depending on the program type that the author selects,
>>> you might end up with a different subset of helper functions, and a
>>> different program input/context. For example tracing does not have the
>>> exact same helpers as socket filters (it might have some that can be used
>>> by both). Also, the eBPF program input (context) for socket filters is a
>>> network packet, wheras for tracing you operate on a set of registers.
>>
>> Changed. Now we have:
>>
>> eBPF program types
>> The eBPF program type (prog_type) determines the subset of a ker‐
>> nel helper functions that the program may call. The program type
>
> s/a//
Fixed.
>> also determines dthe program input (context)—the format of struct
>
> s/dthe/the/
Fixed.
>> bpf_context (which is the data blob passed into the eBPF program
>> as the first argument).
>>
>> For example, a tracing program does not have the exact same sub‐
>> set of helper functions as a socket filter program (though they
>> may have some helpers in common). Similarly, the input (context)
>> for a tracing program is a set of register values, while for a
>> socket filter it is a network packet.
>>
>> The set of functions available to eBPF programs of a given type
>> may increase in the future.
>
> That's fine with me.
Okay.
> [...]
>>> I would also make a note about the JIT compiler here, i.e. that it's disabled
>>> by default, and can be enabled via:
>>>
>>> * Normal mode: echo 1 > /proc/sys/net/core/bpf_jit_enable
>>>
>>> * Debugging mode: echo 2 > /proc/sys/net/core/bpf_jit_enable
>>> [opcodes dumped in hex into the kernel log, which can then be disassembled
>>
>> Here, I assume you mean thet the generated (native) opcodes are dumpeed, right?
>
> Yes.
>
>>> with tools/net/bpf_jit_disasm.c from the kernel tree]
>>>
>>> When enabled, after a eBPF program gets loaded, it's transparently compiled /
>>> translated inside the kernel into machine opcodes for better performance,
>>> currently on x86_64, arm64 and s390.
>>
>> According to Documentation/networking/filter.txt the JIT compiler supports
>> many more architectures:
>>
>> The Linux kernel has a built-in BPF JIT compiler for x86_64,
>> SPARC, PowerPC, ARM, ARM64, MIPS and s390 and can be enabled
>> through CONFIG_BPF_JIT.
>>
>> Or am I misunderstanding something?
>
> The others only work for cBPF and have not (yet) be converted over to eBPF.
>
> For the three mentioned above, the kernel internally migrates cBPF into eBPF
> instructions and then JITs the eBPF result eventually.
Thanks for clearing that up -- I added the following sentence
JIT compiler for eBPF is currently available for the x86-64, arm64,
and s390 architectures.
Okay?
>
>> I added the following:
>>
>> The kernel contains a just-in-time (JIT) compiler that translates
>> eBPF bytecode into native machine code for better performance.
>> The JIT compiler is disabled by default, but its operation can be
>> controlled by writing one of the following values to
>> /proc/sys/net/core/bpf_jit_enable:
>>
>> 0 Disable JIT compilation (default).
>>
>> 1 Normal compilation.
>>
>> 2 Debugging mode. The generated opcodes are dumped in hexadeci‐
>> mal into the kernel log. These opcodes can then be disassem‐
>> bled using the program tools/net/bpf_jit_disasm.c provided in
>> the kernel source tree.
>>
>>>> SEE ALSO
>>>> seccomp(2), socket(7), tc(8), tc-bpf(8)
>>>>
>>>> Both classic and extended BPF are explained in the kernel source
>>>> file Documentation/networking/filter.txt.
>>>>
>>>
>
> Rest looks good for an initial version!
Yup!
Thanks,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Draft 3 of bpf(2) man page for review
2015-07-23 13:36 ` Michael Kerrisk (man-pages)
@ 2015-07-23 13:39 ` Daniel Borkmann
0 siblings, 0 replies; 10+ messages in thread
From: Daniel Borkmann @ 2015-07-23 13:39 UTC (permalink / raw)
To: Michael Kerrisk (man-pages), Alexei Starovoitov
Cc: linux-man, linux-kernel, Silvan Jegen, Walter Harms
On 07/23/2015 03:36 PM, Michael Kerrisk (man-pages) wrote:
...
>> Ok, I guess we could revisit/clarify that at a later point in time. I'd add
>> a TODO comment to the source or the like, as this also is related to the 2nd
>> below use case (aggregation/accounting), where an array is typically used.
>
> Okay. FIXME added.
Great.
[...]
>> I'd maybe only mention in addition that in log_level=0 case, we also must not
>> provide a log_buf and log_size, otherwise we get EINVAL.
>
> I changed the text to:
>
> * log_level verbosity level of the verifier. A value of zero
> means that the verifier will not provide a log; in this case,
> log_buf must be a NULL pointer, and log_size must be zero.
Ok.
[...]
> Thanks for clearing that up -- I added the following sentence
>
> JIT compiler for eBPF is currently available for the x86-64, arm64,
> and s390 architectures.
>
> Okay?
Yep.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2015-07-23 13:39 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-22 18:43 Draft 3 of bpf(2) man page for review Michael Kerrisk (man-pages)
2015-07-22 19:22 ` Alexei Starovoitov
2015-07-22 20:10 ` Michael Kerrisk (man-pages)
2015-07-22 22:12 ` Alexei Starovoitov
2015-07-23 9:58 ` Michael Kerrisk (man-pages)
2015-07-23 9:31 ` Daniel Borkmann
2015-07-23 11:23 ` Michael Kerrisk (man-pages)
2015-07-23 12:47 ` Daniel Borkmann
2015-07-23 13:36 ` Michael Kerrisk (man-pages)
2015-07-23 13:39 ` Daniel Borkmann
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).