bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH bpf-next v6 00/16] BPF support for global data
@ 2019-04-09 21:08 Daniel Borkmann
  2019-04-09 21:08 ` [PATCH bpf-next v6 01/16] bpf: implement lookup-free direct value access for maps Daniel Borkmann
                   ` (7 more replies)
  0 siblings, 8 replies; 10+ messages in thread
From: Daniel Borkmann @ 2019-04-09 21:08 UTC (permalink / raw)
  To: bpf; +Cc: netdev, ast, joe, yhs, andrii.nakryiko, kafai, jannh, Daniel Borkmann

This series is a major rework of previously submitted libbpf
patches [0] in order to add global data support for BPF. The
kernel has been extended to add proper infrastructure that allows
for full .bss/.data/.rodata sections on BPF loader side based
upon feedback from LPC discussions [1]. Latter support is then
also added into libbpf in this series which allows for more
natural C-like programming of BPF programs. For more information
on loader, please refer to 'bpf, libbpf: support global data/bss/
rodata sections' patch in this series.

Thanks a lot!

  v5 -> v6:
   - Removed synchronize_rcu() from map freeze (Jann)
   - Rest as-is
  v4 -> v5:
   - Removed index selection again for ldimm64 (Alexei)
   - Adapted related test cases and added new ones to test
     rejection of off != 0
  v3 -> v4:
   - Various fixes in BTF verification e.g. to disallow
     Var and DataSec to be an intermediate type during resolve (Martin)
   - More BTF test cases added
   - Few cleanups in key-less BTF commit (Martin)
   - Bump libbpf minor version from 2 to 3
   - Renamed and simplified read-only locking
   - Various minor improvements all over the place
  v2 -> v3:
   - Implement BTF support in kernel, libbpf, bpftool, add tests
   - Fix idx + off conversion (Andrii)
   - Document lower / higher bits for direct value access (Andrii)
   - Add tests with small value size (Andrii)
   - Add index selection into ldimm64 (Andrii)
   - Fix missing fdput() (Jann)
   - Reject invalid flags in BPF_F_*_PROG (Jakub)
   - Complete rework of libbpf support, includes:
    - Add objname to map name (Stanislav)
    - Make .rodata map full read-only after setup (Andrii)
    - Merge relocation handling into single one (Andrii)
    - Store global maps into obj->maps array (Andrii, Alexei)
    - Debug message when skipping section (Andrii)
    - Reject non-static global data till we have
      semantics for sharing them (Yonghong, Andrii, Alexei)
    - More test cases and completely reworked prog test (Alexei)
   - Fixes, cleanups, etc all over the set
   - Not yet addressed:
    - Make BTF mandatory for these maps (Alexei)
    -> Waiting till BTF support for these lands first
  v1 -> v2:
    - Instead of 32-bit static data, implement full global
      data support (Alexei)

  [0] https://patchwork.ozlabs.org/cover/1040290/
  [1] http://vger.kernel.org/lpc-bpf2018.html#session-3

Daniel Borkmann (14):
  bpf: implement lookup-free direct value access for maps
  bpf: do not retain flags that are not tied to map lifetime
  bpf: add program side {rd, wr}only support for maps
  bpf: add syscall side map freeze support
  bpf: allow . char as part of the object name
  bpf: add specification for BTF Var and DataSec kinds
  bpf: kernel side support for BTF Var and DataSec
  bpf: allow for key-less BTF in array map
  bpf: sync {btf, bpf}.h uapi header from tools infrastructure
  bpf, libbpf: support global data/bss/rodata sections
  bpf, libbpf: add support for BTF Var and DataSec
  bpf: bpftool support for dumping data/bss/rodata sections
  bpf, selftest: test {rd, wr}only flags and direct value access
  bpf, selftest: add test cases for BTF Var and DataSec

Joe Stringer (2):
  bpf, libbpf: refactor relocation handling
  bpf, selftest: test global data/bss/rodata sections

 Documentation/bpf/btf.rst                     |  57 ++
 include/linux/bpf.h                           |  38 +-
 include/linux/bpf_verifier.h                  |   4 +
 include/linux/btf.h                           |   1 +
 include/uapi/linux/bpf.h                      |  20 +-
 include/uapi/linux/btf.h                      |  32 +-
 kernel/bpf/arraymap.c                         |  53 +-
 kernel/bpf/btf.c                              | 419 ++++++++++-
 kernel/bpf/core.c                             |   3 +-
 kernel/bpf/disasm.c                           |   5 +-
 kernel/bpf/hashtab.c                          |   6 +-
 kernel/bpf/local_storage.c                    |   6 +-
 kernel/bpf/lpm_trie.c                         |   3 +-
 kernel/bpf/queue_stack_maps.c                 |   6 +-
 kernel/bpf/syscall.c                          | 131 +++-
 kernel/bpf/verifier.c                         | 132 +++-
 tools/bpf/bpftool/btf_dumper.c                |  59 ++
 tools/bpf/bpftool/map.c                       |  10 +-
 tools/bpf/bpftool/xlated_dumper.c             |   3 +
 tools/include/linux/filter.h                  |  21 +-
 tools/include/uapi/linux/bpf.h                |  20 +-
 tools/include/uapi/linux/btf.h                |  32 +-
 tools/lib/bpf/Makefile                        |   2 +-
 tools/lib/bpf/bpf.c                           |  10 +
 tools/lib/bpf/bpf.h                           |   1 +
 tools/lib/bpf/btf.c                           |  97 ++-
 tools/lib/bpf/btf.h                           |   3 +
 tools/lib/bpf/libbpf.c                        | 546 +++++++++++---
 tools/lib/bpf/libbpf.h                        |   5 +
 tools/lib/bpf/libbpf.map                      |   7 +
 tools/testing/selftests/bpf/bpf_helpers.h     |   8 +-
 .../selftests/bpf/prog_tests/global_data.c    | 157 +++++
 .../selftests/bpf/progs/test_global_data.c    | 106 +++
 tools/testing/selftests/bpf/test_btf.c        | 665 +++++++++++++++++-
 tools/testing/selftests/bpf/test_verifier.c   |  51 +-
 .../selftests/bpf/verifier/array_access.c     | 159 +++++
 .../bpf/verifier/direct_value_access.c        | 347 +++++++++
 37 files changed, 3018 insertions(+), 207 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/global_data.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_data.c
 create mode 100644 tools/testing/selftests/bpf/verifier/direct_value_access.c

-- 
2.17.1


^ permalink raw reply	[flat|nested] 10+ messages in thread
* [PATCH bpf-next v6 00/16] BPF support for global data
@ 2019-04-09 21:20 Daniel Borkmann
  2019-04-09 21:20 ` [PATCH bpf-next v6 01/16] bpf: implement lookup-free direct value access for maps Daniel Borkmann
  0 siblings, 1 reply; 10+ messages in thread
From: Daniel Borkmann @ 2019-04-09 21:20 UTC (permalink / raw)
  To: bpf; +Cc: netdev, ast, joe, yhs, andrii.nakryiko, kafai, jannh, Daniel Borkmann

This series is a major rework of previously submitted libbpf
patches [0] in order to add global data support for BPF. The
kernel has been extended to add proper infrastructure that allows
for full .bss/.data/.rodata sections on BPF loader side based
upon feedback from LPC discussions [1]. Latter support is then
also added into libbpf in this series which allows for more
natural C-like programming of BPF programs. For more information
on loader, please refer to 'bpf, libbpf: support global data/bss/
rodata sections' patch in this series.

Thanks a lot!

  v5 -> v6:
   - Removed synchronize_rcu() from map freeze (Jann)
   - Rest as-is
  v4 -> v5:
   - Removed index selection again for ldimm64 (Alexei)
   - Adapted related test cases and added new ones to test
     rejection of off != 0
  v3 -> v4:
   - Various fixes in BTF verification e.g. to disallow
     Var and DataSec to be an intermediate type during resolve (Martin)
   - More BTF test cases added
   - Few cleanups in key-less BTF commit (Martin)
   - Bump libbpf minor version from 2 to 3
   - Renamed and simplified read-only locking
   - Various minor improvements all over the place
  v2 -> v3:
   - Implement BTF support in kernel, libbpf, bpftool, add tests
   - Fix idx + off conversion (Andrii)
   - Document lower / higher bits for direct value access (Andrii)
   - Add tests with small value size (Andrii)
   - Add index selection into ldimm64 (Andrii)
   - Fix missing fdput() (Jann)
   - Reject invalid flags in BPF_F_*_PROG (Jakub)
   - Complete rework of libbpf support, includes:
    - Add objname to map name (Stanislav)
    - Make .rodata map full read-only after setup (Andrii)
    - Merge relocation handling into single one (Andrii)
    - Store global maps into obj->maps array (Andrii, Alexei)
    - Debug message when skipping section (Andrii)
    - Reject non-static global data till we have
      semantics for sharing them (Yonghong, Andrii, Alexei)
    - More test cases and completely reworked prog test (Alexei)
   - Fixes, cleanups, etc all over the set
   - Not yet addressed:
    - Make BTF mandatory for these maps (Alexei)
    -> Waiting till BTF support for these lands first
  v1 -> v2:
    - Instead of 32-bit static data, implement full global
      data support (Alexei)

  [0] https://patchwork.ozlabs.org/cover/1040290/
  [1] http://vger.kernel.org/lpc-bpf2018.html#session-3

Daniel Borkmann (14):
  bpf: implement lookup-free direct value access for maps
  bpf: do not retain flags that are not tied to map lifetime
  bpf: add program side {rd, wr}only support for maps
  bpf: add syscall side map freeze support
  bpf: allow . char as part of the object name
  bpf: add specification for BTF Var and DataSec kinds
  bpf: kernel side support for BTF Var and DataSec
  bpf: allow for key-less BTF in array map
  bpf: sync {btf, bpf}.h uapi header from tools infrastructure
  bpf, libbpf: support global data/bss/rodata sections
  bpf, libbpf: add support for BTF Var and DataSec
  bpf: bpftool support for dumping data/bss/rodata sections
  bpf, selftest: test {rd, wr}only flags and direct value access
  bpf, selftest: add test cases for BTF Var and DataSec

Joe Stringer (2):
  bpf, libbpf: refactor relocation handling
  bpf, selftest: test global data/bss/rodata sections

 Documentation/bpf/btf.rst                     |  57 ++
 include/linux/bpf.h                           |  38 +-
 include/linux/bpf_verifier.h                  |   4 +
 include/linux/btf.h                           |   1 +
 include/uapi/linux/bpf.h                      |  20 +-
 include/uapi/linux/btf.h                      |  32 +-
 kernel/bpf/arraymap.c                         |  53 +-
 kernel/bpf/btf.c                              | 419 ++++++++++-
 kernel/bpf/core.c                             |   3 +-
 kernel/bpf/disasm.c                           |   5 +-
 kernel/bpf/hashtab.c                          |   6 +-
 kernel/bpf/local_storage.c                    |   6 +-
 kernel/bpf/lpm_trie.c                         |   3 +-
 kernel/bpf/queue_stack_maps.c                 |   6 +-
 kernel/bpf/syscall.c                          | 131 +++-
 kernel/bpf/verifier.c                         | 132 +++-
 tools/bpf/bpftool/btf_dumper.c                |  59 ++
 tools/bpf/bpftool/map.c                       |  10 +-
 tools/bpf/bpftool/xlated_dumper.c             |   3 +
 tools/include/linux/filter.h                  |  21 +-
 tools/include/uapi/linux/bpf.h                |  20 +-
 tools/include/uapi/linux/btf.h                |  32 +-
 tools/lib/bpf/Makefile                        |   2 +-
 tools/lib/bpf/bpf.c                           |  10 +
 tools/lib/bpf/bpf.h                           |   1 +
 tools/lib/bpf/btf.c                           |  97 ++-
 tools/lib/bpf/btf.h                           |   3 +
 tools/lib/bpf/libbpf.c                        | 546 +++++++++++---
 tools/lib/bpf/libbpf.h                        |   5 +
 tools/lib/bpf/libbpf.map                      |   7 +
 tools/testing/selftests/bpf/bpf_helpers.h     |   8 +-
 .../selftests/bpf/prog_tests/global_data.c    | 157 +++++
 .../selftests/bpf/progs/test_global_data.c    | 106 +++
 tools/testing/selftests/bpf/test_btf.c        | 665 +++++++++++++++++-
 tools/testing/selftests/bpf/test_verifier.c   |  51 +-
 .../selftests/bpf/verifier/array_access.c     | 159 +++++
 .../bpf/verifier/direct_value_access.c        | 347 +++++++++
 37 files changed, 3018 insertions(+), 207 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/global_data.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_data.c
 create mode 100644 tools/testing/selftests/bpf/verifier/direct_value_access.c

-- 
2.17.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2019-04-10  0:10 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-09 21:08 [PATCH bpf-next v6 00/16] BPF support for global data Daniel Borkmann
2019-04-09 21:08 ` [PATCH bpf-next v6 01/16] bpf: implement lookup-free direct value access for maps Daniel Borkmann
2019-04-09 21:08 ` [PATCH bpf-next v6 02/16] bpf: do not retain flags that are not tied to map lifetime Daniel Borkmann
2019-04-09 21:08 ` [PATCH bpf-next v6 03/16] bpf: add program side {rd, wr}only support for maps Daniel Borkmann
2019-04-09 21:08 ` [PATCH bpf-next v6 04/16] bpf: add syscall side map freeze support Daniel Borkmann
2019-04-09 21:08 ` [PATCH bpf-next v6 05/16] bpf: allow . char as part of the object name Daniel Borkmann
2019-04-09 21:09 ` [PATCH bpf-next v6 06/16] bpf: add specification for BTF Var and DataSec kinds Daniel Borkmann
2019-04-09 21:09 ` [PATCH bpf-next v6 07/16] bpf: kernel side support for BTF Var and DataSec Daniel Borkmann
2019-04-10  0:10 ` [PATCH bpf-next v6 00/16] BPF support for global data Alexei Starovoitov
2019-04-09 21:20 Daniel Borkmann
2019-04-09 21:20 ` [PATCH bpf-next v6 01/16] bpf: implement lookup-free direct value access for maps Daniel Borkmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).