Re: [PATCH v4 bpf-next 5/7] selftests/bpf: replace test_progs and test_maps w/ general rule

[Alexei Starovoitov <ast@fb.com>]

On 10/17/19 10:50 AM, Andrii Nakryiko wrote:
> On Tue, Oct 15, 2019 at 11:01 PM Andrii Nakryiko <andriin@fb.com> wrote:
>>
>> Define test runner generation meta-rule that codifies dependencies
>> between test runner, its tests, and its dependent BPF programs. Use that
>> for defining test_progs and test_maps test-runners. Also additionally define
>> 2 flavors of test_progs:
>> - alu32, which builds BPF programs with 32-bit registers codegen;
>> - bpf_gcc, which build BPF programs using GCC, if it supports BPF target.
>>
>> Overall, this is accomplished through $(eval)'ing a set of generic
>> rules, which defines Makefile targets dynamically at runtime. See
>> comments explaining the need for 2 $(evals), though.
>>
>> For each test runner we have (test_maps and test_progs, currently), and,
>> optionally, their flavors, the logic of build process is modeled as
>> follows (using test_progs as an example):
>> - all BPF objects are in progs/:
>>    - BPF object's .o file is built into output directory from
>>      corresponding progs/.c file;
>>    - all BPF objects in progs/*.c depend on all progs/*.h headers;
>>    - all BPF objects depend on bpf_*.h helpers from libbpf (but not
>>      libbpf archive). There is an extra rule to trigger bpf_helper_defs.h
>>      (re-)build, if it's not present/outdated);
>>    - build recipe for BPF object can be re-defined per test runner/flavor;
>> - test files are built from prog_tests/*.c:
>>    - all such test file objects are built on individual file basis;
>>    - currently, every single test file depends on all BPF object files;
>>      this might be improved in follow up patches to do 1-to-1 dependency,
>>      but allowing to customize this per each individual test;
>>    - each test runner definition can specify a list of extra .c and .h
>>      files to be built along test files and test runner binary; all such
>>      headers are becoming automatic dependency of each test .c file;
>>    - due to test files sometimes embedding (using .incbin assembly
>>      directive) contents of some BPF objects at compilation time, which are
>>      expected to be in CWD of compiler, compilation for test file object does
>>      cd into test runner's output directory; to support this mode all the
>>      include paths are turned into absolute paths using $(abspath) make
>>      function;
>> - prog_tests/test.h is automatically (re-)generated with an entry for
>>    each .c file in prog_tests/;
>> - final test runner binary is linked together from test object files and
>>    extra object files, linking together libbpf's archive as well;
>> - it's possible to specify extra "resource" files/targets, which will be
>>    copied into test runner output directory, if it differes from
>>    Makefile-wide $(OUTPUT). This is used to ensure btf_dump test cases and
>>    urandom_read binary is put into a test runner's CWD for tests to find
>>    them in runtime.
>>
>> For flavored test runners, their output directory is a subdirectory of
>> common Makefile-wide $(OUTPUT) directory with flavor name used as
>> subdirectory name.
>>
>> BPF objects targets might be reused between different test runners, so
>> extra checks are employed to not double-define them. Similarly, we have
>> redefinition guards for output directories and test headers.
>>
>> test_verifier follows slightly different patterns and is simple enough
>> to not justify generalizing TEST_RUNNER_DEFINE/TEST_RUNNER_DEFINE_RULES
>> further to accomodate these differences. Instead, rules for
>> test_verifier are minimized and simplified, while preserving correctness
>> of dependencies.
>>
>> Signed-off-by: Andrii Nakryiko <andriin@fb.com>
>> ---
> 
> BTW, if correctness and DRY-ness argument is not strong enough, these
> changes makes clean rebuild from scratch about 2x faster for me:
> 
> BEFORE: `make clean && time make -j50` is 14-15 seconds
> AFTER: `make clean && time make -j50` is 7-8 seconds

I noticed that too and was about to ask "why?" .. :)