All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yucong Sun <fallentree@fb.com>
To: <andrii@kernel.org>
Cc: <sunyucong@gmail.com>, <bpf@vger.kernel.org>,
	Yucong Sun <fallentree@fb.com>
Subject: [RFC 0/1] add parallelism to test_progs
Date: Fri, 27 Aug 2021 16:13:06 -0700	[thread overview]
Message-ID: <20210827231307.3787723-1-fallentree@fb.com> (raw)

This patch added a optional "-p" to test_progs to run tests in multiple
process, speeding up the tests.

Example:

time ./test_progs
real    5m51.393s
user    0m4.695s
sys    5m48.055s

time ./test_progs -p 16 (on a 8 core vm)
real    3m45.673s
user    0m4.434s
sys    5m47.465s

The feedback area I'm looking for :

  1.Some tests are taking too long to run (for example:
  bpf_verif_scale/pyperf* takes almost 80% of the total runtime). If we
  need a work-stealing pool mechanism it would be a bigger change.

  2. The tests output from workers are currently interleaved from all
  workers, making it harder to read, one option would be redirect all
  outputs onto pipes and have main process collect and print in sequence
  for each worker finish, but that will make seeing real time progress
  harder.

  3. If main process want to collect tests results from worker, I plan
  to have each worker writes a stats file to /tmp, or I can use IPC, any
  preference?

  4. Some tests would fail if run in parallel, I think we would need to
  pin some tasks onto worker 0.

Yucong Sun (1):
  selftests/bpf: Add parallelism to test_progs

 tools/testing/selftests/bpf/test_progs.c | 94 ++++++++++++++++++++++--
 tools/testing/selftests/bpf/test_progs.h |  3 +
 2 files changed, 91 insertions(+), 6 deletions(-)

-- 
2.30.2


             reply	other threads:[~2021-08-27 23:13 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-27 23:13 Yucong Sun [this message]
2021-08-27 23:13 ` [RFC 1/1] selftests/bpf: Add parallelism to test_progs Yucong Sun
2021-08-31  3:37   ` Andrii Nakryiko
2021-08-31 12:29     ` sunyucong
2021-08-31  4:03 ` [RFC 0/1] add " Andrii Nakryiko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210827231307.3787723-1-fallentree@fb.com \
    --to=fallentree@fb.com \
    --cc=andrii@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=sunyucong@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.