All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v1 1/2] distributor lib performance enhancements
@ 2016-12-01  4:50 David Hunt
  2016-12-01  4:50 ` [PATCH v1 1/2] lib: distributor " David Hunt
  2016-12-01  4:50 ` [PATCH v1 2/2] example: distributor app modified to use burstAPI David Hunt
  0 siblings, 2 replies; 202+ messages in thread
From: David Hunt @ 2016-12-01  4:50 UTC (permalink / raw)
  To: dev; +Cc: bruce.richardson

This patch aims to improve the throughput of the distributor library.

It uses a similar handshake mechanism to the previous version of
the library, in that bits are used to indicate when packets are ready
to be sent to a worker and ready to be returned from a worker. One main
difference is that instead of sending one packet in a cache line, it makes
use of the 7 free spaces in the same cache line in order to send up to
8 packets at a time to/from a worker.

The flow matching algorithm has had significant re-work, and now keeps an
array of inflight flows and an array of backlog flows, and matches incoming
flows to the inflight/backlog flows of all workers so that flow pinning to
workers can be maintained.

Notes:
   Apps must now work in bursts, as up to 8 are given to a worker at a time
   For performance in matching, Flow ID's are 15-bits
   Original API (and code) is kept for backward compatibility

Performance Gains
   2.2GHz Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
   2 x XL710 40GbE NICS to 2 x 40Gbps traffic generator channels 64b packets
   separate cores for rx, tx, distributor
    1 worker  - 4.8x
    4 workers - 2.9x
    8 workers - 1.8x
   12 workers - 2.1x
   16 workers - 1.8x


[PATCH v1 1/2] lib: distributor performance enhancements
[PATCH v1 2/2] example: distributor app modified to use burstAPI

^ permalink raw reply	[flat|nested] 202+ messages in thread

end of thread, other threads:[~2017-03-29 14:48 UTC | newest]

Thread overview: 202+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-01  4:50 [PATCH v1 1/2] distributor lib performance enhancements David Hunt
2016-12-01  4:50 ` [PATCH v1 1/2] lib: distributor " David Hunt
2016-12-22  4:37   ` [PATCH v2 0/5] distributor library " David Hunt
2016-12-22  4:37     ` [PATCH v2 1/5] lib: distributor " David Hunt
2016-12-22 12:47       ` Jerin Jacob
2016-12-22 16:14         ` Hunt, David
2017-01-02 10:22       ` [WARNING: A/V UNSCANNABLE][PATCH v3 0/6] distributor-performance-improvements David Hunt
2017-01-02 10:22         ` [WARNING: A/V UNSCANNABLE][PATCH v3 1/6] lib: distributor performance enhancements David Hunt
2017-01-02 10:22         ` [WARNING: A/V UNSCANNABLE][PATCH v3 2/6] lib: add distributor vector flow matching David Hunt
2017-01-02 10:22         ` [WARNING: A/V UNSCANNABLE][PATCH v3 3/6] test: unit tests for new distributor burst api David Hunt
2017-01-02 10:22         ` [WARNING: A/V UNSCANNABLE][PATCH v3 4/6] test: add distributor_perf autotest David Hunt
2017-01-02 10:22         ` [WARNING: A/V UNSCANNABLE][PATCH v3 5/6] example: distributor app showing burst api David Hunt
2017-01-02 10:22         ` [WARNING: A/V UNSCANNABLE][PATCH v3 6/6] doc: distributor library changes for new " David Hunt
2017-01-09  7:50       ` [PATCH v4 0/6] distributor library performance enhancements David Hunt
2017-01-09  7:50         ` [PATCH v4 1/6] lib: distributor " David Hunt
2017-01-13 15:19           ` Bruce Richardson
2017-01-19 14:58             ` Hunt, David
2017-01-16 16:36           ` Bruce Richardson
2017-01-19 12:07             ` Hunt, David
2017-01-20  9:18           ` [PATCH v5 0/6] distributor library " David Hunt
2017-01-20  9:18             ` [PATCH v5 1/6] lib: distributor " David Hunt
2017-01-23  9:24               ` [PATCH v6 0/6] distributor library " David Hunt
2017-01-23  9:24                 ` [PATCH v6 1/6] lib: distributor " David Hunt
2017-02-21  3:17                   ` [PATCH v7 0/17] distributor library " David Hunt
2017-02-21  3:17                     ` [PATCH v7 01/17] lib: rename legacy distributor lib files David Hunt
2017-02-21 10:27                       ` Hunt, David
2017-02-24 14:03                       ` Bruce Richardson
2017-03-01  9:55                         ` Hunt, David
2017-03-01  7:47                       ` [PATCH v8 0/18] distributor library performance enhancements David Hunt
2017-03-01  7:47                         ` [PATCH v8 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-06  9:10                           ` [PATCH v9 00/18] distributor lib performance enhancements David Hunt
2017-03-06  9:10                             ` [PATCH v9 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-15  6:19                               ` [PATCH v10 0/18] distributor library performance enhancements David Hunt
2017-03-15  6:19                                 ` [PATCH v10 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-20 10:08                                   ` [PATCH v11 0/18] distributor lib performance enhancements David Hunt
2017-03-20 10:08                                     ` [PATCH v11 01/18] lib: rename legacy distributor lib files David Hunt
2017-03-20 10:08                                     ` [PATCH v11 02/18] lib: create private header file David Hunt
2017-03-20 10:08                                     ` [PATCH v11 03/18] lib: add new distributor code David Hunt
2017-03-20 10:08                                     ` [PATCH v11 04/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-20 10:08                                     ` [PATCH v11 05/18] test/distributor: extra params for autotests David Hunt
2017-03-20 10:08                                     ` [PATCH v11 06/18] lib: switch distributor over to new API David Hunt
2017-03-20 10:08                                     ` [PATCH v11 07/18] lib: make v20 header file private David Hunt
2017-03-27 13:10                                       ` Thomas Monjalon
2017-03-28  8:47                                         ` Hunt, David
2017-03-20 10:08                                     ` [PATCH v11 08/18] lib: add symbol versioning to distributor David Hunt
2017-03-27 13:02                                       ` Thomas Monjalon
2017-03-28  8:25                                         ` Hunt, David
2017-03-20 10:08                                     ` [PATCH v11 09/18] test: test single and burst distributor API David Hunt
2017-03-20 10:08                                     ` [PATCH v11 10/18] test: add perf test for distributor burst mode David Hunt
2017-03-20 10:08                                     ` [PATCH v11 11/18] examples/distributor: allow for extra stats David Hunt
2017-03-20 10:08                                     ` [PATCH v11 12/18] examples/distributor: wait for ports to come up David Hunt
2017-03-20 10:08                                     ` [PATCH v11 13/18] examples/distributor: add dedicated core for dist David Hunt
2017-03-20 10:08                                     ` [PATCH v11 14/18] examples/distributor: tweaks for performance David Hunt
2017-03-27 13:04                                       ` Thomas Monjalon
2017-03-28  8:45                                         ` Hunt, David
2017-03-20 10:08                                     ` [PATCH v11 15/18] examples/distributor: give Rx thread a core David Hunt
2017-03-20 10:08                                     ` [PATCH v11 16/18] doc: distributor library changes for new burst API David Hunt
2017-03-24 14:49                                       ` Mcnamara, John
2017-03-20 10:08                                     ` [PATCH v11 17/18] doc: distributor app " David Hunt
2017-03-20 10:08                                     ` [PATCH v11 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-03-27 13:06                                     ` [PATCH v11 0/18] distributor lib performance enhancements Thomas Monjalon
2017-03-29 14:48                                       ` Thomas Monjalon
2017-03-15  6:19                                 ` [PATCH v10 02/18] lib: create private header file David Hunt
2017-03-15 17:18                                   ` Thomas Monjalon
2017-03-16 10:43                                     ` Hunt, David
2017-03-16 15:40                                       ` Thomas Monjalon
2017-03-15  6:19                                 ` [PATCH v10 03/18] lib: add new distributor code David Hunt
2017-03-15  6:19                                 ` [PATCH v10 04/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-15  6:19                                 ` [PATCH v10 05/18] test/distributor: extra params for autotests David Hunt
2017-03-15  6:19                                 ` [PATCH v10 06/18] lib: switch distributor over to new API David Hunt
2017-03-15  6:19                                 ` [PATCH v10 07/18] lib: make v20 header file private David Hunt
2017-03-15  6:19                                 ` [PATCH v10 08/18] lib: add symbol versioning to distributor David Hunt
2017-03-15  6:19                                 ` [PATCH v10 09/18] test: test single and burst distributor API David Hunt
2017-03-15  6:19                                 ` [PATCH v10 10/18] test: add perf test for distributor burst mode David Hunt
2017-03-15  6:19                                 ` [PATCH v10 11/18] examples/distributor: allow for extra stats David Hunt
2017-03-15  6:19                                 ` [PATCH v10 12/18] examples/distributor: wait for ports to come up David Hunt
2017-03-15  6:19                                 ` [PATCH v10 13/18] examples/distributor: add dedicated core for dist David Hunt
2017-03-15  6:19                                 ` [PATCH v10 14/18] examples/distributor: tweaks for performance David Hunt
2017-03-15  6:19                                 ` [PATCH v10 15/18] examples/distributor: give Rx thread a core David Hunt
2017-03-15  6:19                                 ` [PATCH v10 16/18] doc: distributor library changes for new burst API David Hunt
2017-03-15  6:19                                 ` [PATCH v10 17/18] doc: distributor app " David Hunt
2017-03-15  6:19                                 ` [PATCH v10 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-03-06  9:10                             ` [PATCH v9 02/18] lib: create private header file David Hunt
2017-03-06  9:10                             ` [PATCH v9 03/18] lib: add new burst oriented distributor structs David Hunt
2017-03-06  9:10                             ` [PATCH v9 04/18] lib: add new distributor code David Hunt
2017-03-10 16:03                               ` Bruce Richardson
2017-03-14 10:43                                 ` Hunt, David
2017-03-06  9:10                             ` [PATCH v9 05/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-06  9:10                             ` [PATCH v9 06/18] test/distributor: extra params for autotests David Hunt
2017-03-06  9:10                             ` [PATCH v9 07/18] lib: switch distributor over to new API David Hunt
2017-03-06  9:10                             ` [PATCH v9 08/18] lib: make v20 header file private David Hunt
2017-03-06  9:10                             ` [PATCH v9 09/18] lib: add symbol versioning to distributor David Hunt
2017-03-10 16:22                               ` Bruce Richardson
2017-03-13 10:17                                 ` Hunt, David
2017-03-13 10:28                                 ` Hunt, David
2017-03-13 11:01                                   ` Van Haaren, Harry
2017-03-13 11:02                                     ` Hunt, David
2017-03-06  9:10                             ` [PATCH v9 10/18] test: test single and burst distributor API David Hunt
2017-03-06  9:10                             ` [PATCH v9 11/18] test: add perf test for distributor burst mode David Hunt
2017-03-06  9:10                             ` [PATCH v9 12/18] examples/distributor: allow for extra stats David Hunt
2017-03-10 16:46                               ` Bruce Richardson
2017-03-14 10:44                                 ` Hunt, David
2017-03-06  9:10                             ` [PATCH v9 13/18] sample: distributor: wait for ports to come up David Hunt
2017-03-10 16:48                               ` Bruce Richardson
2017-03-06  9:10                             ` [PATCH v9 14/18] examples/distributor: give distributor a core David Hunt
2017-03-10 16:49                               ` Bruce Richardson
2017-03-14 10:48                                 ` Hunt, David
2017-03-06  9:10                             ` [PATCH v9 15/18] examples/distributor: limit number of Tx rings David Hunt
2017-03-10 16:50                               ` Bruce Richardson
2017-03-14 10:50                                 ` Hunt, David
2017-03-06  9:10                             ` [PATCH v9 16/18] examples/distributor: give Rx thread a core David Hunt
2017-03-10 16:51                               ` Bruce Richardson
2017-03-14  9:34                                 ` Hunt, David
2017-03-06  9:10                             ` [PATCH v9 17/18] doc: distributor library changes for new burst API David Hunt
2017-03-07 17:25                               ` Mcnamara, John
2017-03-06  9:10                             ` [PATCH v9 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-03-10 16:54                             ` [PATCH v9 00/18] distributor lib performance enhancements Bruce Richardson
2017-03-01  7:47                         ` [PATCH v8 02/18] lib: create private header file David Hunt
2017-03-01  7:47                         ` [PATCH v8 03/18] lib: add new burst oriented distributor structs David Hunt
2017-03-01  7:47                         ` [PATCH v8 04/18] lib: add new distributor code David Hunt
2017-03-01  7:47                         ` [PATCH v8 05/18] lib: add SIMD flow matching to distributor David Hunt
2017-03-01  7:47                         ` [PATCH v8 06/18] test/distributor: extra params for autotests David Hunt
2017-03-01  7:47                         ` [PATCH v8 07/18] lib: switch distributor over to new API David Hunt
2017-03-01  7:47                         ` [PATCH v8 08/18] lib: make v20 header file private David Hunt
2017-03-01  7:47                         ` [PATCH v8 09/18] lib: add symbol versioning to distributor David Hunt
2017-03-01 14:50                           ` Hunt, David
2017-03-01  7:47                         ` [PATCH v8 10/18] test: test single and burst distributor API David Hunt
2017-03-01  7:47                         ` [PATCH v8 11/18] test: add perf test for distributor burst mode David Hunt
2017-03-01  7:47                         ` [PATCH v8 12/18] examples/distributor: allow for extra stats David Hunt
2017-03-01  7:47                         ` [PATCH v8 13/18] sample: distributor: wait for ports to come up David Hunt
2017-03-01  7:47                         ` [PATCH v8 14/18] examples/distributor: give distributor a core David Hunt
2017-03-01  7:47                         ` [PATCH v8 15/18] examples/distributor: limit number of Tx rings David Hunt
2017-03-01  7:47                         ` [PATCH v8 16/18] examples/distributor: give Rx thread a core David Hunt
2017-03-01  7:47                         ` [PATCH v8 17/18] doc: distributor library changes for new burst API David Hunt
2017-03-01  7:47                         ` [PATCH v8 18/18] maintainers: add to distributor lib maintainers David Hunt
2017-02-21  3:17                     ` [PATCH v7 02/17] lib: symbol versioning of functions in distributor David Hunt
2017-02-24 14:05                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 03/17] lib: create rte_distributor_private.h David Hunt
2017-02-24 14:07                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 04/17] lib: add new burst oriented distributor structs David Hunt
2017-02-24 14:08                       ` Bruce Richardson
2017-03-01  9:57                         ` Hunt, David
2017-02-24 14:09                       ` Bruce Richardson
2017-03-01  9:58                         ` Hunt, David
2017-02-21  3:17                     ` [PATCH v7 05/17] lib: add new distributor code David Hunt
2017-02-24 14:11                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 06/17] lib: add SIMD flow matching to distributor David Hunt
2017-02-24 14:11                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 07/17] lib: apply symbol versioning to distibutor lib David Hunt
2017-02-21 11:50                       ` Hunt, David
2017-02-24 14:12                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 08/17] test: change params to distributor autotest David Hunt
2017-02-24 14:14                       ` Bruce Richardson
2017-03-01 10:06                         ` Hunt, David
2017-02-21  3:17                     ` [PATCH v7 09/17] test: switch distributor test over to burst API David Hunt
2017-02-21  3:17                     ` [PATCH v7 10/17] test: test single and burst distributor API David Hunt
2017-02-21  3:17                     ` [PATCH v7 11/17] test: add perf test for distributor burst mode David Hunt
2017-02-21  3:17                     ` [PATCH v7 12/17] example: add extra stats to distributor sample David Hunt
2017-02-24 14:16                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 13/17] sample: distributor: wait for ports to come up David Hunt
2017-02-21  3:17                     ` [PATCH v7 14/17] sample: switch to new distributor API David Hunt
2017-02-24 14:16                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 15/17] lib: make v20 header file private David Hunt
2017-02-24 14:18                       ` Bruce Richardson
2017-02-21  3:17                     ` [PATCH v7 16/17] doc: distributor library changes for new burst api David Hunt
2017-02-21 16:18                       ` Mcnamara, John
2017-02-21  3:17                     ` [PATCH v7 17/17] maintainers: add to distributor lib maintainers David Hunt
2017-02-24 14:01                     ` [PATCH v7 0/17] distributor library performance enhancements Bruce Richardson
2017-01-23  9:24                 ` [PATCH v6 2/6] lib: add distributor vector flow matching David Hunt
2017-01-23  9:24                 ` [PATCH v6 3/6] test: unit tests for new distributor burst API David Hunt
2017-01-23  9:24                 ` [PATCH v6 4/6] test: add distributor perf autotest David Hunt
2017-01-23  9:24                 ` [PATCH v6 5/6] examples/distributor_app: showing burst API David Hunt
2017-01-23  9:24                 ` [PATCH v6 6/6] doc: distributor library changes for new " David Hunt
2017-01-23 17:02                 ` [PATCH v6 0/6] distributor library performance enhancements Bruce Richardson
2017-01-24  8:56                 ` Liu, Yong
2017-01-23 12:26               ` [PATCH v5 1/6] lib: distributor " Bruce Richardson
2017-01-20  9:18             ` [PATCH v5 2/6] lib: add distributor vector flow matching David Hunt
2017-01-20  9:18             ` [PATCH v5 3/6] test: unit tests for new distributor burst API David Hunt
2017-01-20  9:18             ` [PATCH v5 4/6] test: add distributor perf autotest David Hunt
2017-01-20  9:18             ` [PATCH v5 5/6] examples/distributor_app: showing burst API David Hunt
2017-01-23 12:31               ` Bruce Richardson
2017-01-20  9:18             ` [PATCH v5 6/6] doc: distributor library changes for new " David Hunt
2017-01-09  7:50         ` [PATCH v4 2/6] lib: add distributor vector flow matching David Hunt
2017-01-13 15:26           ` Bruce Richardson
2017-01-19 14:59             ` Hunt, David
2017-01-16 16:40           ` Bruce Richardson
2017-01-19 12:11             ` Hunt, David
2017-01-09  7:50         ` [PATCH v4 3/6] test: unit tests for new distributor burst api David Hunt
2017-01-13 15:33           ` Bruce Richardson
2017-01-09  7:50         ` [PATCH v4 4/6] test: add distributor_perf autotest David Hunt
2017-01-09  7:50         ` [PATCH v4 5/6] example: distributor app showing burst api David Hunt
2017-01-13 15:36           ` Bruce Richardson
2017-01-13 15:38           ` Bruce Richardson
2017-01-09  7:50         ` [PATCH v4 6/6] doc: distributor library changes for new " David Hunt
2016-12-22  4:37     ` [PATCH v2 2/5] test: unit tests for new distributor " David Hunt
2016-12-22  4:37     ` [PATCH v2 3/5] test: add distributor_perf autotest David Hunt
2016-12-22 12:19       ` Jerin Jacob
2017-01-02 16:24         ` Hunt, David
2017-01-04 13:09           ` Jerin Jacob
2016-12-22  4:37     ` [PATCH v2 4/5] example: distributor app showing burst api David Hunt
2016-12-22  4:37     ` [PATCH v2 5/5] doc: distributor library changes for new " David Hunt
2016-12-01  4:50 ` [PATCH v1 2/2] example: distributor app modified to use burstAPI David Hunt

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.