netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* iproute2 v7: full ss json support and general output simplification
@ 2015-09-10 19:34 Matthias Tafelmeier
  2015-09-10 19:34 ` [PATCH v7 01/10] ss: rooted out ss type declarations for output formatters Matthias Tafelmeier
                   ` (9 more replies)
  0 siblings, 10 replies; 33+ messages in thread
From: Matthias Tafelmeier @ 2015-09-10 19:34 UTC (permalink / raw)
  To: netdev; +Cc: hagen, shemminger, fw, edumazet, daniel

- add full JSON support for ss
- Patchset provides a general and easy to use abstraction to extend ss later
- Patchset size is large to minimize daily use ("user" should not deal with
  	formation (json, human readble) later on)
- Patches 7/10 and 8/10 illustrate how to extend ss for new data to support human readble and json
	output. 
- Example_Usages: 1. ss -jt to print out all tcp related information formatted in json
		  2. ss --json -a to print out all info (also summary) 

STATS:

  ss: rooted out ss type declarations for output formatters
  ss: created formatters for json and hr
  ss: removed obsolet fmt functions
  ss: prepare timer for output handler usage
  ss: replaced old output with new generic output mechanisms
  ss: renaming and export of current_filter
  ss: symmetrical subhandler output extension example
  ss: symmetrical formatter extension example
  ss: fixed free on local array for valid json output
  ss: activate json_writer excluded logic

 include/json_writer.h |   1 +
 lib/json_writer.c     |   7 -
 misc/Makefile         |   2 +-
 misc/ss.c             | 971 +++++++++++++++++---------------------------------
 misc/ss_hr_fmt.c      | 321 +++++++++++++++++
 misc/ss_hr_fmt.h      |   9 +
 misc/ss_json_fmt.c    | 475 ++++++++++++++++++++++++
 misc/ss_json_fmt.h    |  26 ++
 misc/ss_out_fmt.c     | 137 +++++++
 misc/ss_out_fmt.h     |  92 +++++
 misc/ss_types.h       | 186 ++++++++++
 11 files changed, 1567 insertions(+), 660 deletions(-)
 create mode 100644 misc/ss_hr_fmt.c
 create mode 100644 misc/ss_hr_fmt.h
 create mode 100644 misc/ss_json_fmt.c
 create mode 100644 misc/ss_json_fmt.h
 create mode 100644 misc/ss_out_fmt.c
 create mode 100644 misc/ss_out_fmt.h
 create mode 100644 misc/ss_types.h

-- 

Abstract: 

This patch set originates from the necessity to upgrade ss with the possibility
to output in json format. Not to clutter up ss too much, the author of the
patch decided to come up with a simple distributor to handler approach. That
is, the distributor poses the mechanical interface which passes the output
requests coming from ss to the appropriate handler. This simplifies the
interaction with ss and provides a maximum of future extensiblity. Not to
forget, ss loses weight thereby since output implemented in ss itself does
migrate to the appropriate handler. Additionally, because types are shared
amongst handlers, the distributor and ss, the author conceived, that a separate
containter module for types has to be formed. In future, all type declarations
and extensins go there. 

In sum, the patchset has this voluminous extent since there is no viable way
for putting out syntactically correct human readble and json in a simpler manner.
The requirement for convenient extensibility of output and data is
another justification for the patchset size.

Concept sketch:

                                               formatter1 
                                              ************                
                                              *          *                
                                              *          *                
           ss                           ~~~~~~~>zzzzzzz  *                
     ******************                 ~     *          *                
     *                *                 ~   ###>fffffff  *                
     *                *                 ~   # *          *                
     *                *      distributor~   # ************                
     *   --------     *       ********* ~   #                             
     *   -    --------------  *       * ~   #                             
     *   --------     *    -  *       * ~   #                             
     *                *    ---->++++ ~~~~   #                             
     *                *       *       * ~   #  formatter2 
     *                *    ---->==== ######## ************                
     *   --------     *    -  *       * ~   # *          *                
     *   -    --------------  *       * ~   # *          *                
     *   --------     *       ********* ~   # *          *                
     *                *                 ~~~~#~~>zzzzzzz  *                
     *                *                     # *          *                
     *                *                     ###>fffffff	 *                
     ******************                       *          *                
                                              ************                
						                       
At the moment, the distributor is the ss_out_fmt module while two handlers are
up: namely the ss_json_fmt and the ss_hr_fmt (human readable). You can use
those modules as the main reference for own extensions.

Future Extension:
In the following, I will expand on the expandability of the formatter model.
The explanations advances from the minimal to the most sweeping extension in
mind.

Sub Format Handler Output 
Sketch

		  FormatterX
                  ***********************************                     
                  *                                 *                     
                  *   handlerX 			    *                     
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *                     
                  *   °                        °    *                     
                  *   °    xxxxxxxxxxxxxxc<..  °    *                     
                  *   °                     .  °    *                     
                  *   °    xxxxxxxxxxxxxxc<.. potential context                  
                  *   °        new:         .  °    *      
                  *   °    +++++++++++++++... < * * * * * * * * * * *       
                  *   °                        °    *               *      
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *               *      
                  *               .                 *               *      
                  *               .                 *               *      
                  *               .                 *               *      
                  *   handlerY		  	    *               *      
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *               *      
                  *   °                        °    *               *      
                  *   °    xxxxxxxxxxxxxxx     °    *               *      
                  *   °                        °    *               *      
                  *   °    xxxxxxxxxxxxxxx     °    *               *      
                  *   °                        °    *               *   expand symmetrically   
                  *   °                        °    *               *      
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *               *      
                  *                                 *               *      
                  *                                 *               *      
                  *                                 *               *      
                  ***********************************          	    *
								    *
		 		.				    *
				.				    *
				.				    *
				 				    *
		  FormatterY					    *
                  ***********************************               *     
                  *                                 *               *     
                  *   handlerX			    *               *      
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *               *      
                  *   °                        °    *               *      
                  *   °    zzzzzzzzzzzzzzc<..  °    *               *      
                  *   °                     .  °    *               *      
                  *   °    zzzzzzzzzzzzzzc<.. potential context     *              
                  *   °        new:         .  °    *               *     
                  *   °    +++++++++++++++... < * * * * * * * * * * *     
                  *   °                        °    *                     
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *                     
                  *               .                 *                     
                  *               .                 *                     
                  *               .                 *                     
                  *   handlerY		  	    *                     
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *                     
                  *   °                        °    *                     
                  *   °    zzzzzzzzzzzzzzz     °    *                     
                  *   °                        °    *                     
                  *   °    zzzzzzzzzzzzzzz     °    *                     
                  *   °                        °    *                     
                  *   °                        °    *                     
                  *   °°°°°°°°°°°°°°°°°°°°°°°°°°    *                     
                  *                                 *                     
                  *                                 *                     
                  ***********************************     

Explanation:
If you plan to expand a sub out handler function of a formatter, it
essentially boils down to adding a new printf with an according format and
probably a necessary predicate (condition). Nontheless, care must be taken not
to lose possible context interdependecies out of sight. An examble for the
latter would be the interdependecy of json coma setting terms – in compound
types, you do need a coma between consecutive elements.

More important is the issue about symmetric extensions. Except for the
tcp_out_fmt function implementation – where a macro (CHECK_FMT_ADAPT) is in
place to check for adaptions in the basic tcpstat data structure statically –
no general programmatic approach is in place yet which would prevent asymmetric
extensions. Up to someone devices a holistic solution, this patch relies on the
extenders to deal with asymmetries. Is the aim to have a new output feature
available in all semantically related handlers of n different formatters, then
the expander has to adapt n handlers as shown in the sketch above.

Extend with further Format Handler in Formatter
Sketch:
-provides symmetrical extension intendet


                                                       formatter1
                                                      /--------------------------\
                                                      |                          |
                                                      |    spec_handlerX         |
                                                      |     ################ <** |
                                                      |     ################   * |
             distributor                              |            .           * |
           O-------------------------O                |            .           * |
           |    centr_hub            |                |    new_spec_handler    * |
           | ***>01111110--------------------------+  |     ++++++++++++++++<~ * |
           | *   02222220----------------------+   |  |     ++++++++++++++++ ~ * |
           | *                       |         |   |  |                      ~ * |
           | *                       |         |   |  |      handler_hub1    ~ * |
           | *  _______________      |         |   +--------->0#########0****~** |
           | * |gen_handlerX   |     |         |      |	      0+++++++++0~~~~~   |
           | **********        |     |         |      |                          |
           | * |_______________|     |         |      |                          |
           | *         .             |         |      \--------------------------/
           | *         .             |         |                  .
           | *         .             |         |                  .
           | * +++++++++++++++++     |         |       formatterN .
           | ***new_gen_handler+     |         |      /--------------------------\
           |   +               +     |         |      |                          |
           |   +++++++++++++++++     |         |      |    spec_handlerX         |
           |                         |         |      |     ################ <** |
           O-------------------------O         |      |     ################   * |
                                               |      |            .           * |
                                               |      |            .           * |
                                               |      |    new_spec_handler    * |
                                               |      |     ++++++++++++++++<~ * |
                                               |      |     ++++++++++++++++ ~ * |
                                               |      |                      ~ * |
                                               |      |      handler_hubN    ~ * |
                                               +------------->0#########0****~** |
                                                      |	      0+++++++++0~~~~~   |
                                                      |                          |
                                                      |                          |
                                                      \--------------------------/


Explanation:
As the sketch shows, the distributor works with the help of virtual function
pointer in order to act as a call flow switch. It switches to the approriate
formatter module and its handlers depending on the chosen output format by ss
command input. 

So, to add a new formatter handler symmetrically (up to now that is the only
sensibly conceivable case), the extender must implement a new generic handler
in the distributor and the specific handlers in the formatters. Then the hub
vtable structure type has to be broadend to contain the new function pointer
type for the generic handler. After that, he has to extend and update all
handler hubs with the new handlers location information (function pointer). The
latter ensures the generic switching mechanism used by the generic handler
keeps to be upheld.

Example:
Let's say we want the new "foo" data for every output format retrievable via
ss. Up to now, we have the ss_out_fmt module as the distributor and two
specific handlers: for one the ss_hr_fmt and secondly the ss_json_fmt module.
So we need a specific handler implementation in ss_hr_fmt and ss_json_fmt
modules and after that update the corresponding vtables (handler_hubs) in the
modules.  After that, the distributor, namely ss_out_fmt module, has to get a
generic handler that switches via its vtable hub to either the json formatter
or the human readable formatter, depending on what fmt_type has been chosen by ss.
Before the vtables in the specific modules can be updated, struct fmt_op_hub
which is found in ss_out_fmt's interface header has get extendend with the new
function pointer type.

As soon the new generic handler has been exported via the ss_out_fmt.h module
interface, ss can use the new fmt handler to print out info. It can simply call
the generic function and does not have to deal with formatting specific issues.

Extend for another Formatter 
Sketch:
The Sketch for handler extension should be sufficient for conveying the concept.
Just think of another formatter after formatterN and a new entry in the central
vtable of the distributor to reach this new formatter.

Explanation:
Nothing breathtaking has to be done when someone needs an new formatter module
for let's be image – out of pure hypothetical endeavors – xml ss output. First,
implement the new formatter with all the offered interfaces in the
distributor. Register all handlers in the local specific vtable hub.
Then, register the local vtable hub in the generic vtable hub of the
distributor to reach your new handler when chosen. Provide the client
code - here ss - with a new fmt_type option acceptance. Before the
option can do anything, you have to declare the new fmt_type. That's
it. No further adaptions in ss would be necessary.

^ permalink raw reply	[flat|nested] 33+ messages in thread

end of thread, other threads:[~2015-11-03  6:55 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-09-10 19:34 iproute2 v7: full ss json support and general output simplification Matthias Tafelmeier
2015-09-10 19:34 ` [PATCH v7 01/10] ss: rooted out ss type declarations for output formatters Matthias Tafelmeier
2015-09-10 19:35 ` [PATCH v7 02/10] ss: created formatters for json and hr Matthias Tafelmeier
2015-09-23 23:26   ` Stephen Hemminger
2015-09-24 17:43     ` Matthias Tafelmeier
     [not found]     ` <bc9cacb94de24857a7d9e13960bfae56@HQ1WP-EXMB11.corp.brocade.com>
2015-09-24 22:16       ` Stephen Hemminger
2015-09-25  6:01         ` Matthias Tafelmeier
2015-10-28  8:07           ` Matthias Tafelmeier
2015-10-28  8:28             ` Phil Sutter
2015-10-28 11:57               ` Matthias Tafelmeier
2015-10-28 15:05                 ` Phil Sutter
2015-10-28 15:50                   ` Matthias Tafelmeier
     [not found]           ` <b114412a541d4273a9a7a1c0ae4777df@HQ1WP-EXMB11.corp.brocade.com>
2015-11-02 23:06             ` Stephen Hemminger
2015-11-03  6:54               ` Matthias Tafelmeier
2015-10-27 12:21   ` Phil Sutter
2015-10-27 13:47     ` David Miller
2015-10-27 13:35       ` Phil Sutter
2015-09-10 19:35 ` [PATCH v7 03/10] ss: removed obsolet fmt functions Matthias Tafelmeier
2015-09-10 19:35 ` [PATCH v7 04/10] ss: prepare timer for output handler usage Matthias Tafelmeier
2015-09-10 19:35 ` [PATCH v7 05/10] ss: replaced old output with new generic output mechanisms Matthias Tafelmeier
2015-10-27 13:04   ` Phil Sutter
2015-09-10 19:35 ` [PATCH v7 06/10] ss: renaming and export of current_filter Matthias Tafelmeier
2015-10-27 13:10   ` Phil Sutter
2015-09-10 19:35 ` [PATCH v7 07/10] ss: symmetrical subhandler output extension example Matthias Tafelmeier
2015-10-27 13:13   ` Phil Sutter
2015-09-10 19:35 ` [PATCH v7 08/10] ss: symmetrical formatter " Matthias Tafelmeier
2015-10-27 13:17   ` Phil Sutter
2015-09-10 19:35 ` [PATCH v7 09/10] ss: fixed free on local array for valid json output Matthias Tafelmeier
2015-10-27 13:19   ` Phil Sutter
2015-09-10 19:35 ` [PATCH v7 10/10] ss: activate json_writer excluded logic Matthias Tafelmeier
2015-10-27 13:21   ` Phil Sutter
2015-10-28  2:39     ` Stephen Hemminger
2015-10-28  8:17       ` Phil Sutter

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).