DPDK-dev Archive on lore.kernel.org
 help / color / Atom feed
From: David Marchand <david.marchand@redhat.com>
To: Anatoly Burakov <anatoly.burakov@intel.com>
Cc: dev <dev@dpdk.org>, Pawel Modrak <pawelx.modrak@intel.com>,
	 "Mcnamara, John" <john.mcnamara@intel.com>,
	"Kinsella, Ray" <ray.kinsella@intel.com>,
	 Bruce Richardson <bruce.richardson@intel.com>,
	Thomas Monjalon <thomas@monjalon.net>
Subject: Re: [dpdk-dev] [PATCH v5 02/10] buildtools: add script for updating symbols abi version
Date: Wed, 6 Nov 2019 16:38:55 +0100
Message-ID: <CAJFAV8yJzmbkYH2b2c1A6JFwrqRAaFKCUTRd3JqG6=fEE_QmNg@mail.gmail.com> (raw)
In-Reply-To: <0bdec3d7e6659e3a41b9888a8b3a13eb428426f6.1571910363.git.anatoly.burakov@intel.com>

On Thu, Oct 24, 2019 at 11:46 AM Anatoly Burakov
<anatoly.burakov@intel.com> wrote:
>
> From: Pawel Modrak <pawelx.modrak@intel.com>
>
> Add a script that automatically merges all stable ABI's under one
> ABI section with the new version, while leaving experimental
> section exactly as it is.
>
> Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>
> Notes:
>     v3:
>     - Add comments to regex patterns
>
>     v2:
>     - Reworked script to be pep8-compliant and more reliable
>
>  buildtools/update_version_map_abi.py | 170 +++++++++++++++++++++++++++
>  1 file changed, 170 insertions(+)
>  create mode 100755 buildtools/update_version_map_abi.py
>
> diff --git a/buildtools/update_version_map_abi.py b/buildtools/update_version_map_abi.py
> new file mode 100755
> index 0000000000..50283e6a3d
> --- /dev/null
> +++ b/buildtools/update_version_map_abi.py
> @@ -0,0 +1,170 @@
> +#!/usr/bin/env python
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2019 Intel Corporation
> +
> +"""
> +A Python program to update the ABI version and function names in a DPDK
> +lib_*_version.map file. Called from the buildtools/update_abi.sh utility.
> +"""
> +
> +from __future__ import print_function
> +import argparse
> +import sys
> +import re
> +
> +
> +def __parse_map_file(f_in):
> +    # match function name, followed by semicolon, followed by EOL, optionally
> +    # with whitespace inbetween each item
> +    func_line_regex = re.compile(r"\s*"
> +                                 r"(?P<func>[a-zA-Z_0-9]+)"
> +                                 r"\s*"
> +                                 r";"
> +                                 r"\s*"
> +                                 r"$")
> +    # match section name, followed by opening bracked, followed by EOL,
> +    # optionally with whitespace inbetween each item
> +    section_begin_regex = re.compile(r"\s*"
> +                                     r"(?P<version>[a-zA-Z0-9_\.]+)"
> +                                     r"\s*"
> +                                     r"{"
> +                                     r"\s*"
> +                                     r"$")
> +    # match closing bracket, optionally followed by section name (for when we
> +    # inherit from another ABI version), followed by semicolon, followed by
> +    # EOL, optionally with whitespace inbetween each item
> +    section_end_regex = re.compile(r"\s*"
> +                                   r"}"
> +                                   r"\s*"
> +                                   r"(?P<parent>[a-zA-Z0-9_\.]+)?"
> +                                   r"\s*"
> +                                   r";"
> +                                   r"\s*"
> +                                   r"$")
> +
> +    # for stable ABI, we don't care about which version introduced which
> +    # function, we just flatten the list. there are dupes in certain files, so
> +    # use a set instead of a list
> +    stable_lines = set()
> +    # copy experimental section as is
> +    experimental_lines = []
> +    is_experimental = False
> +
> +    # gather all functions
> +    for line in f_in:
> +        # clean up the line
> +        line = line.strip('\n').strip()
> +
> +        # is this an end of section?
> +        match = section_end_regex.match(line)
> +        if match:
> +            # whatever section this was, it's not active any more
> +            is_experimental = False
> +            continue
> +
> +        # if we're in the middle of experimental section, we need to copy
> +        # the section verbatim, so just add the line
> +        if is_experimental:
> +            experimental_lines += [line]
> +            continue
> +
> +        # skip empty lines
> +        if not line:
> +            continue
> +
> +        # is this a beginning of a new section?
> +        match = section_begin_regex.match(line)
> +        if match:
> +            cur_section = match.group("version")
> +            # is it experimental?
> +            is_experimental = cur_section == "EXPERIMENTAL"
> +            continue
> +
> +        # is this a function?
> +        match = func_line_regex.match(line)
> +        if match:
> +            stable_lines.add(match.group("func"))
> +
> +    return stable_lines, experimental_lines
> +
> +
> +def __regenerate_map_file(f_out, abi_version, stable_lines,
> +                          experimental_lines):
> +    # print ABI version header
> +    print("DPDK_{} {{".format(abi_version), file=f_out)

Some libraries are entirely experimental (librte_bpf for example).

https://patchwork.dpdk.org/patch/62018/
+Libraries marked as ``experimental`` are entirely not considered part of an ABI
+version, and may change without warning at any time. Experimental libraries
+always have a major version of ``0`` to indicate they exist outside of
+:ref:`abi_versioning` , with the minor version incremented with each ABI change
+to library.

So you must create a DPDK_XX "stable" block only if the map file
contained a non empty "stable" block before.


--
David Marchand


  reply index

Thread overview: 131+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-30  9:21 [dpdk-dev] [PATCH 1/8] config: change ABI versioning for global Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 2/8] buildtools: scripts for updating symbols abi version Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 3/8] buildtools: add ABI versioning check script Marcin Baran
2019-09-30 10:27   ` Bruce Richardson
2019-09-30  9:21 ` [dpdk-dev] [PATCH 4/8] build: change ABI version to 20.0 Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 5/8] lib: remove dead code from timer Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 6/8] lib: remove dead code from lpm Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 7/8] lib: remove dead code from distributor Marcin Baran
2019-09-30  9:21 ` [dpdk-dev] [PATCH 8/8] lib: change function suffix in distributor Marcin Baran
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 00/10] Implement the new ABI policy and add helper scripts Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 0/9] " Anatoly Burakov
2019-10-17  8:50     ` Bruce Richardson
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 00/10] " Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 " Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 " Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 " Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 00/12] " Anatoly Burakov
2019-11-20 20:17               ` Thomas Monjalon
2019-11-20 22:13                 ` David Marchand
2019-11-21 10:22                   ` Burakov, Anatoly
2019-11-21 13:24                 ` Kinsella, Ray
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 01/12] config: change ABI versioning to global Anatoly Burakov
2019-11-20 19:51               ` David Marchand
2019-11-20 22:01                 ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 02/12] config: remove CONFIG_RTE_MAJOR_ABI option Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 03/12] build: remove individual library versions Anatoly Burakov
2019-11-20 19:35               ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 04/12] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-11-20 20:05               ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 05/12] buildtools: add ABI update shell script Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 06/12] timer: remove deprecated code Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 07/12] lpm: " Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 08/12] distributor: " Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 09/12] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 10/12] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 11/12] build: change ABI version to 20.0 Anatoly Burakov
2019-11-20 20:47               ` David Marchand
2019-11-20 17:23             ` [dpdk-dev] [PATCH v8 12/12] buildtools: add ABI versioning check script Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 01/10] config: change ABI versioning to global Anatoly Burakov
2019-11-19 13:53             ` Thomas Monjalon
2019-11-19 15:48               ` Burakov, Anatoly
2019-11-20 12:10             ` Kinsella, Ray
2019-11-20 13:31               ` Thomas Monjalon
2019-11-20 14:10                 ` Kinsella, Ray
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-11-19 14:01             ` Thomas Monjalon
2019-11-19 15:38               ` Burakov, Anatoly
2019-11-19 16:05                 ` Thomas Monjalon
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-11-19 17:38             ` Thomas Monjalon
2019-11-20 11:50               ` Burakov, Anatoly
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 04/10] timer: remove deprecated code Anatoly Burakov
2019-11-19 21:42             ` David Marchand
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 05/10] lpm: " Anatoly Burakov
2019-11-19 21:43             ` David Marchand
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 06/10] distributor: " Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-11-19 17:46             ` Thomas Monjalon
2019-11-19 21:50             ` David Marchand
2019-11-22  7:07               ` Sachin Saxena
2019-11-08 16:25           ` [dpdk-dev] [PATCH v7 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-11-19 18:16             ` Thomas Monjalon
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 01/10] config: change ABI versioning to global Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 04/10] timer: remove deprecated code Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 05/10] lpm: " Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 06/10] distributor: " Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-11-06 16:54         ` [dpdk-dev] [PATCH v6 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 01/10] config: change ABI versioning to global Anatoly Burakov
2019-11-05 11:05         ` David Marchand
2019-11-05 13:50           ` Bruce Richardson
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-11-06 15:38         ` David Marchand [this message]
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 04/10] timer: remove deprecated code Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 05/10] lpm: " Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 06/10] distributor: " Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-10-24  9:46       ` [dpdk-dev] [PATCH v5 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 01/10] config: change ABI versioning to global Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 04/10] timer: remove deprecated code Anatoly Burakov
2019-10-17 21:04       ` Carrillo, Erik G
2019-10-21 13:24       ` Kevin Traynor
2019-10-24  9:07         ` Burakov, Anatoly
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 05/10] lpm: " Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 06/10] distributor: " Anatoly Burakov
2019-10-17 15:59       ` Hunt, David
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 07/10] distributor: rename v2.0 ABI to _single suffix Anatoly Burakov
2019-10-17 16:00       ` Hunt, David
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-17 14:31     ` [dpdk-dev] [PATCH v4 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-10-17 14:32     ` [dpdk-dev] [PATCH v4 10/10] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 1/9] config: change ABI versioning to global Anatoly Burakov
2019-10-17  8:44     ` Bruce Richardson
2019-10-17 10:25       ` Burakov, Anatoly
2019-10-17 14:09       ` Luca Boccassi
2019-10-17 14:12         ` Bruce Richardson
2019-10-18 10:07         ` Kevin Traynor
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 2/9] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 3/9] buildtools: add ABI update shell script Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 4/9] timer: remove deprecated code Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 5/9] lpm: " Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 6/9] distributor: " Anatoly Burakov
2019-10-17 10:53     ` Hunt, David
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 7/9] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 8/9] build: change ABI version to 20.0 Anatoly Burakov
2019-10-16 17:03   ` [dpdk-dev] [PATCH v3 9/9] buildtools: add ABI versioning check script Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 01/10] config: change ABI versioning for global Anatoly Burakov
2019-10-16 13:22   ` Bruce Richardson
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 02/10] buildtools: add script for updating symbols abi version Anatoly Burakov
2019-10-16 13:25   ` Bruce Richardson
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 03/10] buildtools: add ABI update shell script Anatoly Burakov
2019-10-16 13:33   ` Bruce Richardson
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 04/10] timer: remove deprecated code Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 05/10] lpm: " Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 06/10] distributor: " Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 07/10] lib: change function suffix in distributor Anatoly Burakov
2019-10-16 13:18   ` Hunt, David
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 08/10] drivers/octeontx: add missing public symbol Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 09/10] build: change ABI version to 20.0 Anatoly Burakov
2019-10-16 12:43 ` [dpdk-dev] [PATCH v2 10/10] buildtools: add ABI versioning check script Anatoly Burakov

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJFAV8yJzmbkYH2b2c1A6JFwrqRAaFKCUTRd3JqG6=fEE_QmNg@mail.gmail.com' \
    --to=david.marchand@redhat.com \
    --cc=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=john.mcnamara@intel.com \
    --cc=pawelx.modrak@intel.com \
    --cc=ray.kinsella@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dpdk-dev/0 dpdk-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dpdk-dev dpdk-dev/ https://lore.kernel.org/dpdk-dev \
		dev@dpdk.org
	public-inbox-index dpdk-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git