netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Warning when compiling with python3.12
@ 2023-08-11  8:47 Srikar Dronamraju
  2023-08-16 12:21 ` [PATCH] Fix invalid escape sequence warnings Vishal Chourasia
  0 siblings, 1 reply; 8+ messages in thread
From: Srikar Dronamraju @ 2023-08-11  8:47 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
	Hao Luo, Jiri Olsa, David S. Miller, Jakub Kicinski,
	Jesper Dangaard Brouer, bpf, linux-kernel, netdev, Sachin Sant

Hi,

When trying to build on v6.5-rc4 with python 3.12 aka Python 3.12.0rc1 I am
hitting the below Warning messages.

I didn't see something similar reported upstream, hence thought of reporting.

/home/srikar/linux.git/scripts/bpf_doc.py:62: SyntaxWarning: invalid escape sequence '\w'
  arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$')
/home/srikar/linux.git/scripts/bpf_doc.py:64: SyntaxWarning: invalid escape sequence '\*'
  proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
/home/srikar/linux.git/scripts/bpf_doc.py:117: SyntaxWarning: invalid escape sequence '\*'
  p = re.compile(' \* ?(BPF\w+)$')
/home/srikar/linux.git/scripts/bpf_doc.py:121: SyntaxWarning: invalid escape sequence '\*'
  end_re = re.compile(' \* ?NOTES$')
/home/srikar/linux.git/scripts/bpf_doc.py:136: SyntaxWarning: invalid escape sequence '\*'
  p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
/home/srikar/linux.git/scripts/bpf_doc.py:144: SyntaxWarning: invalid escape sequence '\*'
  p = re.compile(' \* ?(?:\t| {5,8})Description$')
/home/srikar/linux.git/scripts/bpf_doc.py:157: SyntaxWarning: invalid escape sequence '\*'
  p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
/home/srikar/linux.git/scripts/bpf_doc.py:170: SyntaxWarning: invalid escape sequence '\*'
  p = re.compile(' \* ?(?:\t| {5,8})Return$')
/home/srikar/linux.git/scripts/bpf_doc.py:183: SyntaxWarning: invalid escape sequence '\*'
  p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
/home/srikar/linux.git/scripts/bpf_doc.py:222: SyntaxWarning: invalid escape sequence '\s'
  bpf_p = re.compile('\s*(BPF\w+)+')
/home/srikar/linux.git/scripts/bpf_doc.py:227: SyntaxWarning: invalid escape sequence '\s'
  assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)')
/home/srikar/linux.git/scripts/bpf_doc.py:242: SyntaxWarning: invalid escape sequence '\w'
  self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)
/home/srikar/linux.git/scripts/bpf_doc.py:266: SyntaxWarning: invalid escape sequence '\s'
  p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
/home/srikar/linux.git/scripts/bpf_doc.py:281: SyntaxWarning: invalid escape sequence '\('
  self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str)
/home/srikar/linux.git/scripts/bpf_doc.py:428: SyntaxWarning: invalid escape sequence '\*'
  '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
/home/srikar/linux.git/scripts/bpf_doc.py:499: SyntaxWarning: invalid escape sequence '\ '
  footer = '''
/home/srikar/linux.git/scripts/bpf_doc.py:601: SyntaxWarning: invalid escape sequence '\ '
  one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*'))

However I am not seeing this when using python 3.10 (Python 3.10.12) and
python 3.11 (Python 3.11.4). Note this is just a warning and Kernel build does
complete.

-- 
Thanks and Regards
Srikar Dronamraju

^ permalink raw reply	[flat|nested] 8+ messages in thread
* [PATCH] Fix invalid escape sequence warnings
  2023-08-11  8:47 Warning when compiling with python3.12 Srikar Dronamraju
@ 2023-08-16 12:21 ` Vishal Chourasia
  2023-08-22 23:30   ` Andrii Nakryiko
  2023-08-23  8:44   ` [PATCH] " Srikar Dronamraju
  0 siblings, 2 replies; 8+ messages in thread
From: Vishal Chourasia @ 2023-08-16 12:21 UTC (permalink / raw)
  To: srikar
  Cc: andrii, ast, bpf, daniel, davem, haoluo, hawk, john.fastabend,
	jolsa, kpsingh, kuba, linux-kernel, martin.lau, netdev, sachinp,
	sdf, song, yhs, vishalc

The Python script `bpf_doc.py` uses regular expressions with
backslashes in string literals, which results in SyntaxWarnings
during its execution.

This patch addresses these warnings by converting relevant string
literals to raw strings, which interpret backslashes as literal
characters. This ensures that the regular expressions are parsed
correctly without causing any warnings.

Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>

---
 scripts/bpf_doc.py | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/scripts/bpf_doc.py b/scripts/bpf_doc.py
index eaae2ce78381..dfd819c952b2 100755
--- a/scripts/bpf_doc.py
+++ b/scripts/bpf_doc.py
@@ -59,9 +59,9 @@ class Helper(APIElement):
         Break down helper function protocol into smaller chunks: return type,
         name, distincts arguments.
         """
-        arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$')
+        arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$')
         res = {}
-        proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
+        proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
 
         capture = proto_re.match(self.proto)
         res['ret_type'] = capture.group(1)
@@ -114,11 +114,11 @@ class HeaderParser(object):
         return Helper(proto=proto, desc=desc, ret=ret)
 
     def parse_symbol(self):
-        p = re.compile(' \* ?(BPF\w+)$')
+        p = re.compile(r' \* ?(BPF\w+)$')
         capture = p.match(self.line)
         if not capture:
             raise NoSyscallCommandFound
-        end_re = re.compile(' \* ?NOTES$')
+        end_re = re.compile(r' \* ?NOTES$')
         end = end_re.match(self.line)
         if end:
             raise NoSyscallCommandFound
@@ -133,7 +133,7 @@ class HeaderParser(object):
         #   - Same as above, with "const" and/or "struct" in front of type
         #   - "..." (undefined number of arguments, for bpf_trace_printk())
         # There is at least one term ("void"), and at most five arguments.
-        p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
+        p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
         capture = p.match(self.line)
         if not capture:
             raise NoHelperFound
@@ -141,7 +141,7 @@ class HeaderParser(object):
         return capture.group(1)
 
     def parse_desc(self, proto):
-        p = re.compile(' \* ?(?:\t| {5,8})Description$')
+        p = re.compile(r' \* ?(?:\t| {5,8})Description$')
         capture = p.match(self.line)
         if not capture:
             raise Exception("No description section found for " + proto)
@@ -154,7 +154,7 @@ class HeaderParser(object):
             if self.line == ' *\n':
                 desc += '\n'
             else:
-                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
+                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
                 capture = p.match(self.line)
                 if capture:
                     desc_present = True
@@ -167,7 +167,7 @@ class HeaderParser(object):
         return desc
 
     def parse_ret(self, proto):
-        p = re.compile(' \* ?(?:\t| {5,8})Return$')
+        p = re.compile(r' \* ?(?:\t| {5,8})Return$')
         capture = p.match(self.line)
         if not capture:
             raise Exception("No return section found for " + proto)
@@ -180,7 +180,7 @@ class HeaderParser(object):
             if self.line == ' *\n':
                 ret += '\n'
             else:
-                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
+                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
                 capture = p.match(self.line)
                 if capture:
                     ret_present = True
@@ -219,12 +219,12 @@ class HeaderParser(object):
         self.seek_to('enum bpf_cmd {',
                      'Could not find start of bpf_cmd enum', 0)
         # Searches for either one or more BPF\w+ enums
-        bpf_p = re.compile('\s*(BPF\w+)+')
+        bpf_p = re.compile(r'\s*(BPF\w+)+')
         # Searches for an enum entry assigned to another entry,
         # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is
         # not documented hence should be skipped in check to
         # determine if the right number of syscalls are documented
-        assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)')
+        assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)')
         bpf_cmd_str = ''
         while True:
             capture = assign_p.match(self.line)
@@ -239,7 +239,7 @@ class HeaderParser(object):
                 break
             self.line = self.reader.readline()
         # Find the number of occurences of BPF\w+
-        self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)
+        self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str)
 
     def parse_desc_helpers(self):
         self.seek_to(helpersDocStart,
@@ -263,7 +263,7 @@ class HeaderParser(object):
         self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)',
                      'Could not find start of eBPF helper definition list')
         # Searches for one FN(\w+) define or a backslash for newline
-        p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
+        p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
         fn_defines_str = ''
         i = 0
         while True:
@@ -278,7 +278,7 @@ class HeaderParser(object):
                 break
             self.line = self.reader.readline()
         # Find the number of occurences of FN(\w+)
-        self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str)
+        self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str)
 
     def validate_helpers(self):
         last_helper = ''
@@ -425,7 +425,7 @@ class PrinterRST(Printer):
         try:
             cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch',
                    '-L',
-                   '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
+                   r'/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
             date = subprocess.run(cmd, cwd=linuxRoot,
                                   capture_output=True, check=True)
             return date.stdout.decode().rstrip()
@@ -496,7 +496,7 @@ HELPERS
                             date=lastUpdate))
 
     def print_footer(self):
-        footer = '''
+        footer = r'''
 EXAMPLES
 ========
 
@@ -598,7 +598,7 @@ SEE ALSO
             one_arg = '{}{}'.format(comma, a['type'])
             if a['name']:
                 if a['star']:
-                    one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*'))
+                    one_arg += r' {}**\ '.format(a['star'].replace('*', '\\*'))
                 else:
                     one_arg += '** '
                 one_arg += '*{}*\\ **'.format(a['name'])
-- 
2.41.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread
* Re: [PATCH] Fix invalid escape sequence warnings
  2023-08-16 12:21 ` [PATCH] Fix invalid escape sequence warnings Vishal Chourasia
@ 2023-08-22 23:30   ` Andrii Nakryiko
  2023-08-29  7:43     ` Vishal Chourasia
  2023-08-23  8:44   ` [PATCH] " Srikar Dronamraju
  1 sibling, 1 reply; 8+ messages in thread
From: Andrii Nakryiko @ 2023-08-22 23:30 UTC (permalink / raw)
  To: Vishal Chourasia, Quentin Monnet
  Cc: srikar, andrii, ast, bpf, daniel, davem, haoluo, hawk,
	john.fastabend, jolsa, kpsingh, kuba, linux-kernel, martin.lau,
	netdev, sachinp, sdf, song, yhs

On Wed, Aug 16, 2023 at 5:22 AM Vishal Chourasia <vishalc@linux.ibm.com> wrote:
>
> The Python script `bpf_doc.py` uses regular expressions with
> backslashes in string literals, which results in SyntaxWarnings
> during its execution.
>
> This patch addresses these warnings by converting relevant string
> literals to raw strings, which interpret backslashes as literal
> characters. This ensures that the regular expressions are parsed
> correctly without causing any warnings.
>
> Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
> Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>
> ---
>  scripts/bpf_doc.py | 34 +++++++++++++++++-----------------
>  1 file changed, 17 insertions(+), 17 deletions(-)
>
> diff --git a/scripts/bpf_doc.py b/scripts/bpf_doc.py
> index eaae2ce78381..dfd819c952b2 100755
> --- a/scripts/bpf_doc.py
> +++ b/scripts/bpf_doc.py
> @@ -59,9 +59,9 @@ class Helper(APIElement):
>          Break down helper function protocol into smaller chunks: return type,
>          name, distincts arguments.
>          """
> -        arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$')
> +        arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$')
>          res = {}
> -        proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
> +        proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
>
>          capture = proto_re.match(self.proto)
>          res['ret_type'] = capture.group(1)
> @@ -114,11 +114,11 @@ class HeaderParser(object):
>          return Helper(proto=proto, desc=desc, ret=ret)
>
>      def parse_symbol(self):
> -        p = re.compile(' \* ?(BPF\w+)$')
> +        p = re.compile(r' \* ?(BPF\w+)$')
>          capture = p.match(self.line)
>          if not capture:
>              raise NoSyscallCommandFound
> -        end_re = re.compile(' \* ?NOTES$')
> +        end_re = re.compile(r' \* ?NOTES$')
>          end = end_re.match(self.line)
>          if end:
>              raise NoSyscallCommandFound
> @@ -133,7 +133,7 @@ class HeaderParser(object):
>          #   - Same as above, with "const" and/or "struct" in front of type
>          #   - "..." (undefined number of arguments, for bpf_trace_printk())
>          # There is at least one term ("void"), and at most five arguments.
> -        p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
> +        p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
>          capture = p.match(self.line)
>          if not capture:
>              raise NoHelperFound
> @@ -141,7 +141,7 @@ class HeaderParser(object):
>          return capture.group(1)
>
>      def parse_desc(self, proto):
> -        p = re.compile(' \* ?(?:\t| {5,8})Description$')
> +        p = re.compile(r' \* ?(?:\t| {5,8})Description$')
>          capture = p.match(self.line)
>          if not capture:
>              raise Exception("No description section found for " + proto)
> @@ -154,7 +154,7 @@ class HeaderParser(object):
>              if self.line == ' *\n':
>                  desc += '\n'
>              else:
> -                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
> +                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
>                  capture = p.match(self.line)
>                  if capture:
>                      desc_present = True
> @@ -167,7 +167,7 @@ class HeaderParser(object):
>          return desc
>
>      def parse_ret(self, proto):
> -        p = re.compile(' \* ?(?:\t| {5,8})Return$')
> +        p = re.compile(r' \* ?(?:\t| {5,8})Return$')
>          capture = p.match(self.line)
>          if not capture:
>              raise Exception("No return section found for " + proto)
> @@ -180,7 +180,7 @@ class HeaderParser(object):
>              if self.line == ' *\n':
>                  ret += '\n'
>              else:
> -                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
> +                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
>                  capture = p.match(self.line)
>                  if capture:
>                      ret_present = True
> @@ -219,12 +219,12 @@ class HeaderParser(object):
>          self.seek_to('enum bpf_cmd {',
>                       'Could not find start of bpf_cmd enum', 0)
>          # Searches for either one or more BPF\w+ enums
> -        bpf_p = re.compile('\s*(BPF\w+)+')
> +        bpf_p = re.compile(r'\s*(BPF\w+)+')
>          # Searches for an enum entry assigned to another entry,
>          # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is
>          # not documented hence should be skipped in check to
>          # determine if the right number of syscalls are documented
> -        assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)')
> +        assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)')
>          bpf_cmd_str = ''
>          while True:
>              capture = assign_p.match(self.line)
> @@ -239,7 +239,7 @@ class HeaderParser(object):
>                  break
>              self.line = self.reader.readline()
>          # Find the number of occurences of BPF\w+
> -        self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)
> +        self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str)
>
>      def parse_desc_helpers(self):
>          self.seek_to(helpersDocStart,
> @@ -263,7 +263,7 @@ class HeaderParser(object):
>          self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)',
>                       'Could not find start of eBPF helper definition list')
>          # Searches for one FN(\w+) define or a backslash for newline
> -        p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
> +        p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
>          fn_defines_str = ''
>          i = 0
>          while True:
> @@ -278,7 +278,7 @@ class HeaderParser(object):
>                  break
>              self.line = self.reader.readline()
>          # Find the number of occurences of FN(\w+)
> -        self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str)
> +        self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str)
>
>      def validate_helpers(self):
>          last_helper = ''
> @@ -425,7 +425,7 @@ class PrinterRST(Printer):
>          try:
>              cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch',
>                     '-L',
> -                   '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
> +                   r'/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]

this one is not a regex, do we still need to change it?

>              date = subprocess.run(cmd, cwd=linuxRoot,
>                                    capture_output=True, check=True)
>              return date.stdout.decode().rstrip()
> @@ -496,7 +496,7 @@ HELPERS
>                              date=lastUpdate))
>
>      def print_footer(self):
> -        footer = '''
> +        footer = r'''

same here, not a regex string

>  EXAMPLES
>  ========
>
> @@ -598,7 +598,7 @@ SEE ALSO
>              one_arg = '{}{}'.format(comma, a['type'])
>              if a['name']:
>                  if a['star']:
> -                    one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*'))
> +                    one_arg += r' {}**\ '.format(a['star'].replace('*', '\\*'))

and this one as well?

>                  else:
>                      one_arg += '** '
>                  one_arg += '*{}*\\ **'.format(a['name'])
> --
> 2.41.0
>

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [PATCH] Fix invalid escape sequence warnings
  2023-08-16 12:21 ` [PATCH] Fix invalid escape sequence warnings Vishal Chourasia
  2023-08-22 23:30   ` Andrii Nakryiko
@ 2023-08-23  8:44   ` Srikar Dronamraju
  1 sibling, 0 replies; 8+ messages in thread
From: Srikar Dronamraju @ 2023-08-23  8:44 UTC (permalink / raw)
  To: Vishal Chourasia
  Cc: andrii, ast, bpf, daniel, davem, haoluo, hawk, john.fastabend,
	jolsa, kpsingh, kuba, linux-kernel, martin.lau, netdev, sachinp,
	sdf, song, yhs

* Vishal Chourasia <vishalc@linux.ibm.com> [2023-08-16 17:51:33]:

> The Python script `bpf_doc.py` uses regular expressions with
> backslashes in string literals, which results in SyntaxWarnings
> during its execution.
> 
> This patch addresses these warnings by converting relevant string
> literals to raw strings, which interpret backslashes as literal
> characters. This ensures that the regular expressions are parsed
> correctly without causing any warnings.
> 
> Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
> Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>

Thanks, Works for me
Tested-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>

-- 
Thanks and Regards
Srikar Dronamraju

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [PATCH] Fix invalid escape sequence warnings
  2023-08-22 23:30   ` Andrii Nakryiko
@ 2023-08-29  7:43     ` Vishal Chourasia
  2023-08-29  7:49       ` [PATCH v2] " Vishal Chourasia
  0 siblings, 1 reply; 8+ messages in thread
From: Vishal Chourasia @ 2023-08-29  7:43 UTC (permalink / raw)
  To: Andrii Nakryiko, Quentin Monnet
  Cc: srikar, andrii, ast, bpf, daniel, davem, haoluo, hawk,
	john.fastabend, jolsa, kpsingh, kuba, linux-kernel, martin.lau,
	netdev, sachinp, sdf, song, yhs

On 8/23/23 05:00, Andrii Nakryiko wrote:
> On Wed, Aug 16, 2023 at 5:22 AM Vishal Chourasia <vishalc@linux.ibm.com> wrote:
>>
>> The Python script `bpf_doc.py` uses regular expressions with
>> backslashes in string literals, which results in SyntaxWarnings
>> during its execution.
>>
>> This patch addresses these warnings by converting relevant string
>> literals to raw strings, which interpret backslashes as literal
>> characters. This ensures that the regular expressions are parsed
>> correctly without causing any warnings.
>>
>> Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
>> Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>>
>> ---
>>  scripts/bpf_doc.py | 34 +++++++++++++++++-----------------
>>  1 file changed, 17 insertions(+), 17 deletions(-)
>>
>> diff --git a/scripts/bpf_doc.py b/scripts/bpf_doc.py
>> index eaae2ce78381..dfd819c952b2 100755
>> --- a/scripts/bpf_doc.py
>> +++ b/scripts/bpf_doc.py
>> @@ -59,9 +59,9 @@ class Helper(APIElement):
>>          Break down helper function protocol into smaller chunks: return type,
>>          name, distincts arguments.
>>          """
>> -        arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$')
>> +        arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$')
>>          res = {}
>> -        proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
>> +        proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
>>
>>          capture = proto_re.match(self.proto)
>>          res['ret_type'] = capture.group(1)
>> @@ -114,11 +114,11 @@ class HeaderParser(object):
>>          return Helper(proto=proto, desc=desc, ret=ret)
>>
>>      def parse_symbol(self):
>> -        p = re.compile(' \* ?(BPF\w+)$')
>> +        p = re.compile(r' \* ?(BPF\w+)$')
>>          capture = p.match(self.line)
>>          if not capture:
>>              raise NoSyscallCommandFound
>> -        end_re = re.compile(' \* ?NOTES$')
>> +        end_re = re.compile(r' \* ?NOTES$')
>>          end = end_re.match(self.line)
>>          if end:
>>              raise NoSyscallCommandFound
>> @@ -133,7 +133,7 @@ class HeaderParser(object):
>>          #   - Same as above, with "const" and/or "struct" in front of type
>>          #   - "..." (undefined number of arguments, for bpf_trace_printk())
>>          # There is at least one term ("void"), and at most five arguments.
>> -        p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
>> +        p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
>>          capture = p.match(self.line)
>>          if not capture:
>>              raise NoHelperFound
>> @@ -141,7 +141,7 @@ class HeaderParser(object):
>>          return capture.group(1)
>>
>>      def parse_desc(self, proto):
>> -        p = re.compile(' \* ?(?:\t| {5,8})Description$')
>> +        p = re.compile(r' \* ?(?:\t| {5,8})Description$')
>>          capture = p.match(self.line)
>>          if not capture:
>>              raise Exception("No description section found for " + proto)
>> @@ -154,7 +154,7 @@ class HeaderParser(object):
>>              if self.line == ' *\n':
>>                  desc += '\n'
>>              else:
>> -                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
>> +                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
>>                  capture = p.match(self.line)
>>                  if capture:
>>                      desc_present = True
>> @@ -167,7 +167,7 @@ class HeaderParser(object):
>>          return desc
>>
>>      def parse_ret(self, proto):
>> -        p = re.compile(' \* ?(?:\t| {5,8})Return$')
>> +        p = re.compile(r' \* ?(?:\t| {5,8})Return$')
>>          capture = p.match(self.line)
>>          if not capture:
>>              raise Exception("No return section found for " + proto)
>> @@ -180,7 +180,7 @@ class HeaderParser(object):
>>              if self.line == ' *\n':
>>                  ret += '\n'
>>              else:
>> -                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
>> +                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
>>                  capture = p.match(self.line)
>>                  if capture:
>>                      ret_present = True
>> @@ -219,12 +219,12 @@ class HeaderParser(object):
>>          self.seek_to('enum bpf_cmd {',
>>                       'Could not find start of bpf_cmd enum', 0)
>>          # Searches for either one or more BPF\w+ enums
>> -        bpf_p = re.compile('\s*(BPF\w+)+')
>> +        bpf_p = re.compile(r'\s*(BPF\w+)+')
>>          # Searches for an enum entry assigned to another entry,
>>          # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is
>>          # not documented hence should be skipped in check to
>>          # determine if the right number of syscalls are documented
>> -        assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)')
>> +        assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)')
>>          bpf_cmd_str = ''
>>          while True:
>>              capture = assign_p.match(self.line)
>> @@ -239,7 +239,7 @@ class HeaderParser(object):
>>                  break
>>              self.line = self.reader.readline()
>>          # Find the number of occurences of BPF\w+
>> -        self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)
>> +        self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str)
>>
>>      def parse_desc_helpers(self):
>>          self.seek_to(helpersDocStart,
>> @@ -263,7 +263,7 @@ class HeaderParser(object):
>>          self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)',
>>                       'Could not find start of eBPF helper definition list')
>>          # Searches for one FN(\w+) define or a backslash for newline
>> -        p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
>> +        p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
>>          fn_defines_str = ''
>>          i = 0
>>          while True:
>> @@ -278,7 +278,7 @@ class HeaderParser(object):
>>                  break
>>              self.line = self.reader.readline()
>>          # Find the number of occurences of FN(\w+)
>> -        self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str)
>> +        self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str)
>>
>>      def validate_helpers(self):
>>          last_helper = ''
>> @@ -425,7 +425,7 @@ class PrinterRST(Printer):
>>          try:
>>              cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch',
>>                     '-L',
>> -                   '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
>> +                   r'/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
> 
> this one is not a regex, do we still need to change it?
Indeed, it's essential to modify this aspect, especially since we're
encountering warnings when compiling the kernel with Python 3.12.

The choice between using an 'r'-prefixed string or manually escaping
each backslash is largely a matter of preference. It's worth noting that
the 'r' prefix is not only commonly employed for regular expressions but
also in contexts where backslashes should be treated as literal
characters rather than escape sequences.

Should I send another patch escaping backslash for non regex string
literals?
>>              date = subprocess.run(cmd, cwd=linuxRoot,
>>                                    capture_output=True, check=True)
>>              return date.stdout.decode().rstrip()
>> @@ -496,7 +496,7 @@ HELPERS
>>                              date=lastUpdate))
>>
>>      def print_footer(self):
>> -        footer = '''
>> +        footer = r'''
> 
> same here, not a regex string
> 
>>  EXAMPLES
>>  ========
>>
>> @@ -598,7 +598,7 @@ SEE ALSO
>>              one_arg = '{}{}'.format(comma, a['type'])
>>              if a['name']:
>>                  if a['star']:
>> -                    one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*'))
>> +                    one_arg += r' {}**\ '.format(a['star'].replace('*', '\\*'))
> 
> and this one as well?
> 
>>                  else:
>>                      one_arg += '** '
>>                  one_arg += '*{}*\\ **'.format(a['name'])
>> --
>> 2.41.0
>>


^ permalink raw reply	[flat|nested] 8+ messages in thread
* [PATCH v2] Fix invalid escape sequence warnings
  2023-08-29  7:43     ` Vishal Chourasia
@ 2023-08-29  7:49       ` Vishal Chourasia
  2023-08-30 15:21         ` Quentin Monnet
  2023-08-31 12:00         ` patchwork-bot+netdevbpf
  0 siblings, 2 replies; 8+ messages in thread
From: Vishal Chourasia @ 2023-08-29  7:49 UTC (permalink / raw)
  To: vishalc
  Cc: andrii.nakryiko, andrii, ast, bpf, daniel, davem, haoluo, hawk,
	john.fastabend, jolsa, kpsingh, kuba, linux-kernel, martin.lau,
	netdev, quentin, sachinp, sdf, song, srikar, yhs

The script bpf_doc.py generates multiple SyntaxWarnings related to invalid
escape sequences when executed with Python 3.12. These warnings do not appear in
Python 3.10 and 3.11 and do not affect the kernel build, which completes
successfully.

This patch resolves these SyntaxWarnings by converting the relevant string
literals to raw strings or by escaping backslashes. This ensures that
backslashes are interpreted as literal characters, eliminating the warnings.

Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
 scripts/bpf_doc.py | 56 +++++++++++++++++++++++-----------------------
 1 file changed, 28 insertions(+), 28 deletions(-)

diff --git a/scripts/bpf_doc.py b/scripts/bpf_doc.py
index eaae2ce78..61b7ddded 100755
--- a/scripts/bpf_doc.py
+++ b/scripts/bpf_doc.py
@@ -59,9 +59,9 @@ class Helper(APIElement):
         Break down helper function protocol into smaller chunks: return type,
         name, distincts arguments.
         """
-        arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$')
+        arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$')
         res = {}
-        proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
+        proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
 
         capture = proto_re.match(self.proto)
         res['ret_type'] = capture.group(1)
@@ -114,11 +114,11 @@ class HeaderParser(object):
         return Helper(proto=proto, desc=desc, ret=ret)
 
     def parse_symbol(self):
-        p = re.compile(' \* ?(BPF\w+)$')
+        p = re.compile(r' \* ?(BPF\w+)$')
         capture = p.match(self.line)
         if not capture:
             raise NoSyscallCommandFound
-        end_re = re.compile(' \* ?NOTES$')
+        end_re = re.compile(r' \* ?NOTES$')
         end = end_re.match(self.line)
         if end:
             raise NoSyscallCommandFound
@@ -133,7 +133,7 @@ class HeaderParser(object):
         #   - Same as above, with "const" and/or "struct" in front of type
         #   - "..." (undefined number of arguments, for bpf_trace_printk())
         # There is at least one term ("void"), and at most five arguments.
-        p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
+        p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
         capture = p.match(self.line)
         if not capture:
             raise NoHelperFound
@@ -141,7 +141,7 @@ class HeaderParser(object):
         return capture.group(1)
 
     def parse_desc(self, proto):
-        p = re.compile(' \* ?(?:\t| {5,8})Description$')
+        p = re.compile(r' \* ?(?:\t| {5,8})Description$')
         capture = p.match(self.line)
         if not capture:
             raise Exception("No description section found for " + proto)
@@ -154,7 +154,7 @@ class HeaderParser(object):
             if self.line == ' *\n':
                 desc += '\n'
             else:
-                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
+                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
                 capture = p.match(self.line)
                 if capture:
                     desc_present = True
@@ -167,7 +167,7 @@ class HeaderParser(object):
         return desc
 
     def parse_ret(self, proto):
-        p = re.compile(' \* ?(?:\t| {5,8})Return$')
+        p = re.compile(r' \* ?(?:\t| {5,8})Return$')
         capture = p.match(self.line)
         if not capture:
             raise Exception("No return section found for " + proto)
@@ -180,7 +180,7 @@ class HeaderParser(object):
             if self.line == ' *\n':
                 ret += '\n'
             else:
-                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
+                p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
                 capture = p.match(self.line)
                 if capture:
                     ret_present = True
@@ -219,12 +219,12 @@ class HeaderParser(object):
         self.seek_to('enum bpf_cmd {',
                      'Could not find start of bpf_cmd enum', 0)
         # Searches for either one or more BPF\w+ enums
-        bpf_p = re.compile('\s*(BPF\w+)+')
+        bpf_p = re.compile(r'\s*(BPF\w+)+')
         # Searches for an enum entry assigned to another entry,
         # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is
         # not documented hence should be skipped in check to
         # determine if the right number of syscalls are documented
-        assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)')
+        assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)')
         bpf_cmd_str = ''
         while True:
             capture = assign_p.match(self.line)
@@ -239,7 +239,7 @@ class HeaderParser(object):
                 break
             self.line = self.reader.readline()
         # Find the number of occurences of BPF\w+
-        self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)
+        self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str)
 
     def parse_desc_helpers(self):
         self.seek_to(helpersDocStart,
@@ -263,7 +263,7 @@ class HeaderParser(object):
         self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)',
                      'Could not find start of eBPF helper definition list')
         # Searches for one FN(\w+) define or a backslash for newline
-        p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
+        p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
         fn_defines_str = ''
         i = 0
         while True:
@@ -278,7 +278,7 @@ class HeaderParser(object):
                 break
             self.line = self.reader.readline()
         # Find the number of occurences of FN(\w+)
-        self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str)
+        self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str)
 
     def validate_helpers(self):
         last_helper = ''
@@ -425,7 +425,7 @@ class PrinterRST(Printer):
         try:
             cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch',
                    '-L',
-                   '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
+                   '/{}/,/\\*\\//:include/uapi/linux/bpf.h'.format(delimiter)]
             date = subprocess.run(cmd, cwd=linuxRoot,
                                   capture_output=True, check=True)
             return date.stdout.decode().rstrip()
@@ -516,7 +516,7 @@ as "Dual BSD/GPL", may be used). Some helper functions are only accessible to
 programs that are compatible with the GNU Privacy License (GPL).
 
 In order to use such helpers, the eBPF program must be loaded with the correct
-license string passed (via **attr**) to the **bpf**\ () system call, and this
+license string passed (via **attr**) to the **bpf**\\ () system call, and this
 generally translates into the C source code of the program containing a line
 similar to the following:
 
@@ -550,7 +550,7 @@ may be interested in:
 * The bpftool utility can be used to probe the availability of helper functions
   on the system (as well as supported program and map types, and a number of
   other parameters). To do so, run **bpftool feature probe** (see
-  **bpftool-feature**\ (8) for details). Add the **unprivileged** keyword to
+  **bpftool-feature**\\ (8) for details). Add the **unprivileged** keyword to
   list features available to unprivileged users.
 
 Compatibility between helper functions and program types can generally be found
@@ -562,23 +562,23 @@ other functions, themselves allowing access to additional helpers. The
 requirement for GPL license is also in those **struct bpf_func_proto**.
 
 Compatibility between helper functions and map types can be found in the
-**check_map_func_compatibility**\ () function in file *kernel/bpf/verifier.c*.
+**check_map_func_compatibility**\\ () function in file *kernel/bpf/verifier.c*.
 
 Helper functions that invalidate the checks on **data** and **data_end**
 pointers for network processing are listed in function
-**bpf_helper_changes_pkt_data**\ () in file *net/core/filter.c*.
+**bpf_helper_changes_pkt_data**\\ () in file *net/core/filter.c*.
 
 SEE ALSO
 ========
 
-**bpf**\ (2),
-**bpftool**\ (8),
-**cgroups**\ (7),
-**ip**\ (8),
-**perf_event_open**\ (2),
-**sendmsg**\ (2),
-**socket**\ (7),
-**tc-bpf**\ (8)'''
+**bpf**\\ (2),
+**bpftool**\\ (8),
+**cgroups**\\ (7),
+**ip**\\ (8),
+**perf_event_open**\\ (2),
+**sendmsg**\\ (2),
+**socket**\\ (7),
+**tc-bpf**\\ (8)'''
         print(footer)
 
     def print_proto(self, helper):
@@ -598,7 +598,7 @@ SEE ALSO
             one_arg = '{}{}'.format(comma, a['type'])
             if a['name']:
                 if a['star']:
-                    one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*'))
+                    one_arg += ' {}**\\ '.format(a['star'].replace('*', '\\*'))
                 else:
                     one_arg += '** '
                 one_arg += '*{}*\\ **'.format(a['name'])
-- 
2.41.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread
* Re: [PATCH v2] Fix invalid escape sequence warnings
  2023-08-29  7:49       ` [PATCH v2] " Vishal Chourasia
@ 2023-08-30 15:21         ` Quentin Monnet
  2023-08-31 12:00         ` patchwork-bot+netdevbpf
  1 sibling, 0 replies; 8+ messages in thread
From: Quentin Monnet @ 2023-08-30 15:21 UTC (permalink / raw)
  To: Vishal Chourasia
  Cc: andrii.nakryiko, andrii, ast, bpf, daniel, davem, haoluo, hawk,
	john.fastabend, jolsa, kpsingh, kuba, linux-kernel, martin.lau,
	netdev, sachinp, sdf, song, srikar, yhs

On 29/08/2023 08:49, Vishal Chourasia wrote:
> The script bpf_doc.py generates multiple SyntaxWarnings related to invalid
> escape sequences when executed with Python 3.12. These warnings do not appear in
> Python 3.10 and 3.11 and do not affect the kernel build, which completes
> successfully.
> 
> This patch resolves these SyntaxWarnings by converting the relevant string
> literals to raw strings or by escaping backslashes. This ensures that
> backslashes are interpreted as literal characters, eliminating the warnings.
> 
> Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
> Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>

Thanks! I observed that this patch fixes warnings reported by pyright in
my editor. I've also validated that the generated files (helpers man
page, syscall man page, bpf_helper_defs.h) remain unchanged.

Tested-by: Quentin Monnet <quentin@isovalent.com>


^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [PATCH v2] Fix invalid escape sequence warnings
  2023-08-29  7:49       ` [PATCH v2] " Vishal Chourasia
  2023-08-30 15:21         ` Quentin Monnet
@ 2023-08-31 12:00         ` patchwork-bot+netdevbpf
  1 sibling, 0 replies; 8+ messages in thread
From: patchwork-bot+netdevbpf @ 2023-08-31 12:00 UTC (permalink / raw)
  To: Vishal Chourasia
  Cc: andrii.nakryiko, andrii, ast, bpf, daniel, davem, haoluo, hawk,
	john.fastabend, jolsa, kpsingh, kuba, linux-kernel, martin.lau,
	netdev, quentin, sachinp, sdf, song, srikar, yhs

Hello:

This patch was applied to bpf/bpf.git (master)
by Daniel Borkmann <daniel@iogearbox.net>:

On Tue, 29 Aug 2023 13:19:31 +0530 you wrote:
> The script bpf_doc.py generates multiple SyntaxWarnings related to invalid
> escape sequences when executed with Python 3.12. These warnings do not appear in
> Python 3.10 and 3.11 and do not affect the kernel build, which completes
> successfully.
> 
> This patch resolves these SyntaxWarnings by converting the relevant string
> literals to raw strings or by escaping backslashes. This ensures that
> backslashes are interpreted as literal characters, eliminating the warnings.
> 
> [...]

Here is the summary with links:
  - [v2] Fix invalid escape sequence warnings
    https://git.kernel.org/bpf/bpf/c/121fd33bf2d9

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html



^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-08-31 12:00 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-11  8:47 Warning when compiling with python3.12 Srikar Dronamraju
2023-08-16 12:21 ` [PATCH] Fix invalid escape sequence warnings Vishal Chourasia
2023-08-22 23:30   ` Andrii Nakryiko
2023-08-29  7:43     ` Vishal Chourasia
2023-08-29  7:49       ` [PATCH v2] " Vishal Chourasia
2023-08-30 15:21         ` Quentin Monnet
2023-08-31 12:00         ` patchwork-bot+netdevbpf
2023-08-23  8:44   ` [PATCH] " Srikar Dronamraju

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).