linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/7] Fix some issues at get_abi.pl script
@ 2021-03-25 10:38 Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 1/7] scripts: get_abi.pl: better handle escape chars on what: Mauro Carvalho Chehab
                   ` (8 more replies)
  0 siblings, 9 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, linux-kernel, Greg Kroah-Hartman,
	John Hubbard, Minchan Kim, Stephen Rothwell

This series replace this patch:
	https://lore.kernel.org/linux-doc/20210324191722.08d352e4@coco.lan/T/#t

It turns that there were multiple bugs at the get_abi.pl code that
create cross-references.

Patches 1 to 6 fix those issues, and should apply cleanly on the top of
the docs tree (although I tested against next-20210323).

Patch 7 is optional, and independent from the other patches. It is meant
to be applied against akpm's tree.  It makes the description (IMHO) 
clearer, while producing cross references for the two mentioned symbols.

The fix patches are:

patch 1: fix some regexes that match the symbols that need to be
  escaped when parsing "What:". The same regex is also used when
  generating cross-references;

patch 2: makes the check for Documentation/ABI references more
  robust, as right now, it stops at the first occurrence;

patch 3: fix the parser for /sys/foo -> xref conversion. Basically,
  the logic that seeks for start and end boundaries were broken.
  The new logic is a way more robust.

patch 4: generate cross-references for /config/foo and other less
  common ABI occurrences;

patch 5 and 6: don't generate cross-references inside literal blocks.
  Right now, there are a couple of places that would otherwise
  generate references, producing a bad output.

Mauro Carvalho Chehab (7):
  scripts: get_abi.pl: better handle escape chars on what:
  get_abi.pl: seek for all occurrences for Documentation/ABI
  get_abi.pl: fix xref boundaries
  scripts: get_abi.pl: extend xref match to other types
  scripts: get_abi.pl: parse description line per line
  scripts: get_abi: ignore code blocks for cross-references
  ABI: sysfs-kernel-mm-cma: fix two cross-references

 Documentation/ABI/testing/sysfs-kernel-mm-cma |  8 +-
 scripts/get_abi.pl                            | 76 +++++++++++++------
 2 files changed, 57 insertions(+), 27 deletions(-)

-- 
2.30.2



^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH v2 1/7] scripts: get_abi.pl: better handle escape chars on what:
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
@ 2021-03-25 10:38 ` Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 2/7] get_abi.pl: seek for all occurrences for Documentation/ABI Mauro Carvalho Chehab
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, linux-kernel

The parser for the symbols defined on What: doesn't cover all
chars that need to be scaped, like '{' and '}'. Change the logic
to be more generic, and ensure that the same regex will be used
on both What: and when parsing the cross-references.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/get_abi.pl | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 92d9aa6cc4f5..a9348b9bdaa4 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -283,6 +283,7 @@ sub create_labels {
 
 # \b doesn't work well with paths. So, we need to define something else
 my $bondary = qr { (?<![\w\/\`\{])(?=[\w\/\`\{])|(?<=[\w\/\`\{])(?![\w\/\`\{]) }x;
+my $symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff]) }x;
 
 sub output_rest {
 	create_labels();
@@ -305,7 +306,6 @@ sub output_rest {
 		}
 
 		my $w = $what;
-		$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
 
 		if ($type ne "File") {
 			my $cur_part = $what;
@@ -329,6 +329,7 @@ sub output_rest {
 			my $len = 0;
 
 			foreach my $name (@names) {
+				$name =~ s/$symbols/\\$1/g;
 				$name = "**$name**";
 				$len = length($name) if (length($name) > $len);
 			}
@@ -395,7 +396,7 @@ sub output_rest {
 					if (defined($data{$s}) && defined($data{$s}->{label})) {
 						my $xref = $s;
 
-						$xref =~ s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
+						$xref =~ s/$symbols/\\$1/g;
 						$xref = ":ref:`$xref <" . $data{$s}->{label} . ">`";
 
 						$desc =~ s,$bondary$s$bondary,$xref,g;
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 2/7] get_abi.pl: seek for all occurrences for Documentation/ABI
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 1/7] scripts: get_abi.pl: better handle escape chars on what: Mauro Carvalho Chehab
@ 2021-03-25 10:38 ` Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 3/7] get_abi.pl: fix xref boundaries Mauro Carvalho Chehab
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, linux-kernel

Instead of retrieving just one match at most, ensure that the entire
description will be parsed.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/get_abi.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index a9348b9bdaa4..3c82cd188368 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -380,7 +380,7 @@ sub output_rest {
 
 				$desc =~ s,Documentation/(?!devicetree)(\S+)\.rst,:doc:`/$1`,g;
 
-				my @matches = $desc =~ m,Documentation/ABI/([\w\/\-]+),;
+				my @matches = $desc =~ m,Documentation/ABI/([\w\/\-]+),g;
 				foreach my $f (@matches) {
 					my $xref = $f;
 					my $path = $f;
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 3/7] get_abi.pl: fix xref boundaries
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 1/7] scripts: get_abi.pl: better handle escape chars on what: Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 2/7] get_abi.pl: seek for all occurrences for Documentation/ABI Mauro Carvalho Chehab
@ 2021-03-25 10:38 ` Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 4/7] scripts: get_abi.pl: extend xref match to other types Mauro Carvalho Chehab
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, linux-kernel

There are some issues with the regex that seeks for What:
cross references: basically, it is mis-identifying the start
and the end boundaries of the regex, which causes :ref: to
be inseerted for the wrong symbols at the wrong places.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/get_abi.pl | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 3c82cd188368..e87028257d1c 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -281,8 +281,11 @@ sub create_labels {
 # Outputs the book on ReST format
 #
 
-# \b doesn't work well with paths. So, we need to define something else
-my $bondary = qr { (?<![\w\/\`\{])(?=[\w\/\`\{])|(?<=[\w\/\`\{])(?![\w\/\`\{]) }x;
+# \b doesn't work well with paths. So, we need to define something else:
+# Boundaries are punct characters, spaces and end-of-line
+my $start = qr {(^|\s|\() }x;
+my $bondary = qr { ([,.:;\)\s]|\z) }x;
+my $xref_match = qr { $start(\/sys\/[^,.:;\)\s]+)$bondary }x;
 my $symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff]) }x;
 
 sub output_rest {
@@ -390,16 +393,18 @@ sub output_rest {
 					$desc =~ s,\bDocumentation/ABI/$f\b,:ref:`$xref`,g;
 				}
 
-				@matches = $desc =~ m,$bondary(/sys/[^\s\.\,\;\:\*\s\`\'\(\)]+)$bondary,;
+				# Seek for cross reference symbols like /sys/...
+				@matches = $desc =~ m/$xref_match/g;
 
 				foreach my $s (@matches) {
+					next if (!($s =~ m,/,));
 					if (defined($data{$s}) && defined($data{$s}->{label})) {
 						my $xref = $s;
 
 						$xref =~ s/$symbols/\\$1/g;
 						$xref = ":ref:`$xref <" . $data{$s}->{label} . ">`";
 
-						$desc =~ s,$bondary$s$bondary,$xref,g;
+						$desc =~ s,$start$s$bondary,$1$xref$2,g;
 					}
 				}
 
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 4/7] scripts: get_abi.pl: extend xref match to other types
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
                   ` (2 preceding siblings ...)
  2021-03-25 10:38 ` [PATCH v2 3/7] get_abi.pl: fix xref boundaries Mauro Carvalho Chehab
@ 2021-03-25 10:38 ` Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 5/7] scripts: get_abi.pl: parse description line per line Mauro Carvalho Chehab
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, linux-kernel

Currently, there are "What:" symbols for more than just
/sys.

Extend the regex to also cover configfs, /proc /dev and /kvd
symbols.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/get_abi.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index e87028257d1c..eb1a23103afa 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -285,7 +285,7 @@ sub create_labels {
 # Boundaries are punct characters, spaces and end-of-line
 my $start = qr {(^|\s|\() }x;
 my $bondary = qr { ([,.:;\)\s]|\z) }x;
-my $xref_match = qr { $start(\/sys\/[^,.:;\)\s]+)$bondary }x;
+my $xref_match = qr { $start(\/(sys|config|proc|dev|kvd)\/[^,.:;\)\s]+)$bondary }x;
 my $symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff]) }x;
 
 sub output_rest {
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 5/7] scripts: get_abi.pl: parse description line per line
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
                   ` (3 preceding siblings ...)
  2021-03-25 10:38 ` [PATCH v2 4/7] scripts: get_abi.pl: extend xref match to other types Mauro Carvalho Chehab
@ 2021-03-25 10:38 ` Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 6/7] scripts: get_abi: ignore code blocks for cross-references Mauro Carvalho Chehab
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, linux-kernel

Change the description parsing logic in rst mode in order
to parse it line per line.

The end result is the same, but doing line per line allows
to add some code to escape literal blocks when seeking for
cross-references.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/get_abi.pl | 47 ++++++++++++++++++++++++++--------------------
 1 file changed, 27 insertions(+), 20 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index eb1a23103afa..e5d1da492c1e 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -381,34 +381,41 @@ sub output_rest {
 
 				# Enrich text by creating cross-references
 
-				$desc =~ s,Documentation/(?!devicetree)(\S+)\.rst,:doc:`/$1`,g;
+				my $new_desc = "";
+				open(my $fh, "+<", \$desc);
+				while (my $d = <$fh>) {
+					$d =~ s,Documentation/(?!devicetree)(\S+)\.rst,:doc:`/$1`,g;
 
-				my @matches = $desc =~ m,Documentation/ABI/([\w\/\-]+),g;
-				foreach my $f (@matches) {
-					my $xref = $f;
-					my $path = $f;
-					$path =~ s,.*/(.*/.*),$1,;;
-					$path =~ s,[/\-],_,g;;
-					$xref .= " <abi_file_" . $path . ">";
-					$desc =~ s,\bDocumentation/ABI/$f\b,:ref:`$xref`,g;
-				}
+					my @matches = $d =~ m,Documentation/ABI/([\w\/\-]+),g;
+					foreach my $f (@matches) {
+						my $xref = $f;
+						my $path = $f;
+						$path =~ s,.*/(.*/.*),$1,;;
+						$path =~ s,[/\-],_,g;;
+						$xref .= " <abi_file_" . $path . ">";
+						$d =~ s,\bDocumentation/ABI/$f\b,:ref:`$xref`,g;
+					}
 
-				# Seek for cross reference symbols like /sys/...
-				@matches = $desc =~ m/$xref_match/g;
+					# Seek for cross reference symbols like /sys/...
+					@matches = $d =~ m/$xref_match/g;
 
-				foreach my $s (@matches) {
-					next if (!($s =~ m,/,));
-					if (defined($data{$s}) && defined($data{$s}->{label})) {
-						my $xref = $s;
+					foreach my $s (@matches) {
+						next if (!($s =~ m,/,));
+						if (defined($data{$s}) && defined($data{$s}->{label})) {
+							my $xref = $s;
 
-						$xref =~ s/$symbols/\\$1/g;
-						$xref = ":ref:`$xref <" . $data{$s}->{label} . ">`";
+							$xref =~ s/$symbols/\\$1/g;
+							$xref = ":ref:`$xref <" . $data{$s}->{label} . ">`";
 
-						$desc =~ s,$start$s$bondary,$1$xref$2,g;
+							$d =~ s,$start$s$bondary,$1$xref$2,g;
+						}
 					}
+					$new_desc .= $d;
 				}
+				close $fh;
+
 
-				print "$desc\n\n";
+				print "$new_desc\n\n";
 			} else {
 				$desc =~ s/^\s+//;
 
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 6/7] scripts: get_abi: ignore code blocks for cross-references
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
                   ` (4 preceding siblings ...)
  2021-03-25 10:38 ` [PATCH v2 5/7] scripts: get_abi.pl: parse description line per line Mauro Carvalho Chehab
@ 2021-03-25 10:38 ` Mauro Carvalho Chehab
  2021-03-25 10:38 ` [PATCH v2 7/7] ABI: sysfs-kernel-mm-cma: fix two cross-references Mauro Carvalho Chehab
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, linux-kernel

The script should not generate cross-references inside
literal blocks.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/get_abi.pl | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index e5d1da492c1e..d7aa82094296 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -382,8 +382,27 @@ sub output_rest {
 				# Enrich text by creating cross-references
 
 				my $new_desc = "";
+				my $init_indent = -1;
+				my $literal_indent = -1;
+
 				open(my $fh, "+<", \$desc);
 				while (my $d = <$fh>) {
+					my $indent = $d =~ m/^(\s+)/;
+					my $spaces = length($indent);
+					$init_indent = $indent if ($init_indent < 0);
+					if ($literal_indent >= 0) {
+						if ($spaces > $literal_indent) {
+							$new_desc .= $d;
+							next;
+						} else {
+							$literal_indent = -1;
+						}
+					} else {
+						if ($d =~ /()::$/ && !($d =~ /^\s*\.\./)) {
+							$literal_indent = $spaces;
+						}
+					}
+
 					$d =~ s,Documentation/(?!devicetree)(\S+)\.rst,:doc:`/$1`,g;
 
 					my @matches = $d =~ m,Documentation/ABI/([\w\/\-]+),g;
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 7/7] ABI: sysfs-kernel-mm-cma: fix two cross-references
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
                   ` (5 preceding siblings ...)
  2021-03-25 10:38 ` [PATCH v2 6/7] scripts: get_abi: ignore code blocks for cross-references Mauro Carvalho Chehab
@ 2021-03-25 10:38 ` Mauro Carvalho Chehab
  2021-04-01 21:56   ` John Hubbard
  2021-03-25 19:01 ` [PATCH v2 0/7] Fix some issues at get_abi.pl script Jonathan Corbet
  2021-03-31 20:02 ` Jonathan Corbet
  8 siblings, 1 reply; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 10:38 UTC (permalink / raw)
  To: Linux Doc Mailing List, Jonathan Corbet, Andrew Morton
  Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, John Hubbard,
	Minchan Kim, Stephen Rothwell, linux-kernel

Change the text in order to generate cross-references for
alloc_pages_success and alloc_pages_fail symbols.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/ABI/testing/sysfs-kernel-mm-cma | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-kernel-mm-cma b/Documentation/ABI/testing/sysfs-kernel-mm-cma
index 02b2bb60c296..86e261185561 100644
--- a/Documentation/ABI/testing/sysfs-kernel-mm-cma
+++ b/Documentation/ABI/testing/sysfs-kernel-mm-cma
@@ -5,12 +5,10 @@ Description:
 		/sys/kernel/mm/cma/ contains a subdirectory for each CMA
 		heap name (also sometimes called CMA areas).
 
-		Each CMA heap subdirectory (that is, each
-		/sys/kernel/mm/cma/<cma-heap-name> directory) contains the
-		following items:
+		Each CMA heap subdirectory contains the following items:
 
-			alloc_pages_success
-			alloc_pages_fail
+			- /sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_success
+			- /sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_fail
 
 What:		/sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_success
 Date:		Feb 2021
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH v2 0/7] Fix some issues at get_abi.pl script
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
                   ` (6 preceding siblings ...)
  2021-03-25 10:38 ` [PATCH v2 7/7] ABI: sysfs-kernel-mm-cma: fix two cross-references Mauro Carvalho Chehab
@ 2021-03-25 19:01 ` Jonathan Corbet
  2021-03-25 20:36   ` Mauro Carvalho Chehab
  2021-03-31 20:02 ` Jonathan Corbet
  8 siblings, 1 reply; 12+ messages in thread
From: Jonathan Corbet @ 2021-03-25 19:01 UTC (permalink / raw)
  To: Mauro Carvalho Chehab, Linux Doc Mailing List, Andrew Morton
  Cc: Mauro Carvalho Chehab, linux-kernel, Greg Kroah-Hartman,
	John Hubbard, Minchan Kim, Stephen Rothwell

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> This series replace this patch:
> 	https://lore.kernel.org/linux-doc/20210324191722.08d352e4@coco.lan/T/#t
>
> It turns that there were multiple bugs at the get_abi.pl code that
> create cross-references.
>
> Patches 1 to 6 fix those issues, and should apply cleanly on the top of
> the docs tree (although I tested against next-20210323).
>
> Patch 7 is optional, and independent from the other patches. It is meant
> to be applied against akpm's tree.  It makes the description (IMHO) 
> clearer, while producing cross references for the two mentioned symbols.

So perhaps this is the best solution to the problem, but I must confess
to not being entirely happy with it.  get_abi.pl is becoming another
unreadable perlpile like kerneldoc, and this makes it worse.  Doing RST
parsing there seems particularly unwelcome.

Should the cross-reference generation, it now occurs to me, be done in
the automarkup module instead?  Then there's no need to interpret RST,
and we'd get cross-references throughout the kernel docs rather than in
just the ABI stuff.  Am I completely out to lunch here?

Thanks,

jon

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2 0/7] Fix some issues at get_abi.pl script
  2021-03-25 19:01 ` [PATCH v2 0/7] Fix some issues at get_abi.pl script Jonathan Corbet
@ 2021-03-25 20:36   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 12+ messages in thread
From: Mauro Carvalho Chehab @ 2021-03-25 20:36 UTC (permalink / raw)
  To: Jonathan Corbet
  Cc: Linux Doc Mailing List, Andrew Morton, linux-kernel,
	Greg Kroah-Hartman, John Hubbard, Minchan Kim, Stephen Rothwell

Em Thu, 25 Mar 2021 13:01:14 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > This series replace this patch:
> > 	https://lore.kernel.org/linux-doc/20210324191722.08d352e4@coco.lan/T/#t
> >
> > It turns that there were multiple bugs at the get_abi.pl code that
> > create cross-references.
> >
> > Patches 1 to 6 fix those issues, and should apply cleanly on the top of
> > the docs tree (although I tested against next-20210323).
> >
> > Patch 7 is optional, and independent from the other patches. It is meant
> > to be applied against akpm's tree.  It makes the description (IMHO) 
> > clearer, while producing cross references for the two mentioned symbols.  
> 
> So perhaps this is the best solution to the problem, but I must confess
> to not being entirely happy with it.  get_abi.pl is becoming another
> unreadable perlpile like kerneldoc, and this makes it worse.  Doing RST
> parsing there seems particularly unwelcome.

Nah, it is not that complex ;-) 
It has 628 lines of code; kernel-doc has 2488. It is 1/4 the size of
kernel-doc :-D

Btw, 20 lines can be removed from it, if we drop support for --no-rst-source. 
I almost dropped it this time :-)

Ah, the entire code that outputs the rst description and does the
xref has just 34 lines (with 5 lines with comments and 5 blank lines).

It is not much, and some cleanups could be done to make it a little
shorter.

Most of the script complexity is actually at the parsing part. The
thing is that the ABI files are somewhat free-style: some files 
don't strictly follow the format, which requires some extra steps in
order to cope with some "soft" violations, as well as to report
problems when the violation is more severe.

> Should the cross-reference generation, it now occurs to me, be done in
> the automarkup module instead?  Then there's no need to interpret RST,
> and we'd get cross-references throughout the kernel docs rather than in
> just the ABI stuff.  Am I completely out to lunch here?

Yes, I guess we can move the Xref code to automarkup some day,
that is if we can ensure that get_abi.pl will run before automarkup.

There's currently an issue, though. We need first to get rid of those
duplicated symbols:

	Warning: /sys/bus/iio/devices/iio:deviceX/in_accel_x_calibbias is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-icm42600:0  Documentation/ABI/testing/sysfs-bus-iio:395
	Warning: /sys/bus/iio/devices/iio:deviceX/in_accel_y_calibbias is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-icm42600:1  Documentation/ABI/testing/sysfs-bus-iio:396
	Warning: /sys/bus/iio/devices/iio:deviceX/in_accel_z_calibbias is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-icm42600:2  Documentation/ABI/testing/sysfs-bus-iio:397
	Warning: /sys/bus/iio/devices/iio:deviceX/in_anglvel_x_calibbias is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-icm42600:3  Documentation/ABI/testing/sysfs-bus-iio:398
	Warning: /sys/bus/iio/devices/iio:deviceX/in_anglvel_y_calibbias is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-icm42600:4  Documentation/ABI/testing/sysfs-bus-iio:399
	Warning: /sys/bus/iio/devices/iio:deviceX/in_anglvel_z_calibbias is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-icm42600:5  Documentation/ABI/testing/sysfs-bus-iio:400
	Warning: /sys/bus/iio/devices/iio:deviceX/in_count0_preset is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-timer-stm32:100  Documentation/ABI/testing/sysfs-bus-iio-lptimer-stm32:0
	Warning: /sys/bus/iio/devices/iio:deviceX/in_count_quadrature_mode_available is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-counter-104-quad-8:2  Documentation/ABI/testing/sysfs-bus-iio-lptimer-stm32:8
	Warning: /sys/bus/iio/devices/iio:deviceX/out_altvoltageY_frequency is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4371:0  Documentation/ABI/testing/sysfs-bus-iio:600
	Warning: /sys/bus/iio/devices/iio:deviceX/out_altvoltageY_powerdown is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4371:36  Documentation/ABI/testing/sysfs-bus-iio:589
	Warning: /sys/bus/iio/devices/iio:deviceX/out_currentY_raw is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als:43  Documentation/ABI/testing/sysfs-bus-iio-health-afe440x:38
	Warning: /sys/bus/iio/devices/iio:deviceX/out_current_heater_raw is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-humidity-hdc2010:0  Documentation/ABI/testing/sysfs-bus-iio-humidity-hdc100x:0
	Warning: /sys/bus/iio/devices/iio:deviceX/out_current_heater_raw_available is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-humidity-hdc2010:1  Documentation/ABI/testing/sysfs-bus-iio-humidity-hdc100x:1
	Warning: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-distance-srf08:0  Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935:8
	Warning: /sys/bus/iio/devices/triggerX/sampling_frequency is defined 2 times:  Documentation/ABI/testing/sysfs-bus-iio-timer-stm32:92  Documentation/ABI/testing/sysfs-bus-iio:45
	Warning: /sys/class/backlight/<backlight>/l1_daylight_max is defined 2 times:  Documentation/ABI/testing/sysfs-class-backlight-adp8860:12  Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870:4
	Warning: /sys/class/leds/<led>/hw_pattern is defined 2 times:  Documentation/ABI/testing/sysfs-class-led-trigger-pattern:14  Documentation/ABI/testing/sysfs-class-led-driver-sc27xx:0
	Warning: /sys/class/leds/<led>/repeat is defined 2 times:  Documentation/ABI/testing/sysfs-class-led-trigger-pattern:28  Documentation/ABI/testing/sysfs-class-led-driver-el15203000:0
	Warning: /sys/kernel/iommu_groups/reserved_regions is defined 2 times:  Documentation/ABI/testing/sysfs-kernel-iommu_groups:15  Documentation/ABI/testing/sysfs-kernel-iommu_groups:27

Right now, get_abi.pl has some logic to allow multiple "What:" with
identical strings. It does that by adding extra random characters at
the end of the reference name, and keeping track of them on some
internal hash tables.

As the hash table is local to the script, an external script won't be
able to handle it.

Once those gets fixed, we can consider moving the xref code to
automarkup (although I suspect that this will be a way slower).

Thanks,
Mauro

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2 0/7] Fix some issues at get_abi.pl script
  2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
                   ` (7 preceding siblings ...)
  2021-03-25 19:01 ` [PATCH v2 0/7] Fix some issues at get_abi.pl script Jonathan Corbet
@ 2021-03-31 20:02 ` Jonathan Corbet
  8 siblings, 0 replies; 12+ messages in thread
From: Jonathan Corbet @ 2021-03-31 20:02 UTC (permalink / raw)
  To: Mauro Carvalho Chehab, Linux Doc Mailing List, Andrew Morton
  Cc: Mauro Carvalho Chehab, linux-kernel, Greg Kroah-Hartman,
	John Hubbard, Minchan Kim, Stephen Rothwell

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> This series replace this patch:
> 	https://lore.kernel.org/linux-doc/20210324191722.08d352e4@coco.lan/T/#t
>
> It turns that there were multiple bugs at the get_abi.pl code that
> create cross-references.
>
> Patches 1 to 6 fix those issues, and should apply cleanly on the top of
> the docs tree (although I tested against next-20210323).
>
> Patch 7 is optional, and independent from the other patches. It is meant
> to be applied against akpm's tree.  It makes the description (IMHO) 
> clearer, while producing cross references for the two mentioned symbols.

I've gone ahead and applied the set, with the exception of #7 which
doesn't apply here.

Thanks,

jon

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2 7/7] ABI: sysfs-kernel-mm-cma: fix two cross-references
  2021-03-25 10:38 ` [PATCH v2 7/7] ABI: sysfs-kernel-mm-cma: fix two cross-references Mauro Carvalho Chehab
@ 2021-04-01 21:56   ` John Hubbard
  0 siblings, 0 replies; 12+ messages in thread
From: John Hubbard @ 2021-04-01 21:56 UTC (permalink / raw)
  To: Mauro Carvalho Chehab, Linux Doc Mailing List, Jonathan Corbet,
	Andrew Morton
  Cc: Greg Kroah-Hartman, Minchan Kim, Stephen Rothwell, linux-kernel

On 3/25/21 3:38 AM, Mauro Carvalho Chehab wrote:
> Change the text in order to generate cross-references for
> alloc_pages_success and alloc_pages_fail symbols.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>   Documentation/ABI/testing/sysfs-kernel-mm-cma | 8 +++-----
>   1 file changed, 3 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/ABI/testing/sysfs-kernel-mm-cma b/Documentation/ABI/testing/sysfs-kernel-mm-cma
> index 02b2bb60c296..86e261185561 100644
> --- a/Documentation/ABI/testing/sysfs-kernel-mm-cma
> +++ b/Documentation/ABI/testing/sysfs-kernel-mm-cma
> @@ -5,12 +5,10 @@ Description:
>   		/sys/kernel/mm/cma/ contains a subdirectory for each CMA
>   		heap name (also sometimes called CMA areas).
>   
> -		Each CMA heap subdirectory (that is, each
> -		/sys/kernel/mm/cma/<cma-heap-name> directory) contains the
> -		following items:
> +		Each CMA heap subdirectory contains the following items:
>   
> -			alloc_pages_success
> -			alloc_pages_fail
> +			- /sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_success
> +			- /sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_fail
>   

I agree that this is clearer and easier on the reader, who can now see
directly what the full path to each item is.

As for calling it a "fix", that seems a bit much. It's an upgrade.
I'm not sure how many people realize that this sort of change causes
cross refs to magically start working. I certainly didn't until now.

But either way, this improvement is nice to have, so:

Reviewed-by: John Hubbard <jhubbard@nvidia.com>

thanks,
-- 
John Hubbard
NVIDIA

>   What:		/sys/kernel/mm/cma/<cma-heap-name>/alloc_pages_success
>   Date:		Feb 2021
> 


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2021-04-01 21:57 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-25 10:38 [PATCH v2 0/7] Fix some issues at get_abi.pl script Mauro Carvalho Chehab
2021-03-25 10:38 ` [PATCH v2 1/7] scripts: get_abi.pl: better handle escape chars on what: Mauro Carvalho Chehab
2021-03-25 10:38 ` [PATCH v2 2/7] get_abi.pl: seek for all occurrences for Documentation/ABI Mauro Carvalho Chehab
2021-03-25 10:38 ` [PATCH v2 3/7] get_abi.pl: fix xref boundaries Mauro Carvalho Chehab
2021-03-25 10:38 ` [PATCH v2 4/7] scripts: get_abi.pl: extend xref match to other types Mauro Carvalho Chehab
2021-03-25 10:38 ` [PATCH v2 5/7] scripts: get_abi.pl: parse description line per line Mauro Carvalho Chehab
2021-03-25 10:38 ` [PATCH v2 6/7] scripts: get_abi: ignore code blocks for cross-references Mauro Carvalho Chehab
2021-03-25 10:38 ` [PATCH v2 7/7] ABI: sysfs-kernel-mm-cma: fix two cross-references Mauro Carvalho Chehab
2021-04-01 21:56   ` John Hubbard
2021-03-25 19:01 ` [PATCH v2 0/7] Fix some issues at get_abi.pl script Jonathan Corbet
2021-03-25 20:36   ` Mauro Carvalho Chehab
2021-03-31 20:02 ` Jonathan Corbet

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).