* [PATCH 00/14] Add support to generate ABI documentation at admin-guide
@ 2019-06-14 2:04 Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 01/14] ABI: fix some syntax issues at the ABI database Mauro Carvalho Chehab
` (14 more replies)
0 siblings, 15 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, linuxppc-dev, Andrew Donnellan,
Jonathan Cameron, Lars-Peter Clausen, Anton Vorontsov, linux-pm,
Colin Cross, linux-iio, Hartmut Knaack, Sebastian Reichel,
Frederic Barrat, Andreas Klinger, Tony Luck,
Peter Meerwald-Stadler, Stefan Achatz, Kees Cook
Greg,
As promised, I'm resending the patch series with adds the Kernel ABI to
Documentation/admin-guide.
Those patches are basically the version 3 patchset I sent back in 2017,
rebased on the top of linux-next (next-20190613), and with some fixes
in order for it to work.
- The 4 initial patches to fix some ABI descriptions that are violating
the syntax described at Documentation/ABI/README;
- The next 6 patches are the ones originally written in 2017 with a
script with parses the ABI files;
- The 11th patch is a new one: it relaxes a little bit the parser in
order to parse file headers that contains colons on it;
- The 12th patch adds the new script to the documentation build
system, together with a new python Sphinx extension with calls it;
- The 13th patch fixes the python script when running with newer
Sphinx versions (1.7 and upper);
- The final patch fixes an UTF-8 trouble. I noticed it only with Sphinx
1.4, but it could affect other versions too. So, I ended by changing
the UTF-8 encoding logit to work version-independent, just like
what happens with kerneldoc.py extension.
Mauro Carvalho Chehab (14):
ABI: fix some syntax issues at the ABI database
ABI: sysfs-driver-hid: the "What" field doesn't parse fine
ABI: sysfs-class-uwb_rc: remove a duplicated incomplete entry
ABI: better identificate tables
scripts: add an script to parse the ABI files
scripts/get_abi.pl: parse files with text at beginning
scripts/get_abi.pl: avoid use literal blocks when not needed
scripts/get_abi.pl: split label naming from xref logic
scripts/get_abi.pl: add support for searching for ABI symbols
scripts/get_abi.pl: represent what in tables
scripts/get_abi.pl: fix parse issues with some files
doc-rst: add ABI documentation to the admin-guide book
sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+
docs: sphinx/kernel_abi.py: fix UTF-8 support
.../ABI/obsolete/sysfs-driver-hid-roccat-pyra | 2 +-
Documentation/ABI/testing/pstore | 2 +-
.../sysfs-bus-event_source-devices-format | 2 +-
.../ABI/testing/sysfs-bus-i2c-devices-hm6352 | 6 +-
.../ABI/testing/sysfs-bus-iio-distance-srf08 | 4 +-
.../testing/sysfs-bus-iio-proximity-as3935 | 4 +-
.../ABI/testing/sysfs-bus-pci-devices-cciss | 22 +-
.../testing/sysfs-bus-usb-devices-usbsevseg | 12 +-
.../sysfs-class-backlight-driver-lm3533 | 6 +-
Documentation/ABI/testing/sysfs-class-cxl | 6 +-
Documentation/ABI/testing/sysfs-class-devfreq | 2 +-
.../ABI/testing/sysfs-class-led-driver-lm3533 | 8 +-
.../ABI/testing/sysfs-class-leds-gt683r | 4 +-
.../ABI/testing/sysfs-class-powercap | 2 +-
Documentation/ABI/testing/sysfs-class-uwb_rc | 6 -
Documentation/ABI/testing/sysfs-driver-hid | 12 +-
.../ABI/testing/sysfs-driver-hid-roccat-kone | 2 +-
Documentation/ABI/testing/sysfs-kernel-fscaps | 2 +-
.../ABI/testing/sysfs-kernel-vmcoreinfo | 2 +-
Documentation/admin-guide/abi-obsolete.rst | 10 +
Documentation/admin-guide/abi-removed.rst | 4 +
Documentation/admin-guide/abi-stable.rst | 13 +
Documentation/admin-guide/abi-testing.rst | 19 +
Documentation/admin-guide/abi.rst | 11 +
Documentation/admin-guide/index.rst | 1 +
Documentation/conf.py | 2 +-
Documentation/sphinx/kernel_abi.py | 172 +++++++
scripts/get_abi.pl | 450 ++++++++++++++++++
28 files changed, 731 insertions(+), 57 deletions(-)
create mode 100644 Documentation/admin-guide/abi-obsolete.rst
create mode 100644 Documentation/admin-guide/abi-removed.rst
create mode 100644 Documentation/admin-guide/abi-stable.rst
create mode 100644 Documentation/admin-guide/abi-testing.rst
create mode 100644 Documentation/admin-guide/abi.rst
create mode 100644 Documentation/sphinx/kernel_abi.py
create mode 100755 scripts/get_abi.pl
--
2.21.0
^ permalink raw reply [flat|nested] 63+ messages in thread
* [PATCH 01/14] ABI: fix some syntax issues at the ABI database
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 7:20 ` Andrew Donnellan
2019-06-14 8:30 ` Rafael J. Wysocki
2019-06-14 2:04 ` [PATCH 02/14] ABI: sysfs-driver-hid: the "What" field doesn't parse fine Mauro Carvalho Chehab
` (13 subsequent siblings)
14 siblings, 2 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Andreas Klinger, Jonathan Cameron,
Hartmut Knaack, Lars-Peter Clausen, Peter Meerwald-Stadler,
Frederic Barrat, Andrew Donnellan, Sebastian Reichel, Kees Cook,
Anton Vorontsov, Colin Cross, Tony Luck, linux-iio, linuxppc-dev,
linux-pm, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
On those three files, the ABI representation described at
README are violated.
- at sysfs-bus-iio-proximity-as3935:
a ':' character is missing after "What"
- at sysfs-class-devfreq:
there's a typo at Description
- at sysfs-class-cxl, it is using the ":" character at a
file preamble, causing it to be misinterpreted as a
tag.
- On the other files, instead of "What", they use "Where".
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/ABI/testing/pstore | 2 +-
.../sysfs-bus-event_source-devices-format | 2 +-
.../ABI/testing/sysfs-bus-i2c-devices-hm6352 | 6 ++---
.../ABI/testing/sysfs-bus-iio-distance-srf08 | 4 ++--
.../testing/sysfs-bus-iio-proximity-as3935 | 4 ++--
.../ABI/testing/sysfs-bus-pci-devices-cciss | 22 +++++++++----------
.../testing/sysfs-bus-usb-devices-usbsevseg | 12 +++++-----
Documentation/ABI/testing/sysfs-class-cxl | 6 ++---
Documentation/ABI/testing/sysfs-class-devfreq | 2 +-
.../ABI/testing/sysfs-class-powercap | 2 +-
Documentation/ABI/testing/sysfs-kernel-fscaps | 2 +-
.../ABI/testing/sysfs-kernel-vmcoreinfo | 2 +-
12 files changed, 33 insertions(+), 33 deletions(-)
diff --git a/Documentation/ABI/testing/pstore b/Documentation/ABI/testing/pstore
index 5fca9f5e10a3..8d6e48f4e8ef 100644
--- a/Documentation/ABI/testing/pstore
+++ b/Documentation/ABI/testing/pstore
@@ -1,4 +1,4 @@
-Where: /sys/fs/pstore/... (or /dev/pstore/...)
+What: /sys/fs/pstore/... (or /dev/pstore/...)
Date: March 2011
Kernel Version: 2.6.39
Contact: tony.luck@intel.com
diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
index 77f47ff5ee02..b6f8748e0200 100644
--- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
+++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
@@ -1,4 +1,4 @@
-Where: /sys/bus/event_source/devices/<dev>/format
+What: /sys/bus/event_source/devices/<dev>/format
Date: January 2012
Kernel Version: 3.3
Contact: Jiri Olsa <jolsa@redhat.com>
diff --git a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
index feb2e4a87075..29bd447e50a0 100644
--- a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
+++ b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
@@ -1,18 +1,18 @@
-Where: /sys/bus/i2c/devices/.../heading0_input
+What: /sys/bus/i2c/devices/.../heading0_input
Date: April 2010
Kernel Version: 2.6.36?
Contact: alan.cox@intel.com
Description: Reports the current heading from the compass as a floating
point value in degrees.
-Where: /sys/bus/i2c/devices/.../power_state
+What: /sys/bus/i2c/devices/.../power_state
Date: April 2010
Kernel Version: 2.6.36?
Contact: alan.cox@intel.com
Description: Sets the power state of the device. 0 sets the device into
sleep mode, 1 wakes it up.
-Where: /sys/bus/i2c/devices/.../calibration
+What: /sys/bus/i2c/devices/.../calibration
Date: April 2010
Kernel Version: 2.6.36?
Contact: alan.cox@intel.com
diff --git a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08 b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
index 0a1ca1487fa9..a133fd8d081a 100644
--- a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
+++ b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
@@ -1,4 +1,4 @@
-What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
+What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
Date: January 2017
KernelVersion: 4.11
Contact: linux-iio@vger.kernel.org
@@ -6,7 +6,7 @@ Description:
Show or set the gain boost of the amp, from 0-31 range.
default 31
-What /sys/bus/iio/devices/iio:deviceX/sensor_max_range
+What: /sys/bus/iio/devices/iio:deviceX/sensor_max_range
Date: January 2017
KernelVersion: 4.11
Contact: linux-iio@vger.kernel.org
diff --git a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
index 9a17ab5036a4..c59d95346341 100644
--- a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
+++ b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
@@ -1,4 +1,4 @@
-What /sys/bus/iio/devices/iio:deviceX/in_proximity_input
+What: /sys/bus/iio/devices/iio:deviceX/in_proximity_input
Date: March 2014
KernelVersion: 3.15
Contact: Matt Ranostay <matt.ranostay@konsulko.com>
@@ -6,7 +6,7 @@ Description:
Get the current distance in meters of storm (1km steps)
1000-40000 = distance in meters
-What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
+What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
Date: March 2014
KernelVersion: 3.15
Contact: Matt Ranostay <matt.ranostay@konsulko.com>
diff --git a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
index 53d99edd1d75..eb449169c30b 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
+++ b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
@@ -1,66 +1,66 @@
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
Date: March 2009
Kernel Version: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 0 model for logical drive
Y of controller X.
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
Date: March 2009
Kernel Version: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 0 revision for logical
drive Y of controller X.
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
Date: March 2009
Kernel Version: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 83 serial number for logical
drive Y of controller X.
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
Date: March 2009
Kernel Version: 2.6.30
Contact: iss_storagedev@hp.com
Description: Displays the SCSI INQUIRY page 0 vendor for logical drive
Y of controller X.
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
Date: March 2009
Kernel Version: 2.6.30
Contact: iss_storagedev@hp.com
Description: A symbolic link to /sys/block/cciss!cXdY
-Where: /sys/bus/pci/devices/<dev>/ccissX/rescan
+What: /sys/bus/pci/devices/<dev>/ccissX/rescan
Date: August 2009
Kernel Version: 2.6.31
Contact: iss_storagedev@hp.com
Description: Kicks of a rescan of the controller to discover logical
drive topology changes.
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
Date: August 2009
Kernel Version: 2.6.31
Contact: iss_storagedev@hp.com
Description: Displays the 8-byte LUN ID used to address logical
drive Y of controller X.
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
Date: August 2009
Kernel Version: 2.6.31
Contact: iss_storagedev@hp.com
Description: Displays the RAID level of logical drive Y of
controller X.
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
Date: August 2009
Kernel Version: 2.6.31
Contact: iss_storagedev@hp.com
Description: Displays the usage count (number of opens) of logical drive Y
of controller X.
-Where: /sys/bus/pci/devices/<dev>/ccissX/resettable
+What: /sys/bus/pci/devices/<dev>/ccissX/resettable
Date: February 2011
Kernel Version: 2.6.38
Contact: iss_storagedev@hp.com
@@ -71,7 +71,7 @@ Description: Value of 1 indicates the controller can honor the reset_devices
a dump device, as kdump requires resetting the device in order
to work reliably.
-Where: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
+What: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
Date: July 2011
Kernel Version: 3.0
Contact: iss_storagedev@hp.com
diff --git a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
index 70d00dfa443d..f6199b314196 100644
--- a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
+++ b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
@@ -1,12 +1,12 @@
-Where: /sys/bus/usb/.../powered
+What: /sys/bus/usb/.../powered
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
Description: Controls whether the device's display will powered.
A value of 0 is off and a non-zero value is on.
-Where: /sys/bus/usb/.../mode_msb
-Where: /sys/bus/usb/.../mode_lsb
+What: /sys/bus/usb/.../mode_msb
+What: /sys/bus/usb/.../mode_lsb
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
@@ -16,7 +16,7 @@ Description: Controls the devices display mode.
for an 8 character display the values are
MSB 0x08; LSB 0xFF.
-Where: /sys/bus/usb/.../textmode
+What: /sys/bus/usb/.../textmode
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
@@ -25,13 +25,13 @@ Description: Controls the way the device interprets its text buffer.
hex: each character is between 0-15
ascii: each character is between '0'-'9' and 'A'-'F'.
-Where: /sys/bus/usb/.../text
+What: /sys/bus/usb/.../text
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
Description: The text (or data) for the device to display
-Where: /sys/bus/usb/.../decimals
+What: /sys/bus/usb/.../decimals
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <harrisonmetz@gmail.com>
diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
index bbbabffc682a..fc7c6f7c21b3 100644
--- a/Documentation/ABI/testing/sysfs-class-cxl
+++ b/Documentation/ABI/testing/sysfs-class-cxl
@@ -1,6 +1,6 @@
-Note: Attributes that are shared between devices are stored in the directory
-pointed to by the symlink device/.
-Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
+Please notice that attributes that are shared between devices are stored in
+the directory pointed to by the symlink device/.
+For example, the real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
/sys/class/cxl/afu0.0s/device/irqs_max, i.e. /sys/class/cxl/afu0.0/irqs_max.
diff --git a/Documentation/ABI/testing/sysfs-class-devfreq b/Documentation/ABI/testing/sysfs-class-devfreq
index ee39acacf6f8..01196e19afca 100644
--- a/Documentation/ABI/testing/sysfs-class-devfreq
+++ b/Documentation/ABI/testing/sysfs-class-devfreq
@@ -47,7 +47,7 @@ Description:
What: /sys/class/devfreq/.../trans_stat
Date: October 2012
Contact: MyungJoo Ham <myungjoo.ham@samsung.com>
-Descrtiption:
+Description:
This ABI shows the statistics of devfreq behavior on a
specific device. It shows the time spent in each state and
the number of transitions between states.
diff --git a/Documentation/ABI/testing/sysfs-class-powercap b/Documentation/ABI/testing/sysfs-class-powercap
index db3b3ff70d84..f333a0ccc29b 100644
--- a/Documentation/ABI/testing/sysfs-class-powercap
+++ b/Documentation/ABI/testing/sysfs-class-powercap
@@ -147,6 +147,6 @@ What: /sys/class/powercap/.../<power zone>/enabled
Date: September 2013
KernelVersion: 3.13
Contact: linux-pm@vger.kernel.org
-Description
+Description:
This allows to enable/disable power capping at power zone level.
This applies to current power zone and its children.
diff --git a/Documentation/ABI/testing/sysfs-kernel-fscaps b/Documentation/ABI/testing/sysfs-kernel-fscaps
index 50a3033b5e15..bcff34665192 100644
--- a/Documentation/ABI/testing/sysfs-kernel-fscaps
+++ b/Documentation/ABI/testing/sysfs-kernel-fscaps
@@ -2,7 +2,7 @@ What: /sys/kernel/fscaps
Date: February 2011
KernelVersion: 2.6.38
Contact: Ludwig Nussel <ludwig.nussel@suse.de>
-Description
+Description:
Shows whether file system capabilities are honored
when executing a binary
diff --git a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
index 7bd81168e063..1f1087a5f075 100644
--- a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
+++ b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
@@ -4,7 +4,7 @@ KernelVersion: 2.6.24
Contact: Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp>
Kexec Mailing List <kexec@lists.infradead.org>
Vivek Goyal <vgoyal@redhat.com>
-Description
+Description:
Shows physical address and size of vmcoreinfo ELF note.
First value contains physical address of note in hex and
second value contains the size of note in hex. This ELF
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 02/14] ABI: sysfs-driver-hid: the "What" field doesn't parse fine
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 01/14] ABI: fix some syntax issues at the ABI database Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 03/14] ABI: sysfs-class-uwb_rc: remove a duplicated incomplete entry Mauro Carvalho Chehab
` (12 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
The What: field on this ABI description use a different
syntax than expected, causing it to not be properly parsed
by script.
Fix it.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/ABI/testing/sysfs-driver-hid | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-driver-hid b/Documentation/ABI/testing/sysfs-driver-hid
index 48942cacb0bf..a59533410871 100644
--- a/Documentation/ABI/testing/sysfs-driver-hid
+++ b/Documentation/ABI/testing/sysfs-driver-hid
@@ -1,6 +1,6 @@
-What: For USB devices : /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
- For BT devices : /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
- Symlink : /sys/class/hidraw/hidraw<num>/device/report_descriptor
+What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
+What: /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
+What: /sys/class/hidraw/hidraw<num>/device/report_descriptor
Date: Jan 2011
KernelVersion: 2.0.39
Contact: Alan Ott <alan@signal11.us>
@@ -9,9 +9,9 @@ Description: When read, this file returns the device's raw binary HID
This file cannot be written.
Users: HIDAPI library (http://www.signal11.us/oss/hidapi)
-What: For USB devices : /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
- For BT devices : /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
- Symlink : /sys/class/hidraw/hidraw<num>/device/country
+What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
+What: /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
+What: /sys/class/hidraw/hidraw<num>/device/country
Date: February 2015
KernelVersion: 3.19
Contact: Olivier Gay <ogay@logitech.com>
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 03/14] ABI: sysfs-class-uwb_rc: remove a duplicated incomplete entry
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 01/14] ABI: fix some syntax issues at the ABI database Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 02/14] ABI: sysfs-driver-hid: the "What" field doesn't parse fine Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 04/14] ABI: better identificate tables Mauro Carvalho Chehab
` (11 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
There are two entries for /sys/class/uwb_rc/uwbN/<EUI-48>/BPST.
The second one has a missing description.
Get rid of it.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/ABI/testing/sysfs-class-uwb_rc | 6 ------
1 file changed, 6 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-class-uwb_rc b/Documentation/ABI/testing/sysfs-class-uwb_rc
index 85f4875d16ac..a0578751c1e3 100644
--- a/Documentation/ABI/testing/sysfs-class-uwb_rc
+++ b/Documentation/ABI/testing/sysfs-class-uwb_rc
@@ -125,12 +125,6 @@ Description:
The EUI-48 of this device in colon separated hex
octets.
-What: /sys/class/uwb_rc/uwbN/<EUI-48>/BPST
-Date: July 2008
-KernelVersion: 2.6.27
-Contact: linux-usb@vger.kernel.org
-Description:
-
What: /sys/class/uwb_rc/uwbN/<EUI-48>/IEs
Date: July 2008
KernelVersion: 2.6.27
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 04/14] ABI: better identificate tables
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (2 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 03/14] ABI: sysfs-class-uwb_rc: remove a duplicated incomplete entry Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-19 12:51 ` Johan Hovold
2019-06-14 2:04 ` [PATCH 05/14] scripts: add an script to parse the ABI files Mauro Carvalho Chehab
` (10 subsequent siblings)
14 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Stefan Achatz, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
When parsing via script, it is important to know if the script
should consider a description as a literal block that should
be displayed as-is, or if the description can be considered
as a normal text.
Change descriptions to ensure that the preceding line of a table
ends with a colon. That makes easy to identify the need of a
literal block.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra | 2 +-
.../ABI/testing/sysfs-class-backlight-driver-lm3533 | 6 +++---
Documentation/ABI/testing/sysfs-class-led-driver-lm3533 | 8 ++++----
Documentation/ABI/testing/sysfs-class-leds-gt683r | 4 ++--
Documentation/ABI/testing/sysfs-driver-hid-roccat-kone | 2 +-
5 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
index 16020b31ae64..5d41ebadf15e 100644
--- a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
+++ b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
@@ -5,7 +5,7 @@ Description: It is possible to switch the cpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual cpi
setting reported by the mouse. This number has to be further
- processed to receive the real dpi value.
+ processed to receive the real dpi value:
VALUE DPI
1 400
diff --git a/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 b/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533
index 77cf7ac949af..c0e0a9ae7b3d 100644
--- a/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533
+++ b/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533
@@ -4,7 +4,7 @@ KernelVersion: 3.5
Contact: Johan Hovold <jhovold@gmail.com>
Description:
Get the ALS output channel used as input in
- ALS-current-control mode (0, 1), where
+ ALS-current-control mode (0, 1), where:
0 - out_current0 (backlight 0)
1 - out_current1 (backlight 1)
@@ -28,7 +28,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <jhovold@gmail.com>
Description:
- Set the brightness-mapping mode (0, 1), where
+ Set the brightness-mapping mode (0, 1), where:
0 - exponential mode
1 - linear mode
@@ -38,7 +38,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <jhovold@gmail.com>
Description:
- Set the PWM-input control mask (5 bits), where
+ Set the PWM-input control mask (5 bits), where:
bit 5 - PWM-input enabled in Zone 4
bit 4 - PWM-input enabled in Zone 3
diff --git a/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 b/Documentation/ABI/testing/sysfs-class-led-driver-lm3533
index 620ebb3b9baa..e4c89b261546 100644
--- a/Documentation/ABI/testing/sysfs-class-led-driver-lm3533
+++ b/Documentation/ABI/testing/sysfs-class-led-driver-lm3533
@@ -4,7 +4,7 @@ KernelVersion: 3.5
Contact: Johan Hovold <jhovold@gmail.com>
Description:
Set the ALS output channel to use as input in
- ALS-current-control mode (1, 2), where
+ ALS-current-control mode (1, 2), where:
1 - out_current1
2 - out_current2
@@ -22,7 +22,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <jhovold@gmail.com>
Description:
- Set the pattern generator fall and rise times (0..7), where
+ Set the pattern generator fall and rise times (0..7), where:
0 - 2048 us
1 - 262 ms
@@ -45,7 +45,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <jhovold@gmail.com>
Description:
- Set the brightness-mapping mode (0, 1), where
+ Set the brightness-mapping mode (0, 1), where:
0 - exponential mode
1 - linear mode
@@ -55,7 +55,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <jhovold@gmail.com>
Description:
- Set the PWM-input control mask (5 bits), where
+ Set the PWM-input control mask (5 bits), where:
bit 5 - PWM-input enabled in Zone 4
bit 4 - PWM-input enabled in Zone 3
diff --git a/Documentation/ABI/testing/sysfs-class-leds-gt683r b/Documentation/ABI/testing/sysfs-class-leds-gt683r
index e4fae6026e79..6adab27f646e 100644
--- a/Documentation/ABI/testing/sysfs-class-leds-gt683r
+++ b/Documentation/ABI/testing/sysfs-class-leds-gt683r
@@ -5,7 +5,7 @@ Contact: Janne Kanniainen <janne.kanniainen@gmail.com>
Description:
Set the mode of LEDs. You should notice that changing the mode
of one LED will update the mode of its two sibling devices as
- well.
+ well. Possible values are:
0 - normal
1 - audio
@@ -13,4 +13,4 @@ Description:
Normal: LEDs are fully on when enabled
Audio: LEDs brightness depends on sound level
- Breathing: LEDs brightness varies at human breathing rate
\ No newline at end of file
+ Breathing: LEDs brightness varies at human breathing rate
diff --git a/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone b/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone
index 3ca3971109bf..8f7982c70d72 100644
--- a/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone
+++ b/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone
@@ -5,7 +5,7 @@ Description: It is possible to switch the dpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual dpi
setting reported by the mouse. This number has to be further
- processed to receive the real dpi value.
+ processed to receive the real dpi value:
VALUE DPI
1 800
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 05/14] scripts: add an script to parse the ABI files
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (3 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 04/14] ABI: better identificate tables Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 13:31 ` Jani Nikula
2019-06-14 2:04 ` [PATCH 06/14] scripts/get_abi.pl: parse files with text at beginning Mauro Carvalho Chehab
` (9 subsequent siblings)
14 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Add a script to parse the Documentation/ABI files and produce
an output with all entries inside an ABI (sub)directory.
Right now, it outputs its contents on ReST format. It shouldn't
be hard to make it produce other kind of outputs, since the ABI
file parser is implemented in separate than the output generator.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 212 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 212 insertions(+)
create mode 100755 scripts/get_abi.pl
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
new file mode 100755
index 000000000000..f7c9944a833c
--- /dev/null
+++ b/scripts/get_abi.pl
@@ -0,0 +1,212 @@
+#!/usr/bin/perl
+
+use strict;
+use Pod::Usage;
+use Getopt::Long;
+use File::Find;
+use Fcntl ':mode';
+
+my $help;
+my $man;
+my $debug;
+
+GetOptions(
+ "debug|d+" => \$debug,
+ 'help|?' => \$help,
+ man => \$man
+) or pod2usage(2);
+
+pod2usage(1) if $help;
+pod2usage(-exitstatus => 0, -verbose => 2) if $man;
+
+pod2usage(2) if (scalar @ARGV != 1);
+
+my ($prefix) = @ARGV;
+
+require Data::Dumper if ($debug);
+
+my %data;
+
+#
+# Displays an error message, printing file name and line
+#
+sub parse_error($$$$) {
+ my ($file, $ln, $msg, $data) = @_;
+
+ print STDERR "file $file#$ln: $msg at\n\t$data";
+}
+
+#
+# Parse an ABI file, storing its contents at %data
+#
+sub parse_abi {
+ my $file = $File::Find::name;
+
+ my $mode = (stat($file))[2];
+ return if ($mode & S_IFDIR);
+ return if ($file =~ m,/README,);
+
+ my $name = $file;
+ $name =~ s,.*/,,;
+
+ my $type = $file;
+ $type =~ s,.*/(.*)/.*,$1,;
+
+ my $what;
+ my $new_what;
+ my $tag;
+ my $ln;
+
+ print STDERR "Opening $file\n" if ($debug > 1);
+ open IN, $file;
+ while(<IN>) {
+ $ln++;
+ if (m/^(\S+):\s*(.*)/i) {
+ my $new_tag = lc($1);
+ my $content = $2;
+
+ if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
+ if ($tag eq "description") {
+ $data{$what}->{$tag} .= "\n$content";;
+ $data{$what}->{$tag} =~ s/\n+$//;
+ next;
+ } else {
+ parse_error($file, $ln, "tag '$tag' is invalid", $_);
+ }
+ }
+
+ if ($new_tag =~ m/what/) {
+ if ($tag =~ m/what/) {
+ $what .= ", " . $content;
+ } else {
+ $what = $content;
+ $new_what = 1;
+ }
+ $tag = $new_tag;
+ next;
+ }
+
+ $tag = $new_tag;
+
+ if ($new_what) {
+ $new_what = 0;
+
+ $data{$what}->{type} = $type;
+ $data{$what}->{file} = $name;
+ print STDERR "\twhat: $what\n" if ($debug > 1);
+ }
+
+ if (!$what) {
+ parse_error($file, $ln, "'What:' should come first:", $_);
+ next;
+ }
+ $data{$what}->{$tag} = $content;
+ next;
+ }
+
+ # Silently ignore any headers before the database
+ next if (!$tag);
+
+ if (m/^\s*(.*)/) {
+ $data{$what}->{$tag} .= "\n$1";
+ $data{$what}->{$tag} =~ s/\n+$//;
+ next;
+ }
+
+ # Everything else is error
+ parse_error($file, $ln, "Unexpected line:", $_);
+ }
+ close IN;
+}
+
+# Outputs the output on ReST format
+sub output_rest {
+ foreach my $what (sort keys %data) {
+ my $type = $data{$what}->{type};
+ my $file = $data{$what}->{file};
+
+ my $w = $what;
+ $w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
+
+ print "$w\n\n";
+ print "- defined on file $file (type: $type)\n\n::\n\n";
+
+ my $desc = $data{$what}->{description};
+ $desc =~ s/^\s+//;
+
+ # Remove title markups from the description, as they won't work
+ $desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
+
+ # put everything inside a code block
+ $desc =~ s/\n/\n /g;
+
+
+ if (!($desc =~ /^\s*$/)) {
+ print " $desc\n\n";
+ } else {
+ print " DESCRIPTION MISSING\n\n";
+ }
+ }
+}
+
+#
+# Parses all ABI files located at $prefix dir
+#
+find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
+
+print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
+
+#
+# Outputs an ReST file with the ABI contents
+#
+output_rest
+
+
+__END__
+
+=head1 NAME
+
+abi_book.pl - parse the Linux ABI files and produce a ReST book.
+
+=head1 SYNOPSIS
+
+B<abi_book.pl> [--debug] <ABI_DIR>]
+
+=head1 OPTIONS
+
+=over 8
+
+=item B<--debug>
+
+Put the script in verbose mode, useful for debugging. Can be called multiple
+times, to increase verbosity.
+
+=item B<--help>
+
+Prints a brief help message and exits.
+
+=item B<--man>
+
+Prints the manual page and exits.
+
+=back
+
+=head1 DESCRIPTION
+
+Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
+and produce a ReST book containing the Linux ABI.
+
+=head1 BUGS
+
+Report bugs to Mauro Carvalho Chehab <mchehab@s-opensource.com>
+
+=head1 COPYRIGHT
+
+Copyright (c) 2016 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.
+
+=cut
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 06/14] scripts/get_abi.pl: parse files with text at beginning
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (4 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 05/14] scripts: add an script to parse the ABI files Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 07/14] scripts/get_abi.pl: avoid use literal blocks when not needed Mauro Carvalho Chehab
` (8 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
It sounds usefult o parse files with has some text at the
beginning. Add support for it.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 59 ++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 54 insertions(+), 5 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index f7c9944a833c..ac0f057fa3ca 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -55,7 +55,10 @@ sub parse_abi {
my $what;
my $new_what;
my $tag;
+ my $label;
my $ln;
+ my $has_file;
+ my $xrefs;
print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
@@ -67,7 +70,7 @@ sub parse_abi {
if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
if ($tag eq "description") {
- $data{$what}->{$tag} .= "\n$content";;
+ $data{$what}->{$tag} .= "\n$content";
$data{$what}->{$tag} =~ s/\n+$//;
next;
} else {
@@ -83,6 +86,25 @@ sub parse_abi {
$new_what = 1;
}
$tag = $new_tag;
+
+ if ($has_file) {
+ $label = "abi_" . $content . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/[\x00-\x2f]+/_/g;
+ $label =~s/[\x3a-\x40]+/_/g;
+ $label =~s/[\x7b-\xff]+/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ $data{$what}->{label} .= $label;
+
+ # Escape special chars from content
+ $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
+
+ $xrefs .= "- :ref:`$content <$label>`\n\n";
+ }
next;
}
@@ -104,8 +126,18 @@ sub parse_abi {
next;
}
- # Silently ignore any headers before the database
- next if (!$tag);
+ # Store any contents before the database
+ if (!$tag) {
+ next if (/^\n/);
+
+ my $my_what = "File $name";
+ $data{$my_what}->{what} = "File $name";
+ $data{$my_what}->{type} = "File";
+ $data{$my_what}->{file} = $name;
+ $data{$my_what}->{description} .= $_;
+ $has_file = 1;
+ next;
+ }
if (m/^\s*(.*)/) {
$data{$what}->{$tag} .= "\n$1";
@@ -117,6 +149,11 @@ sub parse_abi {
parse_error($file, $ln, "Unexpected line:", $_);
}
close IN;
+
+ if ($has_file) {
+ my $my_what = "File $name";
+ $data{$my_what}->{xrefs} = $xrefs;
+ }
}
# Outputs the output on ReST format
@@ -128,8 +165,17 @@ sub output_rest {
my $w = $what;
$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
+ if ($data{$what}->{label}) {
+ my @labels = split(/\s/, $data{$what}->{label});
+ foreach my $label (@labels) {
+ printf ".. _%s:\n\n", $label;
+ }
+ }
+
print "$w\n\n";
- print "- defined on file $file (type: $type)\n\n::\n\n";
+
+ print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
+ print "::\n\n";
my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
@@ -144,8 +190,11 @@ sub output_rest {
if (!($desc =~ /^\s*$/)) {
print " $desc\n\n";
} else {
- print " DESCRIPTION MISSING\n\n";
+ print " DESCRIPTION MISSING for $what\n\n";
}
+
+ printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
+
}
}
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 07/14] scripts/get_abi.pl: avoid use literal blocks when not needed
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (5 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 06/14] scripts/get_abi.pl: parse files with text at beginning Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 08/14] scripts/get_abi.pl: split label naming from xref logic Mauro Carvalho Chehab
` (7 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
The usage of literal blocks make the document very complex,
causing the browser to take a long time to load.
On most ABI descriptions, they're a plain text, and don't
require a literal block.
So, add a logic there with identifies when a literal block
is needed.
As, on literal blocks, we need to respect the original
document space, the most complex part of this patch is
to preserve the original spacing where needed.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 104 +++++++++++++++++++++++++++++++++------------
1 file changed, 78 insertions(+), 26 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index ac0f057fa3ca..ba8a7466f896 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -59,29 +59,34 @@ sub parse_abi {
my $ln;
my $has_file;
my $xrefs;
+ my $space;
print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
while(<IN>) {
$ln++;
- if (m/^(\S+):\s*(.*)/i) {
+ if (m/^(\S+)(:\s*)(.*)/i) {
my $new_tag = lc($1);
- my $content = $2;
+ my $sep = $2;
+ my $content = $3;
if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
if ($tag eq "description") {
- $data{$what}->{$tag} .= "\n$content";
- $data{$what}->{$tag} =~ s/\n+$//;
- next;
+ # New "tag" is actually part of
+ # description. Don't consider it a tag
+ $new_tag = "";
} else {
parse_error($file, $ln, "tag '$tag' is invalid", $_);
}
}
if ($new_tag =~ m/what/) {
+ $space = "";
if ($tag =~ m/what/) {
$what .= ", " . $content;
} else {
+ parse_error($file, $ln, "What '$what' doesn't have a description", "") if ($what && !$data{$what}->{description});
+
$what = $content;
$new_what = 1;
}
@@ -108,25 +113,38 @@ sub parse_abi {
next;
}
- $tag = $new_tag;
+ if ($new_tag) {
+ $tag = $new_tag;
- if ($new_what) {
- $new_what = 0;
+ if ($new_what) {
+ $new_what = 0;
- $data{$what}->{type} = $type;
- $data{$what}->{file} = $name;
- print STDERR "\twhat: $what\n" if ($debug > 1);
- }
+ $data{$what}->{type} = $type;
+ $data{$what}->{file} = $name;
+ print STDERR "\twhat: $what\n" if ($debug > 1);
+ }
- if (!$what) {
- parse_error($file, $ln, "'What:' should come first:", $_);
+ if (!$what) {
+ parse_error($file, $ln, "'What:' should come first:", $_);
+ next;
+ }
+ if ($tag eq "description") {
+ next if ($content =~ m/^\s*$/);
+ if ($content =~ m/^(\s*)(.*)/) {
+ my $new_content = $2;
+ $space = $new_tag . $sep . $1;
+ while ($space =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $space =~ s/./ /g;
+ $data{$what}->{$tag} .= "$new_content\n";
+ }
+ } else {
+ $data{$what}->{$tag} = $content;
+ }
next;
}
- $data{$what}->{$tag} = $content;
- next;
}
- # Store any contents before the database
+ # Store any contents before tags at the database
if (!$tag) {
next if (/^\n/);
@@ -139,6 +157,32 @@ sub parse_abi {
next;
}
+ if ($tag eq "description") {
+ if (!$data{$what}->{description}) {
+ next if (m/^\s*\n/);
+ if (m/^(\s*)(.*)/) {
+ $space = $1;
+ while ($space =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $data{$what}->{$tag} .= "$2\n";
+ }
+ } else {
+ my $content = $_;
+ if (m/^\s*\n/) {
+ $data{$what}->{$tag} .= $content;
+ next;
+ }
+
+ while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $space = "" if (!($content =~ s/^($space)//));
+
+ # Compress spaces with tabs
+ $content =~ s<^ {8}> <\t>;
+ $content =~ s<^ {1,7}\t> <\t>;
+ $content =~ s< {1,7}\t> <\t>;
+ $data{$what}->{$tag} .= $content;
+ }
+ next;
+ }
if (m/^\s*(.*)/) {
$data{$what}->{$tag} .= "\n$1";
$data{$what}->{$tag} =~ s/\n+$//;
@@ -165,6 +209,9 @@ sub output_rest {
my $w = $what;
$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
+ my $bar = $w;
+ $bar =~ s/./-/g;
+
if ($data{$what}->{label}) {
my @labels = split(/\s/, $data{$what}->{label});
foreach my $label (@labels) {
@@ -172,10 +219,9 @@ sub output_rest {
}
}
- print "$w\n\n";
+ print "$w\n$bar\n\n";
print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
- print "::\n\n";
my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
@@ -183,18 +229,24 @@ sub output_rest {
# Remove title markups from the description, as they won't work
$desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
- # put everything inside a code block
- $desc =~ s/\n/\n /g;
-
-
if (!($desc =~ /^\s*$/)) {
- print " $desc\n\n";
+ if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
+ # put everything inside a code block
+ $desc =~ s/\n/\n /g;
+
+ print "::\n\n";
+ print " $desc\n\n";
+ } else {
+ # Escape any special chars from description
+ $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
+
+ print "$desc\n\n";
+ }
} else {
- print " DESCRIPTION MISSING for $what\n\n";
+ print "DESCRIPTION MISSING for $what\n\n";
}
printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
-
}
}
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 08/14] scripts/get_abi.pl: split label naming from xref logic
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (6 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 07/14] scripts/get_abi.pl: avoid use literal blocks when not needed Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 09/14] scripts/get_abi.pl: add support for searching for ABI symbols Mauro Carvalho Chehab
` (6 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Instead of using a ReST compilant label while parsing,
move the label to ReST output. That makes the parsing logic
more generic, allowing it to provide other types of output.
As a side effect, now all files used to generate the output
will be output. We can later add command line arguments to
filter.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 94 ++++++++++++++++++++++++++--------------------
1 file changed, 53 insertions(+), 41 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index ba8a7466f896..d437e148b1c0 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -49,17 +49,23 @@ sub parse_abi {
my $name = $file;
$name =~ s,.*/,,;
+ my $nametag = "File $name";
+ $data{$nametag}->{what} = "File $name";
+ $data{$nametag}->{type} = "File";
+ $data{$nametag}->{file} = $name;
+ $data{$nametag}->{is_file} = 1;
+
my $type = $file;
$type =~ s,.*/(.*)/.*,$1,;
my $what;
my $new_what;
my $tag;
- my $label;
my $ln;
- my $has_file;
my $xrefs;
my $space;
+ my @labels;
+ my $label;
print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
@@ -88,28 +94,13 @@ sub parse_abi {
parse_error($file, $ln, "What '$what' doesn't have a description", "") if ($what && !$data{$what}->{description});
$what = $content;
+ $label = $content;
$new_what = 1;
}
+ push @labels, [($content, $label)];
$tag = $new_tag;
- if ($has_file) {
- $label = "abi_" . $content . " ";
- $label =~ tr/A-Z/a-z/;
-
- # Convert special chars to "_"
- $label =~s/[\x00-\x2f]+/_/g;
- $label =~s/[\x3a-\x40]+/_/g;
- $label =~s/[\x7b-\xff]+/_/g;
- $label =~ s,_+,_,g;
- $label =~ s,_$,,;
-
- $data{$what}->{label} .= $label;
-
- # Escape special chars from content
- $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
-
- $xrefs .= "- :ref:`$content <$label>`\n\n";
- }
+ push @{$data{$nametag}->{xrefs}}, [($content, $label)] if ($data{$nametag}->{what});
next;
}
@@ -117,6 +108,9 @@ sub parse_abi {
$tag = $new_tag;
if ($new_what) {
+ @{$data{$what}->{label}} = @labels if ($data{$nametag}->{what});
+ @labels = ();
+ $label = "";
$new_what = 0;
$data{$what}->{type} = $type;
@@ -145,15 +139,8 @@ sub parse_abi {
}
# Store any contents before tags at the database
- if (!$tag) {
- next if (/^\n/);
-
- my $my_what = "File $name";
- $data{$my_what}->{what} = "File $name";
- $data{$my_what}->{type} = "File";
- $data{$my_what}->{file} = $name;
- $data{$my_what}->{description} .= $_;
- $has_file = 1;
+ if (!$tag && $data{$nametag}->{what}) {
+ $data{$nametag}->{description} .= $_;
next;
}
@@ -192,12 +179,8 @@ sub parse_abi {
# Everything else is error
parse_error($file, $ln, "Unexpected line:", $_);
}
+ $data{$nametag}->{description} =~ s/^\n+//;
close IN;
-
- if ($has_file) {
- my $my_what = "File $name";
- $data{$my_what}->{xrefs} = $xrefs;
- }
}
# Outputs the output on ReST format
@@ -212,11 +195,22 @@ sub output_rest {
my $bar = $w;
$bar =~ s/./-/g;
- if ($data{$what}->{label}) {
- my @labels = split(/\s/, $data{$what}->{label});
- foreach my $label (@labels) {
- printf ".. _%s:\n\n", $label;
- }
+ foreach my $p (@{$data{$what}->{label}}) {
+ my ($content, $label) = @{$p};
+ $label = "abi_" . $label . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ $data{$what}->{label} .= $label;
+
+ printf ".. _%s:\n\n", $label;
+
+ # only one label is enough
+ last;
}
print "$w\n$bar\n\n";
@@ -243,10 +237,28 @@ sub output_rest {
print "$desc\n\n";
}
} else {
- print "DESCRIPTION MISSING for $what\n\n";
+ print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file});
}
- printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
+ if ($data{$what}->{xrefs}) {
+ printf "Has the following ABI:\n\n";
+
+ foreach my $p(@{$data{$what}->{xrefs}}) {
+ my ($content, $label) = @{$p};
+ $label = "abi_" . $label . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ # Escape special chars from content
+ $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
+
+ print "- :ref:`$content <$label>`\n\n";
+ }
+ }
}
}
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 09/14] scripts/get_abi.pl: add support for searching for ABI symbols
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (7 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 08/14] scripts/get_abi.pl: split label naming from xref logic Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 10/14] scripts/get_abi.pl: represent what in tables Mauro Carvalho Chehab
` (5 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Change its syntax to allow switching between ReST output mode
and a new search mode, with allows to seek for ABI symbols
using regex.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 112 +++++++++++++++++++++++++++++++++++++++++----
1 file changed, 103 insertions(+), 9 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index d437e148b1c0..c202292af1a2 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -9,9 +9,11 @@ use Fcntl ':mode';
my $help;
my $man;
my $debug;
+my $prefix="Documentation/ABI";
GetOptions(
"debug|d+" => \$debug,
+ "dir=s" => \$prefix,
'help|?' => \$help,
man => \$man
) or pod2usage(2);
@@ -19,9 +21,12 @@ GetOptions(
pod2usage(1) if $help;
pod2usage(-exitstatus => 0, -verbose => 2) if $man;
-pod2usage(2) if (scalar @ARGV != 1);
+pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2);
-my ($prefix) = @ARGV;
+my ($cmd, $arg) = @ARGV;
+
+pod2usage(2) if ($cmd ne "search" && $cmd ne "rest");
+pod2usage(2) if ($cmd eq "search" && !$arg);
require Data::Dumper if ($debug);
@@ -53,6 +58,7 @@ sub parse_abi {
$data{$nametag}->{what} = "File $name";
$data{$nametag}->{type} = "File";
$data{$nametag}->{file} = $name;
+ $data{$nametag}->{filepath} = $file;
$data{$nametag}->{is_file} = 1;
my $type = $file;
@@ -115,6 +121,7 @@ sub parse_abi {
$data{$what}->{type} = $type;
$data{$what}->{file} = $name;
+ $data{$what}->{filepath} = $file;
print STDERR "\twhat: $what\n" if ($debug > 1);
}
@@ -183,7 +190,9 @@ sub parse_abi {
close IN;
}
-# Outputs the output on ReST format
+#
+# Outputs the book on ReST format
+#
sub output_rest {
foreach my $what (sort keys %data) {
my $type = $data{$what}->{type};
@@ -262,6 +271,45 @@ sub output_rest {
}
}
+#
+# Searches for ABI symbols
+#
+sub search_symbols {
+ foreach my $what (sort keys %data) {
+ next if (!($what =~ m/($arg)/));
+
+ my $type = $data{$what}->{type};
+ next if ($type eq "File");
+
+ my $file = $data{$what}->{filepath};
+
+ my $bar = $what;
+ $bar =~ s/./-/g;
+
+ print "\n$what\n$bar\n\n";
+
+ my $kernelversion = $data{$what}->{kernelversion};
+ my $contact = $data{$what}->{contact};
+ my $users = $data{$what}->{users};
+ my $date = $data{$what}->{date};
+ my $desc = $data{$what}->{description};
+ $kernelversion =~ s/^\s+//;
+ $contact =~ s/^\s+//;
+ $users =~ s/^\s+//;
+ $users =~ s/\n//g;
+ $date =~ s/^\s+//;
+ $desc =~ s/^\s+//;
+
+ printf "Kernel version:\t\t%s\n", $kernelversion if ($kernelversion);
+ printf "Date:\t\t\t%s\n", $date if ($date);
+ printf "Contact:\t\t%s\n", $contact if ($contact);
+ printf "Users:\t\t\t%s\n", $users if ($users);
+ print "Defined on file:\t$file\n\n";
+ print "Description:\n\n$desc";
+ }
+}
+
+
#
# Parses all ABI files located at $prefix dir
#
@@ -270,9 +318,13 @@ find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
#
-# Outputs an ReST file with the ABI contents
+# Handles the command
#
-output_rest
+if ($cmd eq "rest") {
+ output_rest;
+} else {
+ search_symbols;
+}
__END__
@@ -283,12 +335,27 @@ abi_book.pl - parse the Linux ABI files and produce a ReST book.
=head1 SYNOPSIS
-B<abi_book.pl> [--debug] <ABI_DIR>]
+B<abi_book.pl> [--debug] <COMAND> [<ARGUMENT>]
+
+Where <COMMAND> can be:
+
+=over 8
+
+B<search> [SEARCH_REGEX] - search for [SEARCH_REGEX] inside ABI
+
+B<rest> - output the ABI in ReST markup language
+
+=back
=head1 OPTIONS
=over 8
+=item B<--dir>
+
+Changes the location of the ABI search. By default, it uses
+the Documentation/ABI directory.
+
=item B<--debug>
Put the script in verbose mode, useful for debugging. Can be called multiple
@@ -306,8 +373,35 @@ Prints the manual page and exits.
=head1 DESCRIPTION
-Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
-and produce a ReST book containing the Linux ABI.
+Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI),
+allowing to search for ABI symbols or to produce a ReST book containing
+the Linux ABI documentation.
+
+=head1 EXAMPLES
+
+Search for all stable symbols with the word "usb":
+
+=over 8
+
+$ scripts/get_abi.pl search usb --dir Documentation/ABI/stable
+
+=back
+
+Search for all symbols that match the regex expression "usb.*cap":
+
+=over 8
+
+$ scripts/get_abi.pl search usb.*cap
+
+=back
+
+Output all obsoleted symbols in ReST format
+
+=over 8
+
+$ scripts/get_abi.pl rest --dir Documentation/ABI/obsolete
+
+=back
=head1 BUGS
@@ -315,7 +409,7 @@ Report bugs to Mauro Carvalho Chehab <mchehab@s-opensource.com>
=head1 COPYRIGHT
-Copyright (c) 2016 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
+Copyright (c) 2016-2017 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 10/14] scripts/get_abi.pl: represent what in tables
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (8 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 09/14] scripts/get_abi.pl: add support for searching for ABI symbols Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 11/14] scripts/get_abi.pl: fix parse issues with some files Mauro Carvalho Chehab
` (4 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Several entries at the ABI have multiple What: with the same
description.
Instead of showing those symbols as sections, let's show them
as tables. That makes easier to read on the final output,
and avoid too much recursion at Sphinx parsing.
We need to put file references at the end, as we don't want
non-file tables to be mangled with other entries.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 41 ++++++++++++++++++++++++++++++++++++-----
1 file changed, 36 insertions(+), 5 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index c202292af1a2..a2861fcec745 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -193,16 +193,19 @@ sub parse_abi {
#
# Outputs the book on ReST format
#
+
sub output_rest {
- foreach my $what (sort keys %data) {
+ foreach my $what (sort {
+ ($data{$a}->{type} eq "File") cmp ($data{$b}->{type} eq "File") ||
+ $a cmp $b
+ } keys %data) {
my $type = $data{$what}->{type};
my $file = $data{$what}->{file};
+ my $filepath = $data{$what}->{filepath};
my $w = $what;
$w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
- my $bar = $w;
- $bar =~ s/./-/g;
foreach my $p (@{$data{$what}->{label}}) {
my ($content, $label) = @{$p};
@@ -222,9 +225,37 @@ sub output_rest {
last;
}
- print "$w\n$bar\n\n";
- print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
+ $filepath =~ s,.*/(.*/.*),\1,;;
+ $filepath =~ s,[/\-],_,g;;
+ my $fileref = "abi_file_".$filepath;
+
+ if ($type eq "File") {
+ my $bar = $w;
+ $bar =~ s/./-/g;
+
+ print ".. _$fileref:\n\n";
+ print "$w\n$bar\n\n";
+ } else {
+ my @names = split /\s*,\s*/,$w;
+
+ my $len = 0;
+
+ foreach my $name (@names) {
+ $len = length($name) if (length($name) > $len);
+ }
+
+ print "What:\n\n";
+
+ print "+-" . "-" x $len . "-+\n";
+ foreach my $name (@names) {
+ printf "| %s", $name . " " x ($len - length($name)) . " |\n";
+ print "+-" . "-" x $len . "-+\n";
+ }
+ print "\n";
+ }
+
+ print "Defined on file :ref:`$file <$fileref>`\n\n" if ($type ne "File");
my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 11/14] scripts/get_abi.pl: fix parse issues with some files
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (9 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 10/14] scripts/get_abi.pl: represent what in tables Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book Mauro Carvalho Chehab
` (3 subsequent siblings)
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
A few files are failing to parse:
Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
Documentation/ABI/testing/sysfs-class-pktcdvd
Documentation/ABI/testing/sysfs-bus-nfit
On all three files, the problem is that there is a ":" character
at the initial file description.
Improve the parse in order to handle those special cases.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
scripts/get_abi.pl | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index a2861fcec745..c3ccfdfc3a5d 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -87,7 +87,7 @@ sub parse_abi {
# New "tag" is actually part of
# description. Don't consider it a tag
$new_tag = "";
- } else {
+ } elsif ($tag ne "") {
parse_error($file, $ln, "tag '$tag' is invalid", $_);
}
}
@@ -110,7 +110,7 @@ sub parse_abi {
next;
}
- if ($new_tag) {
+ if ($tag ne "" && $new_tag) {
$tag = $new_tag;
if ($new_what) {
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (10 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 11/14] scripts/get_abi.pl: fix parse issues with some files Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 13:42 ` Jani Nikula
2019-06-14 2:04 ` [PATCH 13/14] sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+ Mauro Carvalho Chehab
` (2 subsequent siblings)
14 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
As we don't want a generic Sphinx extension to execute commands,
change the one proposed to Markus to call the abi_book.pl
script.
Use a script to parse the Documentation/ABI directory and output
it at the admin-guide.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/admin-guide/abi-obsolete.rst | 10 ++
Documentation/admin-guide/abi-removed.rst | 4 +
Documentation/admin-guide/abi-stable.rst | 13 ++
Documentation/admin-guide/abi-testing.rst | 19 +++
Documentation/admin-guide/abi.rst | 11 ++
Documentation/admin-guide/index.rst | 1 +
Documentation/conf.py | 2 +-
Documentation/sphinx/kernel_abi.py | 155 +++++++++++++++++++++
8 files changed, 214 insertions(+), 1 deletion(-)
create mode 100644 Documentation/admin-guide/abi-obsolete.rst
create mode 100644 Documentation/admin-guide/abi-removed.rst
create mode 100644 Documentation/admin-guide/abi-stable.rst
create mode 100644 Documentation/admin-guide/abi-testing.rst
create mode 100644 Documentation/admin-guide/abi.rst
create mode 100644 Documentation/sphinx/kernel_abi.py
diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
new file mode 100644
index 000000000000..cda9168445a5
--- /dev/null
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -0,0 +1,10 @@
+ABI obsolete symbols
+====================
+
+Documents interfaces that are still remaining in the kernel, but are
+marked to be removed at some later point in time.
+
+The description of the interface will document the reason why it is
+obsolete and when it can be expected to be removed.
+
+.. kernel-abi:: $srctree/Documentation/ABI/obsolete
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
new file mode 100644
index 000000000000..497978fc9632
--- /dev/null
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -0,0 +1,4 @@
+ABI removed symbols
+===================
+
+.. kernel-abi:: $srctree/Documentation/ABI/removed
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
new file mode 100644
index 000000000000..7495d7a35048
--- /dev/null
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -0,0 +1,13 @@
+ABI stable symbols
+==================
+
+Documents the interfaces that the developer has defined to be stable.
+
+Userspace programs are free to use these interfaces with no
+restrictions, and backward compatibility for them will be guaranteed
+for at least 2 years.
+
+Most interfaces (like syscalls) are expected to never change and always
+be available.
+
+.. kernel-abi:: $srctree/Documentation/ABI/stable
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
new file mode 100644
index 000000000000..5c886fc50b9e
--- /dev/null
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -0,0 +1,19 @@
+ABI testing symbols
+===================
+
+Documents interfaces that are felt to be stable,
+as the main development of this interface has been completed.
+
+The interface can be changed to add new features, but the
+current interface will not break by doing this, unless grave
+errors or security problems are found in them.
+
+Userspace programs can start to rely on these interfaces, but they must
+be aware of changes that can occur before these interfaces move to
+be marked stable.
+
+Programs that use these interfaces are strongly encouraged to add their
+name to the description of these interfaces, so that the kernel
+developers can easily notify them if any changes occur.
+
+.. kernel-abi:: $srctree/Documentation/ABI/testing
diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
new file mode 100644
index 000000000000..3b9645c77469
--- /dev/null
+++ b/Documentation/admin-guide/abi.rst
@@ -0,0 +1,11 @@
+=====================
+Linux ABI description
+=====================
+
+.. toctree::
+ :maxdepth: 1
+
+ abi-stable
+ abi-testing
+ abi-obsolete
+ abi-removed
diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
index 8001917ee012..20c3020fd73c 100644
--- a/Documentation/admin-guide/index.rst
+++ b/Documentation/admin-guide/index.rst
@@ -16,6 +16,7 @@ etc.
README
kernel-parameters
devices
+ abi
This section describes CPU vulnerabilities and their mitigations.
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 7ace3f8852bd..598256fb5c98 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ needs_sphinx = '1.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi']
# The name of the math extension changed on Sphinx 1.4
if (major == 1 and minor > 3) or (major > 1):
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
new file mode 100644
index 000000000000..32ce90775d96
--- /dev/null
+++ b/Documentation/sphinx/kernel_abi.py
@@ -0,0 +1,155 @@
+# -*- coding: utf-8; mode: python -*-
+u"""
+ kernel-abi
+ ~~~~~~~~~~
+
+ Implementation of the ``kernel-abi`` reST-directive.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
+ scripts/get_abi.pl script to parse the Kernel ABI files.
+
+ Overview of directive's argument and options.
+
+ .. code-block:: rst
+
+ .. kernel-abi:: <ABI directory location>
+ :debug:
+
+ The argument ``<ABI directory location>`` is required. It contains the
+ location of the ABI files to be parsed.
+
+ ``debug``
+ Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
+ what reST is generated.
+
+"""
+
+import sys
+import os
+from os import path
+import subprocess
+
+from sphinx.ext.autodoc import AutodocReporter
+
+from docutils import nodes
+from docutils.parsers.rst import Directive, directives
+from docutils.statemachine import ViewList
+from docutils.utils.error_reporting import ErrorString
+
+
+__version__ = '1.0'
+
+# We can't assume that six is installed
+PY3 = sys.version_info[0] == 3
+PY2 = sys.version_info[0] == 2
+if PY3:
+ # pylint: disable=C0103, W0622
+ unicode = str
+ basestring = str
+
+def setup(app):
+
+ app.add_directive("kernel-abi", KernelCmd)
+ return dict(
+ version = __version__
+ , parallel_read_safe = True
+ , parallel_write_safe = True
+ )
+
+class KernelCmd(Directive):
+
+ u"""KernelABI (``kernel-abi``) directive"""
+
+ required_arguments = 1
+ optional_arguments = 0
+ has_content = False
+ final_argument_whitespace = True
+
+ option_spec = {
+ "debug" : directives.flag
+ }
+
+ def warn(self, message, **replace):
+ replace["fname"] = self.state.document.current_source
+ replace["line_no"] = replace.get("line_no", self.lineno)
+ message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
+ self.state.document.settings.env.app.warn(message, prefix="")
+
+ def run(self):
+
+ doc = self.state.document
+ if not doc.settings.file_insertion_enabled:
+ raise self.warning("docutils: file insertion disabled")
+
+ env = doc.settings.env
+ cwd = path.dirname(doc.current_source)
+ cmd = "get_abi.pl rest --dir "
+ cmd += self.arguments[0]
+
+ srctree = path.abspath(os.environ["srctree"])
+
+ fname = cmd
+
+ # extend PATH with $(srctree)/scripts
+ path_env = os.pathsep.join([
+ srctree + os.sep + "scripts",
+ os.environ["PATH"]
+ ])
+ shell_env = os.environ.copy()
+ shell_env["PATH"] = path_env
+ shell_env["srctree"] = srctree
+
+ lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
+ nodeList = self.nestedParse(lines, fname)
+ return nodeList
+
+ def runCmd(self, cmd, **kwargs):
+ u"""Run command ``cmd`` and return it's stdout as unicode."""
+
+ try:
+ proc = subprocess.Popen(
+ cmd
+ , stdout = subprocess.PIPE
+ , stderr = subprocess.PIPE
+ , universal_newlines = True
+ , **kwargs
+ )
+ out, err = proc.communicate()
+ if err:
+ self.warn(err)
+ if proc.returncode != 0:
+ raise self.severe(
+ u"command '%s' failed with return code %d"
+ % (cmd, proc.returncode)
+ )
+ except OSError as exc:
+ raise self.severe(u"problems with '%s' directive: %s."
+ % (self.name, ErrorString(exc)))
+ return unicode(out)
+
+ def nestedParse(self, lines, fname):
+ content = ViewList()
+ node = nodes.section()
+
+ if "debug" in self.options:
+ code_block = "\n\n.. code-block:: rst\n :linenos:\n"
+ for l in lines.split("\n"):
+ code_block += "\n " + l
+ lines = code_block + "\n\n"
+
+ for c, l in enumerate(lines.split("\n")):
+ content.append(l, fname, c)
+
+ buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
+ self.state.memo.title_styles = []
+ self.state.memo.section_level = 0
+ self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
+ try:
+ self.state.nested_parse(content, 0, node, match_titles=1)
+ finally:
+ self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+ return node.children
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 13/14] sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (11 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support Mauro Carvalho Chehab
2019-06-14 16:20 ` [PATCH 00/14] Add support to generate ABI documentation at admin-guide Greg Kroah-Hartman
14 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
The same way kerneldoc.py needed changes to work with newer
Sphinx, this script needs the same changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 31 ++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 32ce90775d96..7fa7806532dc 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -33,7 +33,16 @@ import os
from os import path
import subprocess
-from sphinx.ext.autodoc import AutodocReporter
+#
+# AutodocReporter is only good up to Sphinx 1.7
+#
+import sphinx
+
+Use_SSI = sphinx.__version__[:3] >= '1.7'
+if Use_SSI:
+ from sphinx.util.docutils import switch_source_input
+else:
+ from sphinx.ext.autodoc import AutodocReporter
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@@ -145,11 +154,17 @@ class KernelCmd(Directive):
content.append(l, fname, c)
buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
- self.state.memo.title_styles = []
- self.state.memo.section_level = 0
- self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
- try:
- self.state.nested_parse(content, 0, node, match_titles=1)
- finally:
- self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+
+ if Use_SSI:
+ with switch_source_input(self.state, content):
+ self.state.nested_parse(content, 0, node, match_titles=1)
+ else:
+ self.state.memo.title_styles = []
+ self.state.memo.section_level = 0
+ self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
+ try:
+ self.state.nested_parse(content, 0, node, match_titles=1)
+ finally:
+ self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+
return node.children
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (12 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 13/14] sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+ Mauro Carvalho Chehab
@ 2019-06-14 2:04 ` Mauro Carvalho Chehab
2019-06-14 16:18 ` Greg Kroah-Hartman
2019-06-14 16:20 ` [PATCH 00/14] Add support to generate ABI documentation at admin-guide Greg Kroah-Hartman
14 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 2:04 UTC (permalink / raw)
To: Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
The parser breaks with UTF-8 characters with Sphinx 1.4.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 7fa7806532dc..460cee48a245 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -1,4 +1,5 @@
-# -*- coding: utf-8; mode: python -*-
+# coding=utf-8
+#
u"""
kernel-abi
~~~~~~~~~~
@@ -28,6 +29,7 @@ u"""
"""
+import codecs
import sys
import os
from os import path
@@ -124,12 +126,12 @@ class KernelCmd(Directive):
cmd
, stdout = subprocess.PIPE
, stderr = subprocess.PIPE
- , universal_newlines = True
, **kwargs
)
out, err = proc.communicate()
- if err:
- self.warn(err)
+
+ out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
+
if proc.returncode != 0:
raise self.severe(
u"command '%s' failed with return code %d"
--
2.21.0
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH 01/14] ABI: fix some syntax issues at the ABI database
2019-06-14 2:04 ` [PATCH 01/14] ABI: fix some syntax issues at the ABI database Mauro Carvalho Chehab
@ 2019-06-14 7:20 ` Andrew Donnellan
2019-06-14 16:16 ` Greg Kroah-Hartman
2019-06-14 8:30 ` Rafael J. Wysocki
1 sibling, 1 reply; 63+ messages in thread
From: Andrew Donnellan @ 2019-06-14 7:20 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Andreas Klinger, Jonathan Cameron,
Hartmut Knaack, Lars-Peter Clausen, Peter Meerwald-Stadler,
Frederic Barrat, Sebastian Reichel, Kees Cook, Anton Vorontsov,
Colin Cross, Tony Luck, linux-iio, linuxppc-dev, linux-pm
On 14/6/19 12:04 pm, Mauro Carvalho Chehab wrote:
> diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
> index bbbabffc682a..fc7c6f7c21b3 100644
> --- a/Documentation/ABI/testing/sysfs-class-cxl
> +++ b/Documentation/ABI/testing/sysfs-class-cxl
> @@ -1,6 +1,6 @@
> -Note: Attributes that are shared between devices are stored in the directory
> -pointed to by the symlink device/.
> -Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> +Please notice that attributes that are shared between devices are stored in
Would prefer "Please note" over "Please notice".
Acked-by: Andrew Donnellan <ajd@linux.ibm.com> # cxl
--
Andrew Donnellan OzLabs, ADL Canberra
ajd@linux.ibm.com IBM Australia Limited
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 01/14] ABI: fix some syntax issues at the ABI database
2019-06-14 2:04 ` [PATCH 01/14] ABI: fix some syntax issues at the ABI database Mauro Carvalho Chehab
2019-06-14 7:20 ` Andrew Donnellan
@ 2019-06-14 8:30 ` Rafael J. Wysocki
1 sibling, 0 replies; 63+ messages in thread
From: Rafael J. Wysocki @ 2019-06-14 8:30 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Greg Kroah-Hartman,
Mauro Carvalho Chehab, Mauro Carvalho Chehab,
Linux Kernel Mailing List, Jonathan Corbet, Andreas Klinger,
Jonathan Cameron, Hartmut Knaack, Lars-Peter Clausen,
Peter Meerwald-Stadler, Frederic Barrat, Andrew Donnellan,
Sebastian Reichel, Kees Cook, Anton Vorontsov, Colin Cross,
Tony Luck, linux-iio, linuxppc-dev, Linux PM
On Fri, Jun 14, 2019 at 4:04 AM Mauro Carvalho Chehab
<mchehab+samsung@kernel.org> wrote:
>
> From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
>
> On those three files, the ABI representation described at
> README are violated.
>
> - at sysfs-bus-iio-proximity-as3935:
> a ':' character is missing after "What"
>
> - at sysfs-class-devfreq:
> there's a typo at Description
>
> - at sysfs-class-cxl, it is using the ":" character at a
> file preamble, causing it to be misinterpreted as a
> tag.
>
> - On the other files, instead of "What", they use "Where".
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> ---
> Documentation/ABI/testing/pstore | 2 +-
> .../sysfs-bus-event_source-devices-format | 2 +-
> .../ABI/testing/sysfs-bus-i2c-devices-hm6352 | 6 ++---
> .../ABI/testing/sysfs-bus-iio-distance-srf08 | 4 ++--
> .../testing/sysfs-bus-iio-proximity-as3935 | 4 ++--
> .../ABI/testing/sysfs-bus-pci-devices-cciss | 22 +++++++++----------
> .../testing/sysfs-bus-usb-devices-usbsevseg | 12 +++++-----
> Documentation/ABI/testing/sysfs-class-cxl | 6 ++---
> Documentation/ABI/testing/sysfs-class-devfreq | 2 +-
> .../ABI/testing/sysfs-class-powercap | 2 +-
> Documentation/ABI/testing/sysfs-kernel-fscaps | 2 +-
> .../ABI/testing/sysfs-kernel-vmcoreinfo | 2 +-
> 12 files changed, 33 insertions(+), 33 deletions(-)
>
> diff --git a/Documentation/ABI/testing/pstore b/Documentation/ABI/testing/pstore
> index 5fca9f5e10a3..8d6e48f4e8ef 100644
> --- a/Documentation/ABI/testing/pstore
> +++ b/Documentation/ABI/testing/pstore
> @@ -1,4 +1,4 @@
> -Where: /sys/fs/pstore/... (or /dev/pstore/...)
> +What: /sys/fs/pstore/... (or /dev/pstore/...)
> Date: March 2011
> Kernel Version: 2.6.39
> Contact: tony.luck@intel.com
> diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
> index 77f47ff5ee02..b6f8748e0200 100644
> --- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
> +++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
> @@ -1,4 +1,4 @@
> -Where: /sys/bus/event_source/devices/<dev>/format
> +What: /sys/bus/event_source/devices/<dev>/format
> Date: January 2012
> Kernel Version: 3.3
> Contact: Jiri Olsa <jolsa@redhat.com>
> diff --git a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
> index feb2e4a87075..29bd447e50a0 100644
> --- a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
> +++ b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
> @@ -1,18 +1,18 @@
> -Where: /sys/bus/i2c/devices/.../heading0_input
> +What: /sys/bus/i2c/devices/.../heading0_input
> Date: April 2010
> Kernel Version: 2.6.36?
> Contact: alan.cox@intel.com
> Description: Reports the current heading from the compass as a floating
> point value in degrees.
>
> -Where: /sys/bus/i2c/devices/.../power_state
> +What: /sys/bus/i2c/devices/.../power_state
> Date: April 2010
> Kernel Version: 2.6.36?
> Contact: alan.cox@intel.com
> Description: Sets the power state of the device. 0 sets the device into
> sleep mode, 1 wakes it up.
>
> -Where: /sys/bus/i2c/devices/.../calibration
> +What: /sys/bus/i2c/devices/.../calibration
> Date: April 2010
> Kernel Version: 2.6.36?
> Contact: alan.cox@intel.com
> diff --git a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08 b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
> index 0a1ca1487fa9..a133fd8d081a 100644
> --- a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
> +++ b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
> @@ -1,4 +1,4 @@
> -What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> +What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> Date: January 2017
> KernelVersion: 4.11
> Contact: linux-iio@vger.kernel.org
> @@ -6,7 +6,7 @@ Description:
> Show or set the gain boost of the amp, from 0-31 range.
> default 31
>
> -What /sys/bus/iio/devices/iio:deviceX/sensor_max_range
> +What: /sys/bus/iio/devices/iio:deviceX/sensor_max_range
> Date: January 2017
> KernelVersion: 4.11
> Contact: linux-iio@vger.kernel.org
> diff --git a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
> index 9a17ab5036a4..c59d95346341 100644
> --- a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
> +++ b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
> @@ -1,4 +1,4 @@
> -What /sys/bus/iio/devices/iio:deviceX/in_proximity_input
> +What: /sys/bus/iio/devices/iio:deviceX/in_proximity_input
> Date: March 2014
> KernelVersion: 3.15
> Contact: Matt Ranostay <matt.ranostay@konsulko.com>
> @@ -6,7 +6,7 @@ Description:
> Get the current distance in meters of storm (1km steps)
> 1000-40000 = distance in meters
>
> -What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> +What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> Date: March 2014
> KernelVersion: 3.15
> Contact: Matt Ranostay <matt.ranostay@konsulko.com>
> diff --git a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
> index 53d99edd1d75..eb449169c30b 100644
> --- a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
> +++ b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
> @@ -1,66 +1,66 @@
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: iss_storagedev@hp.com
> Description: Displays the SCSI INQUIRY page 0 model for logical drive
> Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: iss_storagedev@hp.com
> Description: Displays the SCSI INQUIRY page 0 revision for logical
> drive Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: iss_storagedev@hp.com
> Description: Displays the SCSI INQUIRY page 83 serial number for logical
> drive Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: iss_storagedev@hp.com
> Description: Displays the SCSI INQUIRY page 0 vendor for logical drive
> Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: iss_storagedev@hp.com
> Description: A symbolic link to /sys/block/cciss!cXdY
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/rescan
> +What: /sys/bus/pci/devices/<dev>/ccissX/rescan
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: iss_storagedev@hp.com
> Description: Kicks of a rescan of the controller to discover logical
> drive topology changes.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: iss_storagedev@hp.com
> Description: Displays the 8-byte LUN ID used to address logical
> drive Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: iss_storagedev@hp.com
> Description: Displays the RAID level of logical drive Y of
> controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: iss_storagedev@hp.com
> Description: Displays the usage count (number of opens) of logical drive Y
> of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/resettable
> +What: /sys/bus/pci/devices/<dev>/ccissX/resettable
> Date: February 2011
> Kernel Version: 2.6.38
> Contact: iss_storagedev@hp.com
> @@ -71,7 +71,7 @@ Description: Value of 1 indicates the controller can honor the reset_devices
> a dump device, as kdump requires resetting the device in order
> to work reliably.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
> +What: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
> Date: July 2011
> Kernel Version: 3.0
> Contact: iss_storagedev@hp.com
> diff --git a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
> index 70d00dfa443d..f6199b314196 100644
> --- a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
> +++ b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
> @@ -1,12 +1,12 @@
> -Where: /sys/bus/usb/.../powered
> +What: /sys/bus/usb/.../powered
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <harrisonmetz@gmail.com>
> Description: Controls whether the device's display will powered.
> A value of 0 is off and a non-zero value is on.
>
> -Where: /sys/bus/usb/.../mode_msb
> -Where: /sys/bus/usb/.../mode_lsb
> +What: /sys/bus/usb/.../mode_msb
> +What: /sys/bus/usb/.../mode_lsb
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <harrisonmetz@gmail.com>
> @@ -16,7 +16,7 @@ Description: Controls the devices display mode.
> for an 8 character display the values are
> MSB 0x08; LSB 0xFF.
>
> -Where: /sys/bus/usb/.../textmode
> +What: /sys/bus/usb/.../textmode
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <harrisonmetz@gmail.com>
> @@ -25,13 +25,13 @@ Description: Controls the way the device interprets its text buffer.
> hex: each character is between 0-15
> ascii: each character is between '0'-'9' and 'A'-'F'.
>
> -Where: /sys/bus/usb/.../text
> +What: /sys/bus/usb/.../text
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <harrisonmetz@gmail.com>
> Description: The text (or data) for the device to display
>
> -Where: /sys/bus/usb/.../decimals
> +What: /sys/bus/usb/.../decimals
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <harrisonmetz@gmail.com>
> diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
> index bbbabffc682a..fc7c6f7c21b3 100644
> --- a/Documentation/ABI/testing/sysfs-class-cxl
> +++ b/Documentation/ABI/testing/sysfs-class-cxl
> @@ -1,6 +1,6 @@
> -Note: Attributes that are shared between devices are stored in the directory
> -pointed to by the symlink device/.
> -Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> +Please notice that attributes that are shared between devices are stored in
> +the directory pointed to by the symlink device/.
> +For example, the real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> /sys/class/cxl/afu0.0s/device/irqs_max, i.e. /sys/class/cxl/afu0.0/irqs_max.
>
>
> diff --git a/Documentation/ABI/testing/sysfs-class-devfreq b/Documentation/ABI/testing/sysfs-class-devfreq
> index ee39acacf6f8..01196e19afca 100644
> --- a/Documentation/ABI/testing/sysfs-class-devfreq
> +++ b/Documentation/ABI/testing/sysfs-class-devfreq
> @@ -47,7 +47,7 @@ Description:
> What: /sys/class/devfreq/.../trans_stat
> Date: October 2012
> Contact: MyungJoo Ham <myungjoo.ham@samsung.com>
> -Descrtiption:
> +Description:
> This ABI shows the statistics of devfreq behavior on a
> specific device. It shows the time spent in each state and
> the number of transitions between states.
> diff --git a/Documentation/ABI/testing/sysfs-class-powercap b/Documentation/ABI/testing/sysfs-class-powercap
> index db3b3ff70d84..f333a0ccc29b 100644
> --- a/Documentation/ABI/testing/sysfs-class-powercap
> +++ b/Documentation/ABI/testing/sysfs-class-powercap
> @@ -147,6 +147,6 @@ What: /sys/class/powercap/.../<power zone>/enabled
> Date: September 2013
> KernelVersion: 3.13
> Contact: linux-pm@vger.kernel.org
> -Description
> +Description:
> This allows to enable/disable power capping at power zone level.
> This applies to current power zone and its children.
> diff --git a/Documentation/ABI/testing/sysfs-kernel-fscaps b/Documentation/ABI/testing/sysfs-kernel-fscaps
> index 50a3033b5e15..bcff34665192 100644
> --- a/Documentation/ABI/testing/sysfs-kernel-fscaps
> +++ b/Documentation/ABI/testing/sysfs-kernel-fscaps
> @@ -2,7 +2,7 @@ What: /sys/kernel/fscaps
> Date: February 2011
> KernelVersion: 2.6.38
> Contact: Ludwig Nussel <ludwig.nussel@suse.de>
> -Description
> +Description:
> Shows whether file system capabilities are honored
> when executing a binary
>
> diff --git a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> index 7bd81168e063..1f1087a5f075 100644
> --- a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> +++ b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> @@ -4,7 +4,7 @@ KernelVersion: 2.6.24
> Contact: Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp>
> Kexec Mailing List <kexec@lists.infradead.org>
> Vivek Goyal <vgoyal@redhat.com>
> -Description
> +Description:
> Shows physical address and size of vmcoreinfo ELF note.
> First value contains physical address of note in hex and
> second value contains the size of note in hex. This ELF
> --
> 2.21.0
>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 05/14] scripts: add an script to parse the ABI files
2019-06-14 2:04 ` [PATCH 05/14] scripts: add an script to parse the ABI files Mauro Carvalho Chehab
@ 2019-06-14 13:31 ` Jani Nikula
2019-06-14 13:39 ` Greg Kroah-Hartman
0 siblings, 1 reply; 63+ messages in thread
From: Jani Nikula @ 2019-06-14 13:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
>
> Add a script to parse the Documentation/ABI files and produce
> an output with all entries inside an ABI (sub)directory.
>
> Right now, it outputs its contents on ReST format. It shouldn't
> be hard to make it produce other kind of outputs, since the ABI
> file parser is implemented in separate than the output generator.
Hum, or just convert the ABI files to rst directly.
BR,
Jani.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> scripts/get_abi.pl | 212 +++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 212 insertions(+)
> create mode 100755 scripts/get_abi.pl
>
> diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
> new file mode 100755
> index 000000000000..f7c9944a833c
> --- /dev/null
> +++ b/scripts/get_abi.pl
> @@ -0,0 +1,212 @@
> +#!/usr/bin/perl
> +
> +use strict;
> +use Pod::Usage;
> +use Getopt::Long;
> +use File::Find;
> +use Fcntl ':mode';
> +
> +my $help;
> +my $man;
> +my $debug;
> +
> +GetOptions(
> + "debug|d+" => \$debug,
> + 'help|?' => \$help,
> + man => \$man
> +) or pod2usage(2);
> +
> +pod2usage(1) if $help;
> +pod2usage(-exitstatus => 0, -verbose => 2) if $man;
> +
> +pod2usage(2) if (scalar @ARGV != 1);
> +
> +my ($prefix) = @ARGV;
> +
> +require Data::Dumper if ($debug);
> +
> +my %data;
> +
> +#
> +# Displays an error message, printing file name and line
> +#
> +sub parse_error($$$$) {
> + my ($file, $ln, $msg, $data) = @_;
> +
> + print STDERR "file $file#$ln: $msg at\n\t$data";
> +}
> +
> +#
> +# Parse an ABI file, storing its contents at %data
> +#
> +sub parse_abi {
> + my $file = $File::Find::name;
> +
> + my $mode = (stat($file))[2];
> + return if ($mode & S_IFDIR);
> + return if ($file =~ m,/README,);
> +
> + my $name = $file;
> + $name =~ s,.*/,,;
> +
> + my $type = $file;
> + $type =~ s,.*/(.*)/.*,$1,;
> +
> + my $what;
> + my $new_what;
> + my $tag;
> + my $ln;
> +
> + print STDERR "Opening $file\n" if ($debug > 1);
> + open IN, $file;
> + while(<IN>) {
> + $ln++;
> + if (m/^(\S+):\s*(.*)/i) {
> + my $new_tag = lc($1);
> + my $content = $2;
> +
> + if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
> + if ($tag eq "description") {
> + $data{$what}->{$tag} .= "\n$content";;
> + $data{$what}->{$tag} =~ s/\n+$//;
> + next;
> + } else {
> + parse_error($file, $ln, "tag '$tag' is invalid", $_);
> + }
> + }
> +
> + if ($new_tag =~ m/what/) {
> + if ($tag =~ m/what/) {
> + $what .= ", " . $content;
> + } else {
> + $what = $content;
> + $new_what = 1;
> + }
> + $tag = $new_tag;
> + next;
> + }
> +
> + $tag = $new_tag;
> +
> + if ($new_what) {
> + $new_what = 0;
> +
> + $data{$what}->{type} = $type;
> + $data{$what}->{file} = $name;
> + print STDERR "\twhat: $what\n" if ($debug > 1);
> + }
> +
> + if (!$what) {
> + parse_error($file, $ln, "'What:' should come first:", $_);
> + next;
> + }
> + $data{$what}->{$tag} = $content;
> + next;
> + }
> +
> + # Silently ignore any headers before the database
> + next if (!$tag);
> +
> + if (m/^\s*(.*)/) {
> + $data{$what}->{$tag} .= "\n$1";
> + $data{$what}->{$tag} =~ s/\n+$//;
> + next;
> + }
> +
> + # Everything else is error
> + parse_error($file, $ln, "Unexpected line:", $_);
> + }
> + close IN;
> +}
> +
> +# Outputs the output on ReST format
> +sub output_rest {
> + foreach my $what (sort keys %data) {
> + my $type = $data{$what}->{type};
> + my $file = $data{$what}->{file};
> +
> + my $w = $what;
> + $w =~ s/([\(\)\_\-\*\=\^\~\\])/\\$1/g;
> +
> + print "$w\n\n";
> + print "- defined on file $file (type: $type)\n\n::\n\n";
> +
> + my $desc = $data{$what}->{description};
> + $desc =~ s/^\s+//;
> +
> + # Remove title markups from the description, as they won't work
> + $desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
> +
> + # put everything inside a code block
> + $desc =~ s/\n/\n /g;
> +
> +
> + if (!($desc =~ /^\s*$/)) {
> + print " $desc\n\n";
> + } else {
> + print " DESCRIPTION MISSING\n\n";
> + }
> + }
> +}
> +
> +#
> +# Parses all ABI files located at $prefix dir
> +#
> +find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
> +
> +print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
> +
> +#
> +# Outputs an ReST file with the ABI contents
> +#
> +output_rest
> +
> +
> +__END__
> +
> +=head1 NAME
> +
> +abi_book.pl - parse the Linux ABI files and produce a ReST book.
> +
> +=head1 SYNOPSIS
> +
> +B<abi_book.pl> [--debug] <ABI_DIR>]
> +
> +=head1 OPTIONS
> +
> +=over 8
> +
> +=item B<--debug>
> +
> +Put the script in verbose mode, useful for debugging. Can be called multiple
> +times, to increase verbosity.
> +
> +=item B<--help>
> +
> +Prints a brief help message and exits.
> +
> +=item B<--man>
> +
> +Prints the manual page and exits.
> +
> +=back
> +
> +=head1 DESCRIPTION
> +
> +Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
> +and produce a ReST book containing the Linux ABI.
> +
> +=head1 BUGS
> +
> +Report bugs to Mauro Carvalho Chehab <mchehab@s-opensource.com>
> +
> +=head1 COPYRIGHT
> +
> +Copyright (c) 2016 by Mauro Carvalho Chehab <mchehab@s-opensource.com>.
> +
> +License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
> +
> +This is free software: you are free to change and redistribute it.
> +There is NO WARRANTY, to the extent permitted by law.
> +
> +=cut
--
Jani Nikula, Intel Open Source Graphics Center
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 05/14] scripts: add an script to parse the ABI files
2019-06-14 13:31 ` Jani Nikula
@ 2019-06-14 13:39 ` Greg Kroah-Hartman
2019-06-14 13:58 ` Mauro Carvalho Chehab
2019-06-14 14:00 ` Jani Nikula
0 siblings, 2 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-14 13:39 UTC (permalink / raw)
To: Jani Nikula
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> >
> > Add a script to parse the Documentation/ABI files and produce
> > an output with all entries inside an ABI (sub)directory.
> >
> > Right now, it outputs its contents on ReST format. It shouldn't
> > be hard to make it produce other kind of outputs, since the ABI
> > file parser is implemented in separate than the output generator.
>
> Hum, or just convert the ABI files to rst directly.
And what would that look like?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 2:04 ` [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book Mauro Carvalho Chehab
@ 2019-06-14 13:42 ` Jani Nikula
2019-06-14 14:06 ` Greg Kroah-Hartman
2019-06-14 14:10 ` Markus Heiser
0 siblings, 2 replies; 63+ messages in thread
From: Jani Nikula @ 2019-06-14 13:42 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Linux Doc Mailing List, Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Mauro Carvalho Chehab
On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
>
> As we don't want a generic Sphinx extension to execute commands,
> change the one proposed to Markus to call the abi_book.pl
> script.
>
> Use a script to parse the Documentation/ABI directory and output
> it at the admin-guide.
We had a legacy kernel-doc perl script so we used that instead of
rewriting it in python. Just to keep it bug-for-bug compatible with the
past. That was the only reason.
I see absolutely zero reason to add a new perl monstrosity with a python
extension to call it. All of this could be better done in python,
directly.
Please don't complicate the documentation build. I know you know we all
worked hard to take apart the old DocBook Rube Goldberg machine to
replace it with Sphinx. Please don't turn the Sphinx build to another
complicated mess.
My strong preferences are, in this order:
1) Convert the ABI documentation to reStructuredText
2) Have the python extension read the ABI files directly, without an
extra pipeline.
BR,
Jani.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> Documentation/admin-guide/abi-obsolete.rst | 10 ++
> Documentation/admin-guide/abi-removed.rst | 4 +
> Documentation/admin-guide/abi-stable.rst | 13 ++
> Documentation/admin-guide/abi-testing.rst | 19 +++
> Documentation/admin-guide/abi.rst | 11 ++
> Documentation/admin-guide/index.rst | 1 +
> Documentation/conf.py | 2 +-
> Documentation/sphinx/kernel_abi.py | 155 +++++++++++++++++++++
> 8 files changed, 214 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/admin-guide/abi-obsolete.rst
> create mode 100644 Documentation/admin-guide/abi-removed.rst
> create mode 100644 Documentation/admin-guide/abi-stable.rst
> create mode 100644 Documentation/admin-guide/abi-testing.rst
> create mode 100644 Documentation/admin-guide/abi.rst
> create mode 100644 Documentation/sphinx/kernel_abi.py
>
> diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
> new file mode 100644
> index 000000000000..cda9168445a5
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-obsolete.rst
> @@ -0,0 +1,10 @@
> +ABI obsolete symbols
> +====================
> +
> +Documents interfaces that are still remaining in the kernel, but are
> +marked to be removed at some later point in time.
> +
> +The description of the interface will document the reason why it is
> +obsolete and when it can be expected to be removed.
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/obsolete
> diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
> new file mode 100644
> index 000000000000..497978fc9632
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-removed.rst
> @@ -0,0 +1,4 @@
> +ABI removed symbols
> +===================
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/removed
> diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
> new file mode 100644
> index 000000000000..7495d7a35048
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-stable.rst
> @@ -0,0 +1,13 @@
> +ABI stable symbols
> +==================
> +
> +Documents the interfaces that the developer has defined to be stable.
> +
> +Userspace programs are free to use these interfaces with no
> +restrictions, and backward compatibility for them will be guaranteed
> +for at least 2 years.
> +
> +Most interfaces (like syscalls) are expected to never change and always
> +be available.
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/stable
> diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
> new file mode 100644
> index 000000000000..5c886fc50b9e
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-testing.rst
> @@ -0,0 +1,19 @@
> +ABI testing symbols
> +===================
> +
> +Documents interfaces that are felt to be stable,
> +as the main development of this interface has been completed.
> +
> +The interface can be changed to add new features, but the
> +current interface will not break by doing this, unless grave
> +errors or security problems are found in them.
> +
> +Userspace programs can start to rely on these interfaces, but they must
> +be aware of changes that can occur before these interfaces move to
> +be marked stable.
> +
> +Programs that use these interfaces are strongly encouraged to add their
> +name to the description of these interfaces, so that the kernel
> +developers can easily notify them if any changes occur.
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/testing
> diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
> new file mode 100644
> index 000000000000..3b9645c77469
> --- /dev/null
> +++ b/Documentation/admin-guide/abi.rst
> @@ -0,0 +1,11 @@
> +=====================
> +Linux ABI description
> +=====================
> +
> +.. toctree::
> + :maxdepth: 1
> +
> + abi-stable
> + abi-testing
> + abi-obsolete
> + abi-removed
> diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
> index 8001917ee012..20c3020fd73c 100644
> --- a/Documentation/admin-guide/index.rst
> +++ b/Documentation/admin-guide/index.rst
> @@ -16,6 +16,7 @@ etc.
> README
> kernel-parameters
> devices
> + abi
>
> This section describes CPU vulnerabilities and their mitigations.
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index 7ace3f8852bd..598256fb5c98 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -34,7 +34,7 @@ needs_sphinx = '1.3'
> # Add any Sphinx extension module names here, as strings. They can be
> # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
> # ones.
> -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig']
> +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi']
>
> # The name of the math extension changed on Sphinx 1.4
> if (major == 1 and minor > 3) or (major > 1):
> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> new file mode 100644
> index 000000000000..32ce90775d96
> --- /dev/null
> +++ b/Documentation/sphinx/kernel_abi.py
> @@ -0,0 +1,155 @@
> +# -*- coding: utf-8; mode: python -*-
> +u"""
> + kernel-abi
> + ~~~~~~~~~~
> +
> + Implementation of the ``kernel-abi`` reST-directive.
> +
> + :copyright: Copyright (C) 2016 Markus Heiser
> + :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
> + :license: GPL Version 2, June 1991 see Linux/COPYING for details.
> +
> + The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
> + scripts/get_abi.pl script to parse the Kernel ABI files.
> +
> + Overview of directive's argument and options.
> +
> + .. code-block:: rst
> +
> + .. kernel-abi:: <ABI directory location>
> + :debug:
> +
> + The argument ``<ABI directory location>`` is required. It contains the
> + location of the ABI files to be parsed.
> +
> + ``debug``
> + Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
> + what reST is generated.
> +
> +"""
> +
> +import sys
> +import os
> +from os import path
> +import subprocess
> +
> +from sphinx.ext.autodoc import AutodocReporter
> +
> +from docutils import nodes
> +from docutils.parsers.rst import Directive, directives
> +from docutils.statemachine import ViewList
> +from docutils.utils.error_reporting import ErrorString
> +
> +
> +__version__ = '1.0'
> +
> +# We can't assume that six is installed
> +PY3 = sys.version_info[0] == 3
> +PY2 = sys.version_info[0] == 2
> +if PY3:
> + # pylint: disable=C0103, W0622
> + unicode = str
> + basestring = str
> +
> +def setup(app):
> +
> + app.add_directive("kernel-abi", KernelCmd)
> + return dict(
> + version = __version__
> + , parallel_read_safe = True
> + , parallel_write_safe = True
> + )
> +
> +class KernelCmd(Directive):
> +
> + u"""KernelABI (``kernel-abi``) directive"""
> +
> + required_arguments = 1
> + optional_arguments = 0
> + has_content = False
> + final_argument_whitespace = True
> +
> + option_spec = {
> + "debug" : directives.flag
> + }
> +
> + def warn(self, message, **replace):
> + replace["fname"] = self.state.document.current_source
> + replace["line_no"] = replace.get("line_no", self.lineno)
> + message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
> + self.state.document.settings.env.app.warn(message, prefix="")
> +
> + def run(self):
> +
> + doc = self.state.document
> + if not doc.settings.file_insertion_enabled:
> + raise self.warning("docutils: file insertion disabled")
> +
> + env = doc.settings.env
> + cwd = path.dirname(doc.current_source)
> + cmd = "get_abi.pl rest --dir "
> + cmd += self.arguments[0]
> +
> + srctree = path.abspath(os.environ["srctree"])
> +
> + fname = cmd
> +
> + # extend PATH with $(srctree)/scripts
> + path_env = os.pathsep.join([
> + srctree + os.sep + "scripts",
> + os.environ["PATH"]
> + ])
> + shell_env = os.environ.copy()
> + shell_env["PATH"] = path_env
> + shell_env["srctree"] = srctree
> +
> + lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
> + nodeList = self.nestedParse(lines, fname)
> + return nodeList
> +
> + def runCmd(self, cmd, **kwargs):
> + u"""Run command ``cmd`` and return it's stdout as unicode."""
> +
> + try:
> + proc = subprocess.Popen(
> + cmd
> + , stdout = subprocess.PIPE
> + , stderr = subprocess.PIPE
> + , universal_newlines = True
> + , **kwargs
> + )
> + out, err = proc.communicate()
> + if err:
> + self.warn(err)
> + if proc.returncode != 0:
> + raise self.severe(
> + u"command '%s' failed with return code %d"
> + % (cmd, proc.returncode)
> + )
> + except OSError as exc:
> + raise self.severe(u"problems with '%s' directive: %s."
> + % (self.name, ErrorString(exc)))
> + return unicode(out)
> +
> + def nestedParse(self, lines, fname):
> + content = ViewList()
> + node = nodes.section()
> +
> + if "debug" in self.options:
> + code_block = "\n\n.. code-block:: rst\n :linenos:\n"
> + for l in lines.split("\n"):
> + code_block += "\n " + l
> + lines = code_block + "\n\n"
> +
> + for c, l in enumerate(lines.split("\n")):
> + content.append(l, fname, c)
> +
> + buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
> + self.state.memo.title_styles = []
> + self.state.memo.section_level = 0
> + self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
> + try:
> + self.state.nested_parse(content, 0, node, match_titles=1)
> + finally:
> + self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
> + return node.children
--
Jani Nikula, Intel Open Source Graphics Center
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 05/14] scripts: add an script to parse the ABI files
2019-06-14 13:39 ` Greg Kroah-Hartman
@ 2019-06-14 13:58 ` Mauro Carvalho Chehab
2019-06-14 14:00 ` Jani Nikula
1 sibling, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 13:58 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Jani Nikula, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet
Em Fri, 14 Jun 2019 15:39:33 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> > On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > >
> > > Add a script to parse the Documentation/ABI files and produce
> > > an output with all entries inside an ABI (sub)directory.
> > >
> > > Right now, it outputs its contents on ReST format. It shouldn't
> > > be hard to make it produce other kind of outputs, since the ABI
> > > file parser is implemented in separate than the output generator.
> >
> > Hum, or just convert the ABI files to rst directly.
Converting ABI files to rst could be easily done using a modified
version of my script (plus the produced files will likely need some
manual review).
There is a drawback though: we'll lose the capability of being able
to parse ABI files via an script, as people would be freed to use a
different syntax.
We could minimize it by using things like:
:What:
:Kernel Version:
...
> And what would that look like?
That's the big question :-)
If you prefer something like that, I can write the patches.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 05/14] scripts: add an script to parse the ABI files
2019-06-14 13:39 ` Greg Kroah-Hartman
2019-06-14 13:58 ` Mauro Carvalho Chehab
@ 2019-06-14 14:00 ` Jani Nikula
2019-06-14 14:14 ` Jonathan Corbet
2019-06-14 16:24 ` Greg Kroah-Hartman
1 sibling, 2 replies; 63+ messages in thread
From: Jani Nikula @ 2019-06-14 14:00 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Fri, 14 Jun 2019, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
>> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
>> > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
>> >
>> > Add a script to parse the Documentation/ABI files and produce
>> > an output with all entries inside an ABI (sub)directory.
>> >
>> > Right now, it outputs its contents on ReST format. It shouldn't
>> > be hard to make it produce other kind of outputs, since the ABI
>> > file parser is implemented in separate than the output generator.
>>
>> Hum, or just convert the ABI files to rst directly.
>
> And what would that look like?
That pretty much depends on the requirements we want to set on both the
ABI source files and the generated output. Obviously the requirements
can be conflicting; might be hard to produce fancy output if the input
is very limited.
At the bare minimum, you could convert the files to contain
reStructuredText field lists [1]. Add a colon at the start of the field
name, and make sure field bodies (values) are not empty.
Conversion of a file selected at random; I've only added ":" and "N/A".
diff --git a/Documentation/ABI/stable/sysfs-devices-system-cpu b/Documentation/ABI/stable/sysfs-devices-system-cpu
index 33c133e2a631..34c218b344fb 100644
--- a/Documentation/ABI/stable/sysfs-devices-system-cpu
+++ b/Documentation/ABI/stable/sysfs-devices-system-cpu
@@ -1,19 +1,20 @@
-What: /sys/devices/system/cpu/dscr_default
-Date: 13-May-2014
-KernelVersion: v3.15.0
-Contact:
-Description: Writes are equivalent to writing to
+:What: /sys/devices/system/cpu/dscr_default
+:Date: 13-May-2014
+:KernelVersion: v3.15.0
+:Contact: N/A
+:Description: Writes are equivalent to writing to
/sys/devices/system/cpu/cpuN/dscr on all CPUs.
Reads return the last written value or 0.
This value is not a global default: it is a way to set
all per-CPU defaults at the same time.
-Values: 64 bit unsigned integer (bit field)
+:Values: 64 bit unsigned integer (bit field)
-What: /sys/devices/system/cpu/cpu[0-9]+/dscr
-Date: 13-May-2014
-KernelVersion: v3.15.0
-Contact:
-Description: Default value for the Data Stream Control Register (DSCR) on
+
+:What: /sys/devices/system/cpu/cpu[0-9]+/dscr
+:Date: 13-May-2014
+:KernelVersion: v3.15.0
+:Contact: N/A
+:Description: Default value for the Data Stream Control Register (DSCR) on
a CPU.
This default value is used when the kernel is executing and
for any process that has not set the DSCR itself.
@@ -22,4 +23,4 @@ Description: Default value for the Data Stream Control Register (DSCR) on
on any CPU where it executes (overriding the value described
here).
If set by a process it will be inherited by child processes.
-Values: 64 bit unsigned integer (bit field)
+:Values: 64 bit unsigned integer (bit field)
---
Of course, you'd still need to add higher level files to include the ABI
files.
At the other end, you could add structure and syntax to your heart's
content, and make the output fancier too.
BR,
Jani.
[1] http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#field-lists
--
Jani Nikula, Intel Open Source Graphics Center
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 13:42 ` Jani Nikula
@ 2019-06-14 14:06 ` Greg Kroah-Hartman
2019-06-14 15:27 ` Mauro Carvalho Chehab
2019-06-14 14:10 ` Markus Heiser
1 sibling, 1 reply; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-14 14:06 UTC (permalink / raw)
To: Jani Nikula
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> >
> > As we don't want a generic Sphinx extension to execute commands,
> > change the one proposed to Markus to call the abi_book.pl
> > script.
> >
> > Use a script to parse the Documentation/ABI directory and output
> > it at the admin-guide.
>
> We had a legacy kernel-doc perl script so we used that instead of
> rewriting it in python. Just to keep it bug-for-bug compatible with the
> past. That was the only reason.
>
> I see absolutely zero reason to add a new perl monstrosity with a python
> extension to call it. All of this could be better done in python,
> directly.
>
> Please don't complicate the documentation build. I know you know we all
> worked hard to take apart the old DocBook Rube Goldberg machine to
> replace it with Sphinx. Please don't turn the Sphinx build to another
> complicated mess.
>
> My strong preferences are, in this order:
>
> 1) Convert the ABI documentation to reStructuredText
What would that exactly look like? What would it require for new
developers for when they write new entries? Why not rely on a helper
script, that allows us to validate things better?
> 2) Have the python extension read the ABI files directly, without an
> extra pipeline.
He who writes the script, get's to dictate the language of the script :)
Personally, this looks sane to me, I'm going to apply the ABI fixups to
my tree at least, and then see how the script works out. The script can
always be replaced with a different one in a different language at a
later point in time of people think it really mattes.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 13:42 ` Jani Nikula
2019-06-14 14:06 ` Greg Kroah-Hartman
@ 2019-06-14 14:10 ` Markus Heiser
2019-06-14 14:15 ` Jonathan Corbet
1 sibling, 1 reply; 63+ messages in thread
From: Markus Heiser @ 2019-06-14 14:10 UTC (permalink / raw)
To: Jani Nikula, Mauro Carvalho Chehab, Linux Doc Mailing List,
Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Am 14.06.19 um 15:42 schrieb Jani Nikula:
> On Thu, 13 Jun 2019, Mauro Carvalho Chehab<mchehab+samsung@kernel.org> wrote:
>> From: Mauro Carvalho Chehab<mchehab@s-opensource.com>
>>
>> As we don't want a generic Sphinx extension to execute commands,
>> change the one proposed to Markus to call the abi_book.pl
>> script.
>>
>> Use a script to parse the Documentation/ABI directory and output
>> it at the admin-guide.
> We had a legacy kernel-doc perl script so we used that instead of
> rewriting it in python. Just to keep it bug-for-bug compatible with the
> past. That was the only reason.
>
> I see absolutely zero reason to add a new perl monstrosity with a python
> extension to call it. All of this could be better done in python,
> directly.
>
> Please don't complicate the documentation build. I know you know we all
> worked hard to take apart the old DocBook Rube Goldberg machine to
> replace it with Sphinx. Please don't turn the Sphinx build to another
> complicated mess.
>
> My strong preferences are, in this order:
>
> 1) Convert the ABI documentation to reStructuredText
>
> 2) Have the python extension read the ABI files directly, without an
> extra pipeline.
>
I agree with Jani. No matter how the decision ends, since I can't help here, I'd
rather not show up in the copyright.
-- Markus --
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 05/14] scripts: add an script to parse the ABI files
2019-06-14 14:00 ` Jani Nikula
@ 2019-06-14 14:14 ` Jonathan Corbet
2019-06-14 16:24 ` Greg Kroah-Hartman
1 sibling, 0 replies; 63+ messages in thread
From: Jonathan Corbet @ 2019-06-14 14:14 UTC (permalink / raw)
To: Jani Nikula
Cc: Greg Kroah-Hartman, Mauro Carvalho Chehab,
Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel
On Fri, 14 Jun 2019 17:00:55 +0300
Jani Nikula <jani.nikula@linux.intel.com> wrote:
> On Fri, 14 Jun 2019, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> > On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> >> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> >> > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> >> >
> >> > Add a script to parse the Documentation/ABI files and produce
> >> > an output with all entries inside an ABI (sub)directory.
> >> >
> >> > Right now, it outputs its contents on ReST format. It shouldn't
> >> > be hard to make it produce other kind of outputs, since the ABI
> >> > file parser is implemented in separate than the output generator.
> >>
> >> Hum, or just convert the ABI files to rst directly.
> >
> > And what would that look like?
>
> That pretty much depends on the requirements we want to set on both the
> ABI source files and the generated output. Obviously the requirements
> can be conflicting; might be hard to produce fancy output if the input
> is very limited.
The real question, I guess, is: is there anything else that depends on the
current format of those files? That could make reformatting them harder.
Otherwise it seems like Jani's fieldlist idea might have some potential.
We could consider a *really* simple preprocessing step if we really wanted
("prepend a colon on relevant lines and join with the following line if
need be") to keep the current format nearly unchanged.
jon
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 14:10 ` Markus Heiser
@ 2019-06-14 14:15 ` Jonathan Corbet
2019-06-16 16:04 ` Markus Heiser
0 siblings, 1 reply; 63+ messages in thread
From: Jonathan Corbet @ 2019-06-14 14:15 UTC (permalink / raw)
To: Markus Heiser
Cc: Jani Nikula, Mauro Carvalho Chehab, Linux Doc Mailing List,
Greg Kroah-Hartman, Mauro Carvalho Chehab, Mauro Carvalho Chehab,
linux-kernel
On Fri, 14 Jun 2019 16:10:31 +0200
Markus Heiser <markus.heiser@darmarit.de> wrote:
> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
> rather not show up in the copyright.
Is there something specific you are asking us to do here?
Thanks,
jon
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 14:06 ` Greg Kroah-Hartman
@ 2019-06-14 15:27 ` Mauro Carvalho Chehab
2019-06-17 12:36 ` Jani Nikula
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 15:27 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Jani Nikula, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet
Em Fri, 14 Jun 2019 16:06:03 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> > On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > >
> > > As we don't want a generic Sphinx extension to execute commands,
> > > change the one proposed to Markus to call the abi_book.pl
> > > script.
> > >
> > > Use a script to parse the Documentation/ABI directory and output
> > > it at the admin-guide.
> >
> > We had a legacy kernel-doc perl script so we used that instead of
> > rewriting it in python. Just to keep it bug-for-bug compatible with the
> > past. That was the only reason.
> >
> > I see absolutely zero reason to add a new perl monstrosity with a python
> > extension to call it. All of this could be better done in python,
> > directly.
> >
> > Please don't complicate the documentation build. I know you know we all
> > worked hard to take apart the old DocBook Rube Goldberg machine to
> > replace it with Sphinx. Please don't turn the Sphinx build to another
> > complicated mess.
> >
> > My strong preferences are, in this order:
> >
> > 1) Convert the ABI documentation to reStructuredText
>
> What would that exactly look like? What would it require for new
> developers for when they write new entries? Why not rely on a helper
> script, that allows us to validate things better?
Funny enough, this e-mail arrived here after Greg's reply, and my
reply over his one :-)
-
With regards to the script, my idea is to have it run on a new
"validate" mode, when the Kernel is built with COMPILE_TEST:
https://git.linuxtv.org/mchehab/experimental.git/log/?h=abi_patches_v4
NB: the last patch is not yet... somehow, the building system is not
passing CONFIG_WARN_ABI_ERRORS to Documentation/Makefile. I'm
debugging it.
Personally, I would prefer to keep it the way it is, with two
additions:
1) I would add a SPDX header at the fist line of each file there;
2) It would make sense to have a new field - or indicator - to let
add ReST markups at the description.
The advantage of using a parseable ABI file is that it is possible
to parse it, for example, to search for a symbol:
$ ./scripts/get_abi.pl voltage_max
/sys/class/power_supply/<supply_name>/voltage_max
-------------------------------------------------
Date: January 2008
Contact: linux-pm@vger.kernel.org
Defined on file: Documentation/ABI/testing/sysfs-class-power
Description:
Reports the maximum VBUS voltage the supply can support.
Access: Read
Valid values: Represented in microvolts
...
>
> > 2) Have the python extension read the ABI files directly, without an
> > extra pipeline.
>
> He who writes the script, get's to dictate the language of the script :)
No idea about how much time it would take if written in python,
but this perl script is really fast:
$ time ./scripts/get_abi.pl search voltage_max >/dev/null
real 0m0,139s
user 0m0,132s
sys 0m0,006s
That's the time it takes here (SSD disks) to read all files under
Documentation/ABI, parse them and seek for a string.
That's about half of the time a python script takes to just import the
the sphinx modules and print its version, running at the same machine:
$ time sphinx-build --version >/dev/null
real 0m0,224s
user 0m0,199s
sys 0m0,024s
> Personally, this looks sane to me, I'm going to apply the ABI fixups to
> my tree at least, and then see how the script works out. The script can
> always be replaced with a different one in a different language at a
> later point in time of people think it really mattes.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 01/14] ABI: fix some syntax issues at the ABI database
2019-06-14 7:20 ` Andrew Donnellan
@ 2019-06-14 16:16 ` Greg Kroah-Hartman
0 siblings, 0 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-14 16:16 UTC (permalink / raw)
To: Andrew Donnellan
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Andreas Klinger, Jonathan Cameron,
Hartmut Knaack, Lars-Peter Clausen, Peter Meerwald-Stadler,
Frederic Barrat, Sebastian Reichel, Kees Cook, Anton Vorontsov,
Colin Cross, Tony Luck, linux-iio, linuxppc-dev, linux-pm
On Fri, Jun 14, 2019 at 05:20:29PM +1000, Andrew Donnellan wrote:
> On 14/6/19 12:04 pm, Mauro Carvalho Chehab wrote:
> > diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
> > index bbbabffc682a..fc7c6f7c21b3 100644
> > --- a/Documentation/ABI/testing/sysfs-class-cxl
> > +++ b/Documentation/ABI/testing/sysfs-class-cxl
> > @@ -1,6 +1,6 @@
> > -Note: Attributes that are shared between devices are stored in the directory
> > -pointed to by the symlink device/.
> > -Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> > +Please notice that attributes that are shared between devices are stored in
>
> Would prefer "Please note" over "Please notice".
Now changed, but that's a minor grammer thing, both are correct. :)
> Acked-by: Andrew Donnellan <ajd@linux.ibm.com> # cxl
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-14 2:04 ` [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support Mauro Carvalho Chehab
@ 2019-06-14 16:18 ` Greg Kroah-Hartman
2019-06-14 16:25 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-14 16:18 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> The parser breaks with UTF-8 characters with Sphinx 1.4.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> Documentation/sphinx/kernel_abi.py | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> index 7fa7806532dc..460cee48a245 100644
> --- a/Documentation/sphinx/kernel_abi.py
> +++ b/Documentation/sphinx/kernel_abi.py
> @@ -1,4 +1,5 @@
> -# -*- coding: utf-8; mode: python -*-
> +# coding=utf-8
> +#
Is this an emacs vs. vim fight?
Why change this?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 00/14] Add support to generate ABI documentation at admin-guide
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
` (13 preceding siblings ...)
2019-06-14 2:04 ` [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support Mauro Carvalho Chehab
@ 2019-06-14 16:20 ` Greg Kroah-Hartman
14 siblings, 0 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-14 16:20 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, linuxppc-dev, Andrew Donnellan,
Jonathan Cameron, Lars-Peter Clausen, Anton Vorontsov, linux-pm,
Colin Cross, linux-iio, Hartmut Knaack, Sebastian Reichel,
Frederic Barrat, Andreas Klinger, Tony Luck,
Peter Meerwald-Stadler, Stefan Achatz, Kees Cook
On Thu, Jun 13, 2019 at 11:04:06PM -0300, Mauro Carvalho Chehab wrote:
> Greg,
>
> As promised, I'm resending the patch series with adds the Kernel ABI to
> Documentation/admin-guide.
>
> Those patches are basically the version 3 patchset I sent back in 2017,
> rebased on the top of linux-next (next-20190613), and with some fixes
> in order for it to work.
>
> - The 4 initial patches to fix some ABI descriptions that are violating
> the syntax described at Documentation/ABI/README;
These 4 are now applied to my driver-core tree, thanks.
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 05/14] scripts: add an script to parse the ABI files
2019-06-14 14:00 ` Jani Nikula
2019-06-14 14:14 ` Jonathan Corbet
@ 2019-06-14 16:24 ` Greg Kroah-Hartman
1 sibling, 0 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-14 16:24 UTC (permalink / raw)
To: Jani Nikula
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Fri, Jun 14, 2019 at 05:00:55PM +0300, Jani Nikula wrote:
> On Fri, 14 Jun 2019, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> > On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> >> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> >> > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> >> >
> >> > Add a script to parse the Documentation/ABI files and produce
> >> > an output with all entries inside an ABI (sub)directory.
> >> >
> >> > Right now, it outputs its contents on ReST format. It shouldn't
> >> > be hard to make it produce other kind of outputs, since the ABI
> >> > file parser is implemented in separate than the output generator.
> >>
> >> Hum, or just convert the ABI files to rst directly.
> >
> > And what would that look like?
>
> That pretty much depends on the requirements we want to set on both the
> ABI source files and the generated output. Obviously the requirements
> can be conflicting; might be hard to produce fancy output if the input
> is very limited.
>
> At the bare minimum, you could convert the files to contain
> reStructuredText field lists [1]. Add a colon at the start of the field
> name, and make sure field bodies (values) are not empty.
>
> Conversion of a file selected at random; I've only added ":" and "N/A".
N/A should be allowed to just drop the line entirely, right?
And what does this end up looking like?
I also hate "flag days" where all of a chunk of stuff needs to be
converted into another style. Also it doesn't deal with merges from the
100+ different trees that all end up adding stuff to this directory over
time (slowly though, unfortunately)
So ideally, I'd like to keep the original format if at all possible.
Having the tool here allows people to do nice things like search for a
specific file easily, or a device type, which is something that I know I
have wanted, and others have asked for in the past as well.
It also might allow us to find out where we are missing documentation, a
long-term goal of mine.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-14 16:18 ` Greg Kroah-Hartman
@ 2019-06-14 16:25 ` Mauro Carvalho Chehab
2019-06-15 6:16 ` Greg Kroah-Hartman
2019-06-16 15:43 ` Markus Heiser
0 siblings, 2 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-14 16:25 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Em Fri, 14 Jun 2019 18:18:37 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> > The parser breaks with UTF-8 characters with Sphinx 1.4.
> >
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > ---
> > Documentation/sphinx/kernel_abi.py | 10 ++++++----
> > 1 file changed, 6 insertions(+), 4 deletions(-)
> >
> > diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> > index 7fa7806532dc..460cee48a245 100644
> > --- a/Documentation/sphinx/kernel_abi.py
> > +++ b/Documentation/sphinx/kernel_abi.py
> > @@ -1,4 +1,5 @@
> > -# -*- coding: utf-8; mode: python -*-
> > +# coding=utf-8
> > +#
>
> Is this an emacs vs. vim fight?
No. This is a python-specific thing:
https://www.python.org/dev/peps/pep-0263/
>
> Why change this?
Just to keep the "header" part of the script closer to kerneldoc.py.
You may keep the previous syntax if you want, as both ways are
equally recognized, as python actually checks for anything that
matches this regex at the first or second line:
^[ \t\f]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-14 16:25 ` Mauro Carvalho Chehab
@ 2019-06-15 6:16 ` Greg Kroah-Hartman
2019-06-16 15:43 ` Markus Heiser
1 sibling, 0 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-15 6:16 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Fri, Jun 14, 2019 at 01:25:30PM -0300, Mauro Carvalho Chehab wrote:
> Em Fri, 14 Jun 2019 18:18:37 +0200
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
>
> > On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> > > The parser breaks with UTF-8 characters with Sphinx 1.4.
> > >
> > > Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> > > ---
> > > Documentation/sphinx/kernel_abi.py | 10 ++++++----
> > > 1 file changed, 6 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> > > index 7fa7806532dc..460cee48a245 100644
> > > --- a/Documentation/sphinx/kernel_abi.py
> > > +++ b/Documentation/sphinx/kernel_abi.py
> > > @@ -1,4 +1,5 @@
> > > -# -*- coding: utf-8; mode: python -*-
> > > +# coding=utf-8
> > > +#
> >
> > Is this an emacs vs. vim fight?
>
> No. This is a python-specific thing:
>
> https://www.python.org/dev/peps/pep-0263/
Ah, thanks, didn't know that.
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-14 16:25 ` Mauro Carvalho Chehab
2019-06-15 6:16 ` Greg Kroah-Hartman
@ 2019-06-16 15:43 ` Markus Heiser
2019-06-17 9:16 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 63+ messages in thread
From: Markus Heiser @ 2019-06-16 15:43 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Am 14.06.19 um 18:25 schrieb Mauro Carvalho Chehab:
> Em Fri, 14 Jun 2019 18:18:37 +0200
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
>
>> On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
>>> The parser breaks with UTF-8 characters with Sphinx 1.4.
>>>
>>> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
>>> ---
>>> Documentation/sphinx/kernel_abi.py | 10 ++++++----
>>> 1 file changed, 6 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
>>> index 7fa7806532dc..460cee48a245 100644
>>> --- a/Documentation/sphinx/kernel_abi.py
>>> +++ b/Documentation/sphinx/kernel_abi.py
>>> @@ -1,4 +1,5 @@
>>> -# -*- coding: utf-8; mode: python -*-
>>> +# coding=utf-8
>>> +#
>>
>> Is this an emacs vs. vim fight?
>
> No. This is a python-specific thing:
>
> https://www.python.org/dev/peps/pep-0263/
No need to change, the emacs notation is also OK, see your link
"""or (using formats recognized by popular editors):"""
https://www.python.org/dev/peps/pep-0263/#defining-the-encoding
I prefer emacs notation, this is also evaluated by many other editors / tools.
-- Markus --
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 14:15 ` Jonathan Corbet
@ 2019-06-16 16:04 ` Markus Heiser
2019-06-17 9:11 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Markus Heiser @ 2019-06-16 16:04 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Jani Nikula, Mauro Carvalho Chehab, Linux Doc Mailing List,
Greg Kroah-Hartman, Mauro Carvalho Chehab, Mauro Carvalho Chehab,
linux-kernel
Am 14.06.19 um 16:15 schrieb Jonathan Corbet:
> On Fri, 14 Jun 2019 16:10:31 +0200
> Markus Heiser <markus.heiser@darmarit.de> wrote:
>
>> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
>> rather not show up in the copyright.
>
> Is there something specific you are asking us to do here?
>
I have lost the overview, but there was a patch Mauro added a
kernel_abi.py. There was my name (Markus Heiser) listed with a
copyright notation.
I guess Mauro picked up some old RFC or an other old patch of
mine from 2016 and made some C&P .. whatever .. ATM I do not have
time to give any support on parsing ABI and I'am not interested
in holding copyrights on a C&P of a old source ;)
-- Markus --
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-16 16:04 ` Markus Heiser
@ 2019-06-17 9:11 ` Mauro Carvalho Chehab
2019-06-17 16:31 ` Markus Heiser
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-17 9:11 UTC (permalink / raw)
To: Markus Heiser
Cc: Jonathan Corbet, Jani Nikula, Linux Doc Mailing List,
Greg Kroah-Hartman, Mauro Carvalho Chehab, Mauro Carvalho Chehab,
linux-kernel
Em Sun, 16 Jun 2019 18:04:01 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> Am 14.06.19 um 16:15 schrieb Jonathan Corbet:
> > On Fri, 14 Jun 2019 16:10:31 +0200
> > Markus Heiser <markus.heiser@darmarit.de> wrote:
> >
> >> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
> >> rather not show up in the copyright.
> >
> > Is there something specific you are asking us to do here?
> >
>
>
> I have lost the overview, but there was a patch Mauro added a
> kernel_abi.py. There was my name (Markus Heiser) listed with a
> copyright notation.
>
> I guess Mauro picked up some old RFC or an other old patch of
> mine from 2016 and made some C&P .. whatever .. ATM I do not have
> time to give any support on parsing ABI and I'am not interested
> in holding copyrights on a C&P of a old source ;)
Well, the code was basically written by you :-)
It was written to be a script capable of running a generic
script. On that time, my contribution to it was basically
to hardcode it to run "get_abi.pl".
This came from an old branch where the last change was back in 2017.
It was resurrected due to a discussion at KS ML.
There, the discussion was related to what's left to be converted
to ReST.
While I can't simply remove your copyright, would you be happy
with something like that?
Thanks,
Mauro
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 2d5d582207f7..ef91b1e1ff4b 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -7,7 +7,8 @@ u"""
Implementation of the ``kernel-abi`` reST-directive.
:copyright: Copyright (C) 2016 Markus Heiser
- :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
+ :copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
+ :maintained-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
:license: GPL Version 2, June 1991 see Linux/COPYING for details.
The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-16 15:43 ` Markus Heiser
@ 2019-06-17 9:16 ` Mauro Carvalho Chehab
2019-06-17 13:56 ` Jonathan Corbet
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-17 9:16 UTC (permalink / raw)
To: Markus Heiser
Cc: Greg Kroah-Hartman, Linux Doc Mailing List,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet
Em Sun, 16 Jun 2019 17:43:50 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> Am 14.06.19 um 18:25 schrieb Mauro Carvalho Chehab:
> > Em Fri, 14 Jun 2019 18:18:37 +0200
> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> >
> >> On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> >>> The parser breaks with UTF-8 characters with Sphinx 1.4.
> >>>
> >>> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> >>> ---
> >>> Documentation/sphinx/kernel_abi.py | 10 ++++++----
> >>> 1 file changed, 6 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> >>> index 7fa7806532dc..460cee48a245 100644
> >>> --- a/Documentation/sphinx/kernel_abi.py
> >>> +++ b/Documentation/sphinx/kernel_abi.py
> >>> @@ -1,4 +1,5 @@
> >>> -# -*- coding: utf-8; mode: python -*-
> >>> +# coding=utf-8
> >>> +#
> >>
> >> Is this an emacs vs. vim fight?
> >
> > No. This is a python-specific thing:
> >
> > https://www.python.org/dev/peps/pep-0263/
>
> No need to change, the emacs notation is also OK, see your link
>
> """or (using formats recognized by popular editors):"""
>
> https://www.python.org/dev/peps/pep-0263/#defining-the-encoding
>
> I prefer emacs notation, this is also evaluated by many other editors / tools.
The usage of emacs notation is something that we don't like at the
Linux Kernel. With ~4K developers per release, if we add tags to
every single editor people use, it would be really messy, as one
developer would be adding a tag and the next one replacing it by its
some other favorite editor's tag.
There's even an item at Documentation/process/coding-style.rst
due to that (item 9).
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-14 15:27 ` Mauro Carvalho Chehab
@ 2019-06-17 12:36 ` Jani Nikula
2019-06-17 12:54 ` Greg Kroah-Hartman
2019-06-17 13:51 ` Mauro Carvalho Chehab
0 siblings, 2 replies; 63+ messages in thread
From: Jani Nikula @ 2019-06-17 12:36 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet
On Fri, 14 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> Em Fri, 14 Jun 2019 16:06:03 +0200
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
>
>> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
>> > 2) Have the python extension read the ABI files directly, without an
>> > extra pipeline.
>>
>> He who writes the script, get's to dictate the language of the script :)
The point is, it's an extension to a python based tool, written in perl,
using pipes for communication, and losing any advantages of integrating
with the tool it's extending.
I doubt you'd want to see system() to be used to subsequently extend the
perl tool.
I think it's just sad to see the documentation system slowly drift
further away from the ideals we had, and towards the old ways we worked
so hard to fix.
> No idea about how much time it would take if written in python,
> but this perl script is really fast:
>
> $ time ./scripts/get_abi.pl search voltage_max >/dev/null
> real 0m0,139s
> user 0m0,132s
> sys 0m0,006s
>
> That's the time it takes here (SSD disks) to read all files under
> Documentation/ABI, parse them and seek for a string.
>
> That's about half of the time a python script takes to just import the
> the sphinx modules and print its version, running at the same machine:
>
> $ time sphinx-build --version >/dev/null
>
> real 0m0,224s
> user 0m0,199s
> sys 0m0,024s
Please at least use fair and sensible comparisons. If you want to make
the extension usable standalone on the command-line, bypassing Sphinx,
you can do that. No need to factor in Sphinx to your comparisons.
BR,
Jani.
--
Jani Nikula, Intel Open Source Graphics Center
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-17 12:36 ` Jani Nikula
@ 2019-06-17 12:54 ` Greg Kroah-Hartman
2019-06-17 13:48 ` Jonathan Corbet
2019-06-17 13:50 ` Jani Nikula
2019-06-17 13:51 ` Mauro Carvalho Chehab
1 sibling, 2 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-17 12:54 UTC (permalink / raw)
To: Jani Nikula
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Mon, Jun 17, 2019 at 03:36:17PM +0300, Jani Nikula wrote:
> On Fri, 14 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > Em Fri, 14 Jun 2019 16:06:03 +0200
> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> >
> >> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> >> > 2) Have the python extension read the ABI files directly, without an
> >> > extra pipeline.
> >>
> >> He who writes the script, get's to dictate the language of the script :)
>
> The point is, it's an extension to a python based tool, written in perl,
> using pipes for communication, and losing any advantages of integrating
> with the tool it's extending.
>
> I doubt you'd want to see system() to be used to subsequently extend the
> perl tool.
>
> I think it's just sad to see the documentation system slowly drift
> further away from the ideals we had, and towards the old ways we worked
> so hard to fix.
What are those ideals?
I thought the goal was to be able to write documentation in a as much
as a normal text file as possible and have automation turn those files
into "pretty" documentation that we can all use.
And I think that fits with the way this patch set goes, right? We are
not on a quest for purity of scripts to generate the documentation at
the expense of having to force hundreds, or thousands, of developers to
change their ways, or to force a "flag day" conversion of existing
documentation resulting in a huge merge mess.
So, we are stuck with the current structure that I totally made up for
Documentation/ABI/. Turns out it is almost parsable, as Mauro's tool
shows. His tool also validates the existing text, which is great, and
has caused fixes for it.
If someone wants to write that tool in some other language, like python,
wonderful, I have no objection, but as it is, this is a useful tool
already, allowing us to validate, and search, existing documentation
entries that we have never been able to do before. It also provides an
output that can be turned into pretty html/pdf/whatever files by other
tools in the pipeline, a totally bonus benefit.
So what is going backwards here?
Maybe the processing pipeline isn't as nice as you would like, but
remember to view this from a normal developer's point of view, not a
documentation pipeline developer's point of view please.
So, in short, my requirements are:
- keep Documentation/ABI/ file formats as close as possible to
what we have today, preventing any flag-day issues or merge
problems
- be able to query and validate Documentation/ABI/
- be able to turn Documentation/ABI into pretty documentation.
If you object to the mechanics of the last requirement here, I don't
object either, provide something else that works better. But don't
throw away the whole thing just because you don't like how things are
hooked up here.
I'm going to go apply most of the rest of these patches to my
driver-core tree, stopping at the "hook it up to the kernel
documentation" point. Is that ok?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-17 12:54 ` Greg Kroah-Hartman
@ 2019-06-17 13:48 ` Jonathan Corbet
2019-06-17 13:50 ` Jani Nikula
1 sibling, 0 replies; 63+ messages in thread
From: Jonathan Corbet @ 2019-06-17 13:48 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Jani Nikula, Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel
On Mon, 17 Jun 2019 14:54:38 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> > I think it's just sad to see the documentation system slowly drift
> > further away from the ideals we had, and towards the old ways we worked
> > so hard to fix.
>
> What are those ideals?
>
> I thought the goal was to be able to write documentation in a as much
> as a normal text file as possible and have automation turn those files
> into "pretty" documentation that we can all use.
That was indeed one of the goals. Another was to replace the incredible
pile of fragile duct tape that the docs build system had become with
something more robust, understandable, and maintainable. We did that, to
an extent at least, and life is better.
Jani worries that we have been regressing toward duct-tape mode, and I
suspect he may be right. I'm certainly as guilty as anybody of tossing
stuff in because it's expedient right now. It is right to ask whether we
should continue in that direction.
Can we slow down just a bit on the ABI files? It may be that Mauro's
solution is the best one, but I would really like to think a bit about
how all this stuff fits together, and life isn't really even giving me
time to tie my shoes these days. I don't think that this is screamingly
urgent right now.
Thanks,
jon
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-17 12:54 ` Greg Kroah-Hartman
2019-06-17 13:48 ` Jonathan Corbet
@ 2019-06-17 13:50 ` Jani Nikula
1 sibling, 0 replies; 63+ messages in thread
From: Jani Nikula @ 2019-06-17 13:50 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Mon, 17 Jun 2019, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> On Mon, Jun 17, 2019 at 03:36:17PM +0300, Jani Nikula wrote:
>> On Fri, 14 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
>> > Em Fri, 14 Jun 2019 16:06:03 +0200
>> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
>> >
>> >> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
>> >> > 2) Have the python extension read the ABI files directly, without an
>> >> > extra pipeline.
>> >>
>> >> He who writes the script, get's to dictate the language of the script :)
>>
>> The point is, it's an extension to a python based tool, written in perl,
>> using pipes for communication, and losing any advantages of integrating
>> with the tool it's extending.
>>
>> I doubt you'd want to see system() to be used to subsequently extend the
>> perl tool.
>>
>> I think it's just sad to see the documentation system slowly drift
>> further away from the ideals we had, and towards the old ways we worked
>> so hard to fix.
>
> What are those ideals?
For example, have a single coherent system, instead of a fragile pipe
with each stage written in a different language, each having its own
idiosynchracies, each step losing something in translation.
Have a system that a normal developer can actually look at and
understand. It didn't use to be that way.
> I thought the goal was to be able to write documentation in a as much
> as a normal text file as possible and have automation turn those files
> into "pretty" documentation that we can all use.
>
> And I think that fits with the way this patch set goes, right? We are
> not on a quest for purity of scripts to generate the documentation at
> the expense of having to force hundreds, or thousands, of developers to
> change their ways, or to force a "flag day" conversion of existing
> documentation resulting in a huge merge mess.
Fair enough, let's dismiss the thought of changing the ABI files. But I
never meant that would somehow be for the "purity of scripts", or that
those two would somehow be at odds here.
> So, we are stuck with the current structure that I totally made up for
> Documentation/ABI/. Turns out it is almost parsable, as Mauro's tool
> shows. His tool also validates the existing text, which is great, and
> has caused fixes for it.
>
> If someone wants to write that tool in some other language, like python,
> wonderful, I have no objection, but as it is, this is a useful tool
> already, allowing us to validate, and search, existing documentation
> entries that we have never been able to do before. It also provides an
> output that can be turned into pretty html/pdf/whatever files by other
> tools in the pipeline, a totally bonus benefit.
>
> So what is going backwards here?
>
> Maybe the processing pipeline isn't as nice as you would like, but
> remember to view this from a normal developer's point of view, not a
> documentation pipeline developer's point of view please.
>
> So, in short, my requirements are:
> - keep Documentation/ABI/ file formats as close as possible to
> what we have today, preventing any flag-day issues or merge
> problems
> - be able to query and validate Documentation/ABI/
> - be able to turn Documentation/ABI into pretty documentation.
>
> If you object to the mechanics of the last requirement here, I don't
> object either, provide something else that works better. But don't
> throw away the whole thing just because you don't like how things are
> hooked up here.
>
> I'm going to go apply most of the rest of these patches to my
> driver-core tree, stopping at the "hook it up to the kernel
> documentation" point. Is that ok?
I'll leave it all up to Jon's discretion; I trust he'll understand my
concerns. I have no authority beyond the opinion I've voiced here
anyway.
BR,
Jani.
--
Jani Nikula, Intel Open Source Graphics Center
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-17 12:36 ` Jani Nikula
2019-06-17 12:54 ` Greg Kroah-Hartman
@ 2019-06-17 13:51 ` Mauro Carvalho Chehab
2019-06-18 8:47 ` Jani Nikula
1 sibling, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-17 13:51 UTC (permalink / raw)
To: Jani Nikula
Cc: Greg Kroah-Hartman, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Em Mon, 17 Jun 2019 15:36:17 +0300
Jani Nikula <jani.nikula@linux.intel.com> escreveu:
> On Fri, 14 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > Em Fri, 14 Jun 2019 16:06:03 +0200
> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> >
> >> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> >> > 2) Have the python extension read the ABI files directly, without an
> >> > extra pipeline.
> >>
> >> He who writes the script, get's to dictate the language of the script :)
>
> The point is, it's an extension to a python based tool, written in perl,
> using pipes for communication, and losing any advantages of integrating
> with the tool it's extending.
>
> I doubt you'd want to see system() to be used to subsequently extend the
> perl tool.
>
> I think it's just sad to see the documentation system slowly drift
> further away from the ideals we had, and towards the old ways we worked
> so hard to fix.
Actually, it is a perl script that can be used standalone (just like
get_maintainers.pl and kernel-doc) with have some features including
producing a ReST output. We could easily get rid of the python extension,
if we add this to the Makefile (adjusted to work with O= option):
./scripts/get_api.pl rest > Documentation/output/admin-guide/abi.rst
>
> > No idea about how much time it would take if written in python,
> > but this perl script is really fast:
> >
> > $ time ./scripts/get_abi.pl search voltage_max >/dev/null
> > real 0m0,139s
> > user 0m0,132s
> > sys 0m0,006s
> >
> > That's the time it takes here (SSD disks) to read all files under
> > Documentation/ABI, parse them and seek for a string.
> >
> > That's about half of the time a python script takes to just import the
> > the sphinx modules and print its version, running at the same machine:
> >
> > $ time sphinx-build --version >/dev/null
> >
> > real 0m0,224s
> > user 0m0,199s
> > sys 0m0,024s
>
> Please at least use fair and sensible comparisons. If you want to make
> the extension usable standalone on the command-line, bypassing Sphinx,
> you can do that. No need to factor in Sphinx to your comparisons.
Yeah, I guess it should be possible to do that. How a python script
can identify if it was called by Sphinx, or if it was called directly?
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-17 9:16 ` Mauro Carvalho Chehab
@ 2019-06-17 13:56 ` Jonathan Corbet
2019-06-17 15:55 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Jonathan Corbet @ 2019-06-17 13:56 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Markus Heiser, Greg Kroah-Hartman, Linux Doc Mailing List,
Mauro Carvalho Chehab, linux-kernel
On Mon, 17 Jun 2019 06:16:59 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > No need to change, the emacs notation is also OK, see your link
> >
> > """or (using formats recognized by popular editors):"""
> >
> > https://www.python.org/dev/peps/pep-0263/#defining-the-encoding
> >
> > I prefer emacs notation, this is also evaluated by many other editors / tools.
>
> The usage of emacs notation is something that we don't like at the
> Linux Kernel. With ~4K developers per release, if we add tags to
> every single editor people use, it would be really messy, as one
> developer would be adding a tag and the next one replacing it by its
> some other favorite editor's tag.
So "we" like a language-specific notation instead? That seems a little
strange to me. Lots of things understand the Emacs notation, it doesn't
seem like something that needs to be actively avoided here.
Thanks,
jon
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support
2019-06-17 13:56 ` Jonathan Corbet
@ 2019-06-17 15:55 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-17 15:55 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Markus Heiser, Greg Kroah-Hartman, Linux Doc Mailing List,
Mauro Carvalho Chehab, linux-kernel
Em Mon, 17 Jun 2019 07:56:08 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> On Mon, 17 Jun 2019 06:16:59 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
>
> > > No need to change, the emacs notation is also OK, see your link
> > >
> > > """or (using formats recognized by popular editors):"""
> > >
> > > https://www.python.org/dev/peps/pep-0263/#defining-the-encoding
> > >
> > > I prefer emacs notation, this is also evaluated by many other editors / tools.
> >
> > The usage of emacs notation is something that we don't like at the
> > Linux Kernel. With ~4K developers per release, if we add tags to
> > every single editor people use, it would be really messy, as one
> > developer would be adding a tag and the next one replacing it by its
> > some other favorite editor's tag.
>
> So "we" like a language-specific notation instead? That seems a little
> strange to me. Lots of things understand the Emacs notation, it doesn't
> seem like something that needs to be actively avoided here.
From my side, I don't have any strong preference. Just saying that
people usually complain when e-macs or vim specific tags appear at the
Kernel. That's why I would prefer an editor-agnostic macro.
It won't make any difference for me, anyway, as the editors I use
don't recognize it.
Whatever you want is OK to me, provided that we use the same notation on
all Sphinx extensions... right now there's a mix of notations.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-17 9:11 ` Mauro Carvalho Chehab
@ 2019-06-17 16:31 ` Markus Heiser
0 siblings, 0 replies; 63+ messages in thread
From: Markus Heiser @ 2019-06-17 16:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Jani Nikula, Linux Doc Mailing List,
Greg Kroah-Hartman, Mauro Carvalho Chehab, Mauro Carvalho Chehab,
linux-kernel
Am 17.06.19 um 11:11 schrieb Mauro Carvalho Chehab:
> Em Sun, 16 Jun 2019 18:04:01 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
>> Am 14.06.19 um 16:15 schrieb Jonathan Corbet:
>>> On Fri, 14 Jun 2019 16:10:31 +0200
>>> Markus Heiser <markus.heiser@darmarit.de> wrote:
>>>
>>>> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
>>>> rather not show up in the copyright.
>>>
>>> Is there something specific you are asking us to do here?
>>>
>>
>>
>> I have lost the overview, but there was a patch Mauro added a
>> kernel_abi.py. There was my name (Markus Heiser) listed with a
>> copyright notation.
>>
>> I guess Mauro picked up some old RFC or an other old patch of
>> mine from 2016 and made some C&P .. whatever .. ATM I do not have
>> time to give any support on parsing ABI and I'am not interested
>> in holding copyrights on a C&P of a old source ;)
>
> Well, the code was basically written by you :-)
>
> It was written to be a script capable of running a generic
> script. On that time, my contribution to it was basically
> to hardcode it to run "get_abi.pl".
Thanks for clarifying.
>
> This came from an old branch where the last change was back in 2017.
> It was resurrected due to a discussion at KS ML.
>
> There, the discussion was related to what's left to be converted
> to ReST.
>
> While I can't simply remove your copyright, would you be happy
> with something like that?
Yes, but basically I share Jani's and Jon's doubts about this solution.
-- Markus --
>
>
> Thanks,
> Mauro
>
> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> index 2d5d582207f7..ef91b1e1ff4b 100644
> --- a/Documentation/sphinx/kernel_abi.py
> +++ b/Documentation/sphinx/kernel_abi.py
> @@ -7,7 +7,8 @@ u"""
> Implementation of the ``kernel-abi`` reST-directive.
>
> :copyright: Copyright (C) 2016 Markus Heiser
> - :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
> + :copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
> + :maintained-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> :license: GPL Version 2, June 1991 see Linux/COPYING for details.
>
> The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
>
>
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-17 13:51 ` Mauro Carvalho Chehab
@ 2019-06-18 8:47 ` Jani Nikula
2019-06-19 16:37 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Jani Nikula @ 2019-06-18 8:47 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Greg Kroah-Hartman, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
On Mon, 17 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> Yeah, I guess it should be possible to do that. How a python script
> can identify if it was called by Sphinx, or if it was called directly?
if __name__ == '__main__':
# run on the command-line, not imported
BR,
Jani.
--
Jani Nikula, Intel Open Source Graphics Center
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-14 2:04 ` [PATCH 04/14] ABI: better identificate tables Mauro Carvalho Chehab
@ 2019-06-19 12:51 ` Johan Hovold
2019-06-19 13:56 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Johan Hovold @ 2019-06-19 12:51 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Doc Mailing List, Greg Kroah-Hartman,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Stefan Achatz
On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
>
> When parsing via script, it is important to know if the script
> should consider a description as a literal block that should
> be displayed as-is, or if the description can be considered
> as a normal text.
>
> Change descriptions to ensure that the preceding line of a table
> ends with a colon. That makes easy to identify the need of a
> literal block.
In the cover letter you say that the first four patches of this series,
including this one, "fix some ABI descriptions that are violating the
syntax described at Documentation/ABI/README". This seems a bit harsh,
given that it's you that is now *introducing* a new syntax requirement
to assist your script.
Specifically, this new requirement isn't documented anywhere AFAICT, so
how will anyone adding new ABI descriptions learn about it?
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
> Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra | 2 +-
> .../ABI/testing/sysfs-class-backlight-driver-lm3533 | 6 +++---
> Documentation/ABI/testing/sysfs-class-led-driver-lm3533 | 8 ++++----
> Documentation/ABI/testing/sysfs-class-leds-gt683r | 4 ++--
> Documentation/ABI/testing/sysfs-driver-hid-roccat-kone | 2 +-
> 5 files changed, 11 insertions(+), 11 deletions(-)
>
> diff --git a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
> index 16020b31ae64..5d41ebadf15e 100644
> --- a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
> +++ b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
> @@ -5,7 +5,7 @@ Description: It is possible to switch the cpi setting of the mouse with the
> press of a button.
> When read, this file returns the raw number of the actual cpi
> setting reported by the mouse. This number has to be further
> - processed to receive the real dpi value.
> + processed to receive the real dpi value:
>
> VALUE DPI
> 1 400
Johan
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-19 12:51 ` Johan Hovold
@ 2019-06-19 13:56 ` Mauro Carvalho Chehab
2019-06-19 15:02 ` Greg Kroah-Hartman
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-19 13:56 UTC (permalink / raw)
To: Johan Hovold, Greg Kroah-Hartman
Cc: Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
Hi Johan,
Em Wed, 19 Jun 2019 14:51:35 +0200
Johan Hovold <johan@kernel.org> escreveu:
> On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> >
> > When parsing via script, it is important to know if the script
> > should consider a description as a literal block that should
> > be displayed as-is, or if the description can be considered
> > as a normal text.
> >
> > Change descriptions to ensure that the preceding line of a table
> > ends with a colon. That makes easy to identify the need of a
> > literal block.
>
> In the cover letter you say that the first four patches of this series,
> including this one, "fix some ABI descriptions that are violating the
> syntax described at Documentation/ABI/README". This seems a bit harsh,
> given that it's you that is now *introducing* a new syntax requirement
> to assist your script.
Yeah, what's there at the cover letter doesn't apply to this specific
patch. The thing is that I wrote this series a lot of time ago (2016/17).
I revived those per a request at KS ML, as we still need to expose the
ABI content on some book that will be used by userspace people.
So, I just rebased it on the top of curent Kernel, add a cover letter
with the things I remembered and re-sent.
In the specific case of this patch, the ":" there actually makes sense
for someone that it is reading it as a text file, and it is an easy
hack to make it parse better.
> Specifically, this new requirement isn't documented anywhere AFAICT, so
> how will anyone adding new ABI descriptions learn about it?
Yeah, either that or provide an alternative to "Description" tag, to be
used with more complex ABI descriptions.
One of the things that occurred to me, back on 2017, is that we should
have a way to to specify that an specific ABI description would have
a rich format. Something like:
What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
Date: August 2010
Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
RST-Description:
It is possible to switch the cpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual cpi
setting reported by the mouse. This number has to be further
processed to receive the real dpi value:
===== =====
VALUE DPI
===== =====
1 400
2 800
4 1600
===== =====
With that, the script will know that the description contents will be using
the ReST markup, and parse it accordingly. Right now, what it does, instead,
is to place the description on a code-block, e. g. it will produce this
output for the description:
::
It is possible to switch the cpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual cpi
setting reported by the mouse. This number has to be further
processed to receive the real dpi value:
VALUE DPI
1 400
2 800
4 1600
Greg,
what do you think?
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-19 13:56 ` Mauro Carvalho Chehab
@ 2019-06-19 15:02 ` Greg Kroah-Hartman
2019-06-19 16:14 ` Mauro Carvalho Chehab
2019-06-20 12:01 ` Johan Hovold
0 siblings, 2 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-19 15:02 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> Hi Johan,
>
> Em Wed, 19 Jun 2019 14:51:35 +0200
> Johan Hovold <johan@kernel.org> escreveu:
>
> > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > >
> > > When parsing via script, it is important to know if the script
> > > should consider a description as a literal block that should
> > > be displayed as-is, or if the description can be considered
> > > as a normal text.
> > >
> > > Change descriptions to ensure that the preceding line of a table
> > > ends with a colon. That makes easy to identify the need of a
> > > literal block.
> >
> > In the cover letter you say that the first four patches of this series,
> > including this one, "fix some ABI descriptions that are violating the
> > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > given that it's you that is now *introducing* a new syntax requirement
> > to assist your script.
>
> Yeah, what's there at the cover letter doesn't apply to this specific
> patch. The thing is that I wrote this series a lot of time ago (2016/17).
>
> I revived those per a request at KS ML, as we still need to expose the
> ABI content on some book that will be used by userspace people.
>
> So, I just rebased it on the top of curent Kernel, add a cover letter
> with the things I remembered and re-sent.
>
> In the specific case of this patch, the ":" there actually makes sense
> for someone that it is reading it as a text file, and it is an easy
> hack to make it parse better.
>
> > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > how will anyone adding new ABI descriptions learn about it?
>
> Yeah, either that or provide an alternative to "Description" tag, to be
> used with more complex ABI descriptions.
>
> One of the things that occurred to me, back on 2017, is that we should
> have a way to to specify that an specific ABI description would have
> a rich format. Something like:
>
> What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> Date: August 2010
> Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
> RST-Description:
> It is possible to switch the cpi setting of the mouse with the
> press of a button.
> When read, this file returns the raw number of the actual cpi
> setting reported by the mouse. This number has to be further
> processed to receive the real dpi value:
>
> ===== =====
> VALUE DPI
> ===== =====
> 1 400
> 2 800
> 4 1600
> ===== =====
>
> With that, the script will know that the description contents will be using
> the ReST markup, and parse it accordingly. Right now, what it does, instead,
> is to place the description on a code-block, e. g. it will produce this
> output for the description:
>
> ::
>
> It is possible to switch the cpi setting of the mouse with the
> press of a button.
> When read, this file returns the raw number of the actual cpi
> setting reported by the mouse. This number has to be further
> processed to receive the real dpi value:
>
> VALUE DPI
> 1 400
> 2 800
> 4 1600
>
>
> Greg,
>
> what do you think?
I don't know when "Description" and "RST-Description" would be used.
Why not just parse "Description" like rst text and if things are "messy"
we fix them up as found, like you did with the ":" here? It doesn't
have to be complex, we can always fix them up after-the-fact if new
stuff gets added that doesn't quite parse properly.
Just like we do for most kernel-doc formatting :)
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-19 15:02 ` Greg Kroah-Hartman
@ 2019-06-19 16:14 ` Mauro Carvalho Chehab
2019-06-20 14:27 ` Mauro Carvalho Chehab
2019-06-20 12:01 ` Johan Hovold
1 sibling, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-19 16:14 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
Em Wed, 19 Jun 2019 17:02:07 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > Hi Johan,
> >
> > Em Wed, 19 Jun 2019 14:51:35 +0200
> > Johan Hovold <johan@kernel.org> escreveu:
> >
> > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > > >
> > > > When parsing via script, it is important to know if the script
> > > > should consider a description as a literal block that should
> > > > be displayed as-is, or if the description can be considered
> > > > as a normal text.
> > > >
> > > > Change descriptions to ensure that the preceding line of a table
> > > > ends with a colon. That makes easy to identify the need of a
> > > > literal block.
> > >
> > > In the cover letter you say that the first four patches of this series,
> > > including this one, "fix some ABI descriptions that are violating the
> > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > given that it's you that is now *introducing* a new syntax requirement
> > > to assist your script.
> >
> > Yeah, what's there at the cover letter doesn't apply to this specific
> > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> >
> > I revived those per a request at KS ML, as we still need to expose the
> > ABI content on some book that will be used by userspace people.
> >
> > So, I just rebased it on the top of curent Kernel, add a cover letter
> > with the things I remembered and re-sent.
> >
> > In the specific case of this patch, the ":" there actually makes sense
> > for someone that it is reading it as a text file, and it is an easy
> > hack to make it parse better.
> >
> > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > how will anyone adding new ABI descriptions learn about it?
> >
> > Yeah, either that or provide an alternative to "Description" tag, to be
> > used with more complex ABI descriptions.
> >
> > One of the things that occurred to me, back on 2017, is that we should
> > have a way to to specify that an specific ABI description would have
> > a rich format. Something like:
> >
> > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > Date: August 2010
> > Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
> > RST-Description:
> > It is possible to switch the cpi setting of the mouse with the
> > press of a button.
> > When read, this file returns the raw number of the actual cpi
> > setting reported by the mouse. This number has to be further
> > processed to receive the real dpi value:
> >
> > ===== =====
> > VALUE DPI
> > ===== =====
> > 1 400
> > 2 800
> > 4 1600
> > ===== =====
> >
> > With that, the script will know that the description contents will be using
> > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > is to place the description on a code-block, e. g. it will produce this
> > output for the description:
> >
> > ::
> >
> > It is possible to switch the cpi setting of the mouse with the
> > press of a button.
> > When read, this file returns the raw number of the actual cpi
> > setting reported by the mouse. This number has to be further
> > processed to receive the real dpi value:
> >
> > VALUE DPI
> > 1 400
> > 2 800
> > 4 1600
> >
> >
> > Greg,
> >
> > what do you think?
>
> I don't know when "Description" and "RST-Description" would be used.
> Why not just parse "Description" like rst text and if things are "messy"
> we fix them up as found, like you did with the ":" here? It doesn't
> have to be complex, we can always fix them up after-the-fact if new
> stuff gets added that doesn't quite parse properly.
>
> Just like we do for most kernel-doc formatting :)
Works for me. Yet, I guess I tried that, back on 2017.
If I'm not mistaken, the initial patchset to solve the broken things
won't be small, and will be require a lot of attention in order to
identify what's broken and where.
Btw, one thing is to pass at ReST validation. Another thing is to
produce something that people can read.
Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
is here:
if (!($desc =~ /^\s*$/)) {
if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
# put everything inside a code block
$desc =~ s/\n/\n /g;
print "::\n\n";
print " $desc\n\n";
} else {
# Escape any special chars from description
$desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
print "$desc\n\n";
}
}
If it discovers something weird enough, it just places everything
into a comment block. Otherwise, it assumes that it is a plain
text and that any special characters should be escaped.
If the above block is replaced by a simple:
print "$desc\n\n";
The description content will be handled as a ReST file.
I don't have any time right now to do this change and to handle the
warnings that will start to popup.
Btw, a single replace there is enough to show the amount of problems that
it will rise, as it will basically break Sphinx build with:
reading sources... [ 1%] admin-guide/abi-testing
reST markup error:
get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.
==========================
PCIe Device AER statistics
These attributes show up under all the devices that are AER capable. These
Thanks,
Mauro
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 7bc619b6890c..e75f441afdcd 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -288,18 +288,18 @@ sub output_rest {
$desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
if (!($desc =~ /^\s*$/)) {
- if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
- # put everything inside a code block
- $desc =~ s/\n/\n /g;
+# if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
+# # put everything inside a code block
+# $desc =~ s/\n/\n /g;
- print "::\n\n";
- print " $desc\n\n";
- } else {
- # Escape any special chars from description
- $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
+# print "::\n\n";
+# print " $desc\n\n";
+# } else {
+# # Escape any special chars from description
+# $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
print "$desc\n\n";
- }
+# }
} else {
print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file});
}
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-18 8:47 ` Jani Nikula
@ 2019-06-19 16:37 ` Mauro Carvalho Chehab
2019-06-21 14:27 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-19 16:37 UTC (permalink / raw)
To: Jani Nikula
Cc: Greg Kroah-Hartman, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet
Em Tue, 18 Jun 2019 11:47:32 +0300
Jani Nikula <jani.nikula@linux.intel.com> escreveu:
> On Mon, 17 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > Yeah, I guess it should be possible to do that. How a python script
> > can identify if it was called by Sphinx, or if it was called directly?
>
> if __name__ == '__main__':
> # run on the command-line, not imported
Ok, when I have some spare time, I may try to convert one script
to python and see how it behaves.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-19 15:02 ` Greg Kroah-Hartman
2019-06-19 16:14 ` Mauro Carvalho Chehab
@ 2019-06-20 12:01 ` Johan Hovold
2019-06-20 12:54 ` Greg Kroah-Hartman
1 sibling, 1 reply; 63+ messages in thread
From: Johan Hovold @ 2019-06-20 12:01 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Mauro Carvalho Chehab, Johan Hovold, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Stefan Achatz
On Wed, Jun 19, 2019 at 05:02:07PM +0200, Greg Kroah-Hartman wrote:
> On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > Hi Johan,
> >
> > Em Wed, 19 Jun 2019 14:51:35 +0200
> > Johan Hovold <johan@kernel.org> escreveu:
> >
> > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > > >
> > > > When parsing via script, it is important to know if the script
> > > > should consider a description as a literal block that should
> > > > be displayed as-is, or if the description can be considered
> > > > as a normal text.
> > > >
> > > > Change descriptions to ensure that the preceding line of a table
> > > > ends with a colon. That makes easy to identify the need of a
> > > > literal block.
> > >
> > > In the cover letter you say that the first four patches of this series,
> > > including this one, "fix some ABI descriptions that are violating the
> > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > given that it's you that is now *introducing* a new syntax requirement
> > > to assist your script.
> >
> > Yeah, what's there at the cover letter doesn't apply to this specific
> > patch. The thing is that I wrote this series a lot of time ago (2016/17).
Got it, thanks.
[...]
> > In the specific case of this patch, the ":" there actually makes sense
> > for someone that it is reading it as a text file, and it is an easy
> > hack to make it parse better.
Human readers probably depend on more on tabulation and white space.
When the preceding description wasn't using a colon to begin with (and
you just replace s/\./:/) it can even look weird, but no big deal.
> > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > how will anyone adding new ABI descriptions learn about it?
> >
> > Yeah, either that or provide an alternative to "Description" tag, to be
> > used with more complex ABI descriptions.
> >
> > One of the things that occurred to me, back on 2017, is that we should
> > have a way to to specify that an specific ABI description would have
> > a rich format. Something like:
[...]
> I don't know when "Description" and "RST-Description" would be used.
> Why not just parse "Description" like rst text and if things are "messy"
> we fix them up as found, like you did with the ":" here? It doesn't
> have to be complex, we can always fix them up after-the-fact if new
> stuff gets added that doesn't quite parse properly.
>
> Just like we do for most kernel-doc formatting :)
But kernel-doc has a documented format, which was sort of the point I
was trying to make. If the new get_abi.pl scripts expects a colon I
think it should be mentioned somewhere (e.g. Documentation/ABI/README).
Grepping for attribute entries in linux-next still reveals a number
descriptions that still lack that colon and use varying formatting. More
are bound to be added later, but perhaps that's ok depending on what
you're aiming at here.
Johan
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-20 12:01 ` Johan Hovold
@ 2019-06-20 12:54 ` Greg Kroah-Hartman
2019-06-20 14:20 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-20 12:54 UTC (permalink / raw)
To: Johan Hovold
Cc: Mauro Carvalho Chehab, Linux Doc Mailing List,
Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Stefan Achatz
On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > I don't know when "Description" and "RST-Description" would be used.
> > Why not just parse "Description" like rst text and if things are "messy"
> > we fix them up as found, like you did with the ":" here? It doesn't
> > have to be complex, we can always fix them up after-the-fact if new
> > stuff gets added that doesn't quite parse properly.
> >
> > Just like we do for most kernel-doc formatting :)
>
> But kernel-doc has a documented format, which was sort of the point I
> was trying to make. If the new get_abi.pl scripts expects a colon I
> think it should be mentioned somewhere (e.g. Documentation/ABI/README).
>
> Grepping for attribute entries in linux-next still reveals a number
> descriptions that still lack that colon and use varying formatting. More
> are bound to be added later, but perhaps that's ok depending on what
> you're aiming at here.
I'm aiming for "good enough" to start with, and then we can work through
the exceptions.
But given that Mauro hasn't resent the script that does the conversion
of the files, I don't know if that will even matter... {hint}
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-20 12:54 ` Greg Kroah-Hartman
@ 2019-06-20 14:20 ` Mauro Carvalho Chehab
2019-06-20 16:29 ` Greg Kroah-Hartman
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-20 14:20 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
Em Thu, 20 Jun 2019 14:54:13 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > I don't know when "Description" and "RST-Description" would be used.
> > > Why not just parse "Description" like rst text and if things are "messy"
> > > we fix them up as found, like you did with the ":" here? It doesn't
> > > have to be complex, we can always fix them up after-the-fact if new
> > > stuff gets added that doesn't quite parse properly.
> > >
> > > Just like we do for most kernel-doc formatting :)
> >
> > But kernel-doc has a documented format, which was sort of the point I
> > was trying to make. If the new get_abi.pl scripts expects a colon I
> > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> >
> > Grepping for attribute entries in linux-next still reveals a number
> > descriptions that still lack that colon and use varying formatting. More
> > are bound to be added later, but perhaps that's ok depending on what
> > you're aiming at here.
>
> I'm aiming for "good enough" to start with, and then we can work through
> the exceptions.
>
> But given that Mauro hasn't resent the script that does the conversion
> of the files, I don't know if that will even matter... {hint}
It sounds I missed something... are you expecting a new version?
If so, what changes are you expecting?
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-19 16:14 ` Mauro Carvalho Chehab
@ 2019-06-20 14:27 ` Mauro Carvalho Chehab
2019-06-21 7:21 ` Greg Kroah-Hartman
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-20 14:27 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
Em Wed, 19 Jun 2019 13:14:08 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:
> Em Wed, 19 Jun 2019 17:02:07 +0200
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
>
> > On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > > Hi Johan,
> > >
> > > Em Wed, 19 Jun 2019 14:51:35 +0200
> > > Johan Hovold <johan@kernel.org> escreveu:
> > >
> > > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > > > >
> > > > > When parsing via script, it is important to know if the script
> > > > > should consider a description as a literal block that should
> > > > > be displayed as-is, or if the description can be considered
> > > > > as a normal text.
> > > > >
> > > > > Change descriptions to ensure that the preceding line of a table
> > > > > ends with a colon. That makes easy to identify the need of a
> > > > > literal block.
> > > >
> > > > In the cover letter you say that the first four patches of this series,
> > > > including this one, "fix some ABI descriptions that are violating the
> > > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > > given that it's you that is now *introducing* a new syntax requirement
> > > > to assist your script.
> > >
> > > Yeah, what's there at the cover letter doesn't apply to this specific
> > > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> > >
> > > I revived those per a request at KS ML, as we still need to expose the
> > > ABI content on some book that will be used by userspace people.
> > >
> > > So, I just rebased it on the top of curent Kernel, add a cover letter
> > > with the things I remembered and re-sent.
> > >
> > > In the specific case of this patch, the ":" there actually makes sense
> > > for someone that it is reading it as a text file, and it is an easy
> > > hack to make it parse better.
> > >
> > > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > > how will anyone adding new ABI descriptions learn about it?
> > >
> > > Yeah, either that or provide an alternative to "Description" tag, to be
> > > used with more complex ABI descriptions.
> > >
> > > One of the things that occurred to me, back on 2017, is that we should
> > > have a way to to specify that an specific ABI description would have
> > > a rich format. Something like:
> > >
> > > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > > Date: August 2010
> > > Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
> > > RST-Description:
> > > It is possible to switch the cpi setting of the mouse with the
> > > press of a button.
> > > When read, this file returns the raw number of the actual cpi
> > > setting reported by the mouse. This number has to be further
> > > processed to receive the real dpi value:
> > >
> > > ===== =====
> > > VALUE DPI
> > > ===== =====
> > > 1 400
> > > 2 800
> > > 4 1600
> > > ===== =====
> > >
> > > With that, the script will know that the description contents will be using
> > > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > > is to place the description on a code-block, e. g. it will produce this
> > > output for the description:
> > >
> > > ::
> > >
> > > It is possible to switch the cpi setting of the mouse with the
> > > press of a button.
> > > When read, this file returns the raw number of the actual cpi
> > > setting reported by the mouse. This number has to be further
> > > processed to receive the real dpi value:
> > >
> > > VALUE DPI
> > > 1 400
> > > 2 800
> > > 4 1600
> > >
> > >
> > > Greg,
> > >
> > > what do you think?
> >
> > I don't know when "Description" and "RST-Description" would be used.
> > Why not just parse "Description" like rst text and if things are "messy"
> > we fix them up as found, like you did with the ":" here? It doesn't
> > have to be complex, we can always fix them up after-the-fact if new
> > stuff gets added that doesn't quite parse properly.
> >
> > Just like we do for most kernel-doc formatting :)
>
> Works for me. Yet, I guess I tried that, back on 2017.
>
> If I'm not mistaken, the initial patchset to solve the broken things
> won't be small, and will be require a lot of attention in order to
> identify what's broken and where.
>
> Btw, one thing is to pass at ReST validation. Another thing is to
> produce something that people can read.
>
> Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
> is here:
>
> if (!($desc =~ /^\s*$/)) {
> if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
> # put everything inside a code block
> $desc =~ s/\n/\n /g;
>
> print "::\n\n";
> print " $desc\n\n";
> } else {
> # Escape any special chars from description
> $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
>
> print "$desc\n\n";
> }
> }
>
> If it discovers something weird enough, it just places everything
> into a comment block. Otherwise, it assumes that it is a plain
> text and that any special characters should be escaped.
>
> If the above block is replaced by a simple:
>
> print "$desc\n\n";
>
> The description content will be handled as a ReST file.
>
> I don't have any time right now to do this change and to handle the
> warnings that will start to popup.
>
> Btw, a single replace there is enough to show the amount of problems that
> it will rise, as it will basically break Sphinx build with:
>
> reading sources... [ 1%] admin-guide/abi-testing
> reST markup error:
> get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.
>
> ==========================
> PCIe Device AER statistics
> These attributes show up under all the devices that are AER capable. These
To be clear here: the problem with the above is that ReST has zero
tolerance and actually behaves like a crash, if it receives something like:
==========================
PCIe Device AER statistics
==========================
For it to work, it has to have zero spaces before ===..=== line, e. g.:
==========================
PCIe Device AER statistics
==========================
Ok, maybe we could try to teach the parser a way to identify the initial
spacing of the first description line and remove that amount of
spaces/tabs for the following lines, but it may require some heuristics.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-20 14:20 ` Mauro Carvalho Chehab
@ 2019-06-20 16:29 ` Greg Kroah-Hartman
2019-06-20 17:16 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-20 16:29 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
On Thu, Jun 20, 2019 at 11:20:34AM -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 20 Jun 2019 14:54:13 +0200
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
>
> > On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > > I don't know when "Description" and "RST-Description" would be used.
> > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > have to be complex, we can always fix them up after-the-fact if new
> > > > stuff gets added that doesn't quite parse properly.
> > > >
> > > > Just like we do for most kernel-doc formatting :)
> > >
> > > But kernel-doc has a documented format, which was sort of the point I
> > > was trying to make. If the new get_abi.pl scripts expects a colon I
> > > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> > >
> > > Grepping for attribute entries in linux-next still reveals a number
> > > descriptions that still lack that colon and use varying formatting. More
> > > are bound to be added later, but perhaps that's ok depending on what
> > > you're aiming at here.
> >
> > I'm aiming for "good enough" to start with, and then we can work through
> > the exceptions.
> >
> > But given that Mauro hasn't resent the script that does the conversion
> > of the files, I don't know if that will even matter... {hint}
>
> It sounds I missed something... are you expecting a new version?
Yes, the last round of patches didn't have a SPDX header on the script,
so I couldn't add it to the tree :(
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-20 16:29 ` Greg Kroah-Hartman
@ 2019-06-20 17:16 ` Mauro Carvalho Chehab
2019-06-20 17:55 ` Greg Kroah-Hartman
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-20 17:16 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
Em Thu, 20 Jun 2019 18:29:45 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> On Thu, Jun 20, 2019 at 11:20:34AM -0300, Mauro Carvalho Chehab wrote:
> > Em Thu, 20 Jun 2019 14:54:13 +0200
> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> >
> > > On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > > > I don't know when "Description" and "RST-Description" would be used.
> > > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > > have to be complex, we can always fix them up after-the-fact if new
> > > > > stuff gets added that doesn't quite parse properly.
> > > > >
> > > > > Just like we do for most kernel-doc formatting :)
> > > >
> > > > But kernel-doc has a documented format, which was sort of the point I
> > > > was trying to make. If the new get_abi.pl scripts expects a colon I
> > > > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> > > >
> > > > Grepping for attribute entries in linux-next still reveals a number
> > > > descriptions that still lack that colon and use varying formatting. More
> > > > are bound to be added later, but perhaps that's ok depending on what
> > > > you're aiming at here.
> > >
> > > I'm aiming for "good enough" to start with, and then we can work through
> > > the exceptions.
> > >
> > > But given that Mauro hasn't resent the script that does the conversion
> > > of the files, I don't know if that will even matter... {hint}
> >
> > It sounds I missed something... are you expecting a new version?
>
> Yes, the last round of patches didn't have a SPDX header on the script,
> so I couldn't add it to the tree :(
I could swear I sent you a version with SPDX on it... anyway, I'm
re-sending the hole thing. The SPDX header addition is on a separate
patch.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-20 17:16 ` Mauro Carvalho Chehab
@ 2019-06-20 17:55 ` Greg Kroah-Hartman
0 siblings, 0 replies; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-20 17:55 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
On Thu, Jun 20, 2019 at 02:16:28PM -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 20 Jun 2019 18:29:45 +0200
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
>
> > On Thu, Jun 20, 2019 at 11:20:34AM -0300, Mauro Carvalho Chehab wrote:
> > > Em Thu, 20 Jun 2019 14:54:13 +0200
> > > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> > >
> > > > On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > > > > I don't know when "Description" and "RST-Description" would be used.
> > > > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > > > have to be complex, we can always fix them up after-the-fact if new
> > > > > > stuff gets added that doesn't quite parse properly.
> > > > > >
> > > > > > Just like we do for most kernel-doc formatting :)
> > > > >
> > > > > But kernel-doc has a documented format, which was sort of the point I
> > > > > was trying to make. If the new get_abi.pl scripts expects a colon I
> > > > > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> > > > >
> > > > > Grepping for attribute entries in linux-next still reveals a number
> > > > > descriptions that still lack that colon and use varying formatting. More
> > > > > are bound to be added later, but perhaps that's ok depending on what
> > > > > you're aiming at here.
> > > >
> > > > I'm aiming for "good enough" to start with, and then we can work through
> > > > the exceptions.
> > > >
> > > > But given that Mauro hasn't resent the script that does the conversion
> > > > of the files, I don't know if that will even matter... {hint}
> > >
> > > It sounds I missed something... are you expecting a new version?
> >
> > Yes, the last round of patches didn't have a SPDX header on the script,
> > so I couldn't add it to the tree :(
>
> I could swear I sent you a version with SPDX on it... anyway, I'm
> re-sending the hole thing. The SPDX header addition is on a separate
> patch.
Ah, you didn't cc: me on the whole thread this time, that's why I missed
it last time. I sucked it off lkml now, will look at it in the morning.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-20 14:27 ` Mauro Carvalho Chehab
@ 2019-06-21 7:21 ` Greg Kroah-Hartman
2019-06-21 9:49 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Greg Kroah-Hartman @ 2019-06-21 7:21 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
On Thu, Jun 20, 2019 at 11:27:49AM -0300, Mauro Carvalho Chehab wrote:
> Em Wed, 19 Jun 2019 13:14:08 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:
>
> > Em Wed, 19 Jun 2019 17:02:07 +0200
> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> >
> > > On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > > > Hi Johan,
> > > >
> > > > Em Wed, 19 Jun 2019 14:51:35 +0200
> > > > Johan Hovold <johan@kernel.org> escreveu:
> > > >
> > > > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > > > > >
> > > > > > When parsing via script, it is important to know if the script
> > > > > > should consider a description as a literal block that should
> > > > > > be displayed as-is, or if the description can be considered
> > > > > > as a normal text.
> > > > > >
> > > > > > Change descriptions to ensure that the preceding line of a table
> > > > > > ends with a colon. That makes easy to identify the need of a
> > > > > > literal block.
> > > > >
> > > > > In the cover letter you say that the first four patches of this series,
> > > > > including this one, "fix some ABI descriptions that are violating the
> > > > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > > > given that it's you that is now *introducing* a new syntax requirement
> > > > > to assist your script.
> > > >
> > > > Yeah, what's there at the cover letter doesn't apply to this specific
> > > > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> > > >
> > > > I revived those per a request at KS ML, as we still need to expose the
> > > > ABI content on some book that will be used by userspace people.
> > > >
> > > > So, I just rebased it on the top of curent Kernel, add a cover letter
> > > > with the things I remembered and re-sent.
> > > >
> > > > In the specific case of this patch, the ":" there actually makes sense
> > > > for someone that it is reading it as a text file, and it is an easy
> > > > hack to make it parse better.
> > > >
> > > > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > > > how will anyone adding new ABI descriptions learn about it?
> > > >
> > > > Yeah, either that or provide an alternative to "Description" tag, to be
> > > > used with more complex ABI descriptions.
> > > >
> > > > One of the things that occurred to me, back on 2017, is that we should
> > > > have a way to to specify that an specific ABI description would have
> > > > a rich format. Something like:
> > > >
> > > > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > > > Date: August 2010
> > > > Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
> > > > RST-Description:
> > > > It is possible to switch the cpi setting of the mouse with the
> > > > press of a button.
> > > > When read, this file returns the raw number of the actual cpi
> > > > setting reported by the mouse. This number has to be further
> > > > processed to receive the real dpi value:
> > > >
> > > > ===== =====
> > > > VALUE DPI
> > > > ===== =====
> > > > 1 400
> > > > 2 800
> > > > 4 1600
> > > > ===== =====
> > > >
> > > > With that, the script will know that the description contents will be using
> > > > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > > > is to place the description on a code-block, e. g. it will produce this
> > > > output for the description:
> > > >
> > > > ::
> > > >
> > > > It is possible to switch the cpi setting of the mouse with the
> > > > press of a button.
> > > > When read, this file returns the raw number of the actual cpi
> > > > setting reported by the mouse. This number has to be further
> > > > processed to receive the real dpi value:
> > > >
> > > > VALUE DPI
> > > > 1 400
> > > > 2 800
> > > > 4 1600
> > > >
> > > >
> > > > Greg,
> > > >
> > > > what do you think?
> > >
> > > I don't know when "Description" and "RST-Description" would be used.
> > > Why not just parse "Description" like rst text and if things are "messy"
> > > we fix them up as found, like you did with the ":" here? It doesn't
> > > have to be complex, we can always fix them up after-the-fact if new
> > > stuff gets added that doesn't quite parse properly.
> > >
> > > Just like we do for most kernel-doc formatting :)
> >
> > Works for me. Yet, I guess I tried that, back on 2017.
> >
> > If I'm not mistaken, the initial patchset to solve the broken things
> > won't be small, and will be require a lot of attention in order to
> > identify what's broken and where.
> >
> > Btw, one thing is to pass at ReST validation. Another thing is to
> > produce something that people can read.
> >
> > Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
> > is here:
> >
> > if (!($desc =~ /^\s*$/)) {
> > if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
> > # put everything inside a code block
> > $desc =~ s/\n/\n /g;
> >
> > print "::\n\n";
> > print " $desc\n\n";
> > } else {
> > # Escape any special chars from description
> > $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
> >
> > print "$desc\n\n";
> > }
> > }
> >
> > If it discovers something weird enough, it just places everything
> > into a comment block. Otherwise, it assumes that it is a plain
> > text and that any special characters should be escaped.
> >
> > If the above block is replaced by a simple:
> >
> > print "$desc\n\n";
> >
> > The description content will be handled as a ReST file.
> >
> > I don't have any time right now to do this change and to handle the
> > warnings that will start to popup.
> >
> > Btw, a single replace there is enough to show the amount of problems that
> > it will rise, as it will basically break Sphinx build with:
> >
> > reading sources... [ 1%] admin-guide/abi-testing
> > reST markup error:
> > get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.
> >
> > ==========================
> > PCIe Device AER statistics
> > These attributes show up under all the devices that are AER capable. These
>
> To be clear here: the problem with the above is that ReST has zero
> tolerance and actually behaves like a crash, if it receives something like:
>
> ==========================
> PCIe Device AER statistics
> ==========================
>
> For it to work, it has to have zero spaces before ===..=== line, e. g.:
>
> ==========================
> PCIe Device AER statistics
> ==========================
>
> Ok, maybe we could try to teach the parser a way to identify the initial
> spacing of the first description line and remove that amount of
> spaces/tabs for the following lines, but it may require some heuristics.
Or we can clean this type of thing up by hand :)
Let's see how bad this gets, the documentation in these files should not
be very complex as they _should_ only be one-value-per-file, but as you
have shown in this very example, that rule is violated :(
thanks,
greg k-h
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 04/14] ABI: better identificate tables
2019-06-21 7:21 ` Greg Kroah-Hartman
@ 2019-06-21 9:49 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-21 9:49 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Johan Hovold, Linux Doc Mailing List, Mauro Carvalho Chehab,
Mauro Carvalho Chehab, linux-kernel, Jonathan Corbet,
Stefan Achatz
Em Fri, 21 Jun 2019 09:21:52 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> On Thu, Jun 20, 2019 at 11:27:49AM -0300, Mauro Carvalho Chehab wrote:
> > Em Wed, 19 Jun 2019 13:14:08 -0300
> > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:
> >
> > > Em Wed, 19 Jun 2019 17:02:07 +0200
> > > Greg Kroah-Hartman <gregkh@linuxfoundation.org> escreveu:
> > >
> > > > On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > > > > Hi Johan,
> > > > >
> > > > > Em Wed, 19 Jun 2019 14:51:35 +0200
> > > > > Johan Hovold <johan@kernel.org> escreveu:
> > > > >
> > > > > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > > > > From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > > > > > >
> > > > > > > When parsing via script, it is important to know if the script
> > > > > > > should consider a description as a literal block that should
> > > > > > > be displayed as-is, or if the description can be considered
> > > > > > > as a normal text.
> > > > > > >
> > > > > > > Change descriptions to ensure that the preceding line of a table
> > > > > > > ends with a colon. That makes easy to identify the need of a
> > > > > > > literal block.
> > > > > >
> > > > > > In the cover letter you say that the first four patches of this series,
> > > > > > including this one, "fix some ABI descriptions that are violating the
> > > > > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > > > > given that it's you that is now *introducing* a new syntax requirement
> > > > > > to assist your script.
> > > > >
> > > > > Yeah, what's there at the cover letter doesn't apply to this specific
> > > > > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> > > > >
> > > > > I revived those per a request at KS ML, as we still need to expose the
> > > > > ABI content on some book that will be used by userspace people.
> > > > >
> > > > > So, I just rebased it on the top of curent Kernel, add a cover letter
> > > > > with the things I remembered and re-sent.
> > > > >
> > > > > In the specific case of this patch, the ":" there actually makes sense
> > > > > for someone that it is reading it as a text file, and it is an easy
> > > > > hack to make it parse better.
> > > > >
> > > > > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > > > > how will anyone adding new ABI descriptions learn about it?
> > > > >
> > > > > Yeah, either that or provide an alternative to "Description" tag, to be
> > > > > used with more complex ABI descriptions.
> > > > >
> > > > > One of the things that occurred to me, back on 2017, is that we should
> > > > > have a way to to specify that an specific ABI description would have
> > > > > a rich format. Something like:
> > > > >
> > > > > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > > > > Date: August 2010
> > > > > Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
> > > > > RST-Description:
> > > > > It is possible to switch the cpi setting of the mouse with the
> > > > > press of a button.
> > > > > When read, this file returns the raw number of the actual cpi
> > > > > setting reported by the mouse. This number has to be further
> > > > > processed to receive the real dpi value:
> > > > >
> > > > > ===== =====
> > > > > VALUE DPI
> > > > > ===== =====
> > > > > 1 400
> > > > > 2 800
> > > > > 4 1600
> > > > > ===== =====
> > > > >
> > > > > With that, the script will know that the description contents will be using
> > > > > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > > > > is to place the description on a code-block, e. g. it will produce this
> > > > > output for the description:
> > > > >
> > > > > ::
> > > > >
> > > > > It is possible to switch the cpi setting of the mouse with the
> > > > > press of a button.
> > > > > When read, this file returns the raw number of the actual cpi
> > > > > setting reported by the mouse. This number has to be further
> > > > > processed to receive the real dpi value:
> > > > >
> > > > > VALUE DPI
> > > > > 1 400
> > > > > 2 800
> > > > > 4 1600
> > > > >
> > > > >
> > > > > Greg,
> > > > >
> > > > > what do you think?
> > > >
> > > > I don't know when "Description" and "RST-Description" would be used.
> > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > have to be complex, we can always fix them up after-the-fact if new
> > > > stuff gets added that doesn't quite parse properly.
> > > >
> > > > Just like we do for most kernel-doc formatting :)
> > >
> > > Works for me. Yet, I guess I tried that, back on 2017.
> > >
> > > If I'm not mistaken, the initial patchset to solve the broken things
> > > won't be small, and will be require a lot of attention in order to
> > > identify what's broken and where.
> > >
> > > Btw, one thing is to pass at ReST validation. Another thing is to
> > > produce something that people can read.
> > >
> > > Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
> > > is here:
> > >
> > > if (!($desc =~ /^\s*$/)) {
> > > if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
> > > # put everything inside a code block
> > > $desc =~ s/\n/\n /g;
> > >
> > > print "::\n\n";
> > > print " $desc\n\n";
> > > } else {
> > > # Escape any special chars from description
> > > $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
> > >
> > > print "$desc\n\n";
> > > }
> > > }
> > >
> > > If it discovers something weird enough, it just places everything
> > > into a comment block. Otherwise, it assumes that it is a plain
> > > text and that any special characters should be escaped.
> > >
> > > If the above block is replaced by a simple:
> > >
> > > print "$desc\n\n";
> > >
> > > The description content will be handled as a ReST file.
> > >
> > > I don't have any time right now to do this change and to handle the
> > > warnings that will start to popup.
> > >
> > > Btw, a single replace there is enough to show the amount of problems that
> > > it will rise, as it will basically break Sphinx build with:
> > >
> > > reading sources... [ 1%] admin-guide/abi-testing
> > > reST markup error:
> > > get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.
> > >
> > > ==========================
> > > PCIe Device AER statistics
> > > These attributes show up under all the devices that are AER capable. These
> >
> > To be clear here: the problem with the above is that ReST has zero
> > tolerance and actually behaves like a crash, if it receives something like:
> >
> > ==========================
> > PCIe Device AER statistics
> > ==========================
> >
> > For it to work, it has to have zero spaces before ===..=== line, e. g.:
> >
> > ==========================
> > PCIe Device AER statistics
> > ==========================
> >
> > Ok, maybe we could try to teach the parser a way to identify the initial
> > spacing of the first description line and remove that amount of
> > spaces/tabs for the following lines, but it may require some heuristics.
>
> Or we can clean this type of thing up by hand :)
>
> Let's see how bad this gets, the documentation in these files should not
> be very complex as they _should_ only be one-value-per-file, but as you
> have shown in this very example, that rule is violated :(
Hi Greg,
Doing a manual review - still incomplete (just stable/ dir) - and I didn't
try to run the script over ABI file yet - this is what I found so
far[1]: 17 files over 37 needed adjustments (45%).
That trick of detecting a complex file when it finds a ':' and parse the
hole description as a literal block solves most of them.
From the changes below, this file:
Documentation/ABI/stable/sysfs-driver-mlxreg-io
Found a "creative way" to place the "What:" field avoiding long lines...
At least this one should be fixed, as the parser won't handle its
content right (it won't complain, but the result output will be wrong).
[1] I also added a note at README that files should be ReST-compatible
diff --git a/Documentation/ABI/README b/Documentation/ABI/README
index 3121029dce21..8bac9cb09a6d 100644
--- a/Documentation/ABI/README
+++ b/Documentation/ABI/README
@@ -32,7 +32,7 @@ The different levels of stability are:
layout of the files below for details on how to do this.)
obsolete/
- This directory documents interfaces that are still remaining in
+ This directory documents interfaces that are still remaining in
the kernel, but are marked to be removed at some later point in
time. The description of the interface will document the reason
why it is obsolete and when it can be expected to be removed.
@@ -58,6 +58,14 @@ Users: All users of this interface who wish to be notified when
be changed further.
+Note:
+ The fields should be use a simple notation, compatible with ReST markup.
+ Also, the file **should not** have a top-level index, like::
+
+ ===
+ foo
+ ===
+
How things move between levels:
Interfaces in stable may move to obsolete, as long as the proper
diff --git a/Documentation/ABI/stable/firewire-cdev b/Documentation/ABI/stable/firewire-cdev
index f72ed653878a..d8df3004c318 100644
--- a/Documentation/ABI/stable/firewire-cdev
+++ b/Documentation/ABI/stable/firewire-cdev
@@ -20,6 +20,7 @@ Description:
- Query node ID
- Query maximum speed of the path between this node
and local node
+
- The 1394 bus (i.e. "card") to which the node is attached to:
- Isochronous stream transmission and reception
- Asynchronous stream transmission and reception
@@ -31,6 +32,7 @@ Description:
manager
- Query cycle time
- Bus reset initiation, bus reset event reception
+
- All 1394 buses:
- Allocation of IEEE 1212 address ranges on the local
link layers, reception of inbound requests to such
diff --git a/Documentation/ABI/stable/sysfs-acpi-pmprofile b/Documentation/ABI/stable/sysfs-acpi-pmprofile
index 964c7a8afb26..3b0e34eef83c 100644
--- a/Documentation/ABI/stable/sysfs-acpi-pmprofile
+++ b/Documentation/ABI/stable/sysfs-acpi-pmprofile
@@ -11,12 +11,15 @@ Values: For possible values see ACPI specification:
Field: Preferred_PM_Profile
Currently these values are defined by spec:
- 0 Unspecified
- 1 Desktop
- 2 Mobile
- 3 Workstation
- 4 Enterprise Server
- 5 SOHO Server
- 6 Appliance PC
- 7 Performance Server
+
+ == =================
+ 0 Unspecified
+ 1 Desktop
+ 2 Mobile
+ 3 Workstation
+ 4 Enterprise Server
+ 5 SOHO Server
+ 6 Appliance PC
+ 7 Performance Server
>7 Reserved
+ == =================
diff --git a/Documentation/ABI/stable/sysfs-bus-firewire b/Documentation/ABI/stable/sysfs-bus-firewire
index 41e5a0cd1e3e..0c8c023855b6 100644
--- a/Documentation/ABI/stable/sysfs-bus-firewire
+++ b/Documentation/ABI/stable/sysfs-bus-firewire
@@ -125,7 +125,9 @@ Description:
Read-only attribute, immutable during the target's lifetime.
Format, as exposed by firewire-sbp2 since 2.6.22, May 2007:
Colon-separated hexadecimal string representations of
+
u64 EUI-64 : u24 directory_ID : u16 LUN
+
without 0x prefixes, without whitespace. The former sbp2 driver
(removed in 2.6.37 after being superseded by firewire-sbp2) used
a somewhat shorter format which was not as close to SAM.
diff --git a/Documentation/ABI/stable/sysfs-bus-nvmem b/Documentation/ABI/stable/sysfs-bus-nvmem
index 9ffba8576f7b..c399323f37de 100644
--- a/Documentation/ABI/stable/sysfs-bus-nvmem
+++ b/Documentation/ABI/stable/sysfs-bus-nvmem
@@ -9,13 +9,14 @@ Description:
Note: This file is only present if CONFIG_NVMEM_SYSFS
is enabled
- ex:
- hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
+ ex::
- 0000000 0000 0000 0000 0000 0000 0000 0000 0000
- *
- 00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
- 0000000 0000 0000 0000 0000 0000 0000 0000 0000
- ...
- *
- 0001000
+ hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
+
+ 0000000 0000 0000 0000 0000 0000 0000 0000 0000
+ *
+ 00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
+ 0000000 0000 0000 0000 0000 0000 0000 0000 0000
+ ...
+ *
+ 0001000
diff --git a/Documentation/ABI/stable/sysfs-bus-usb b/Documentation/ABI/stable/sysfs-bus-usb
index b832eeff9999..cad4bc232520 100644
--- a/Documentation/ABI/stable/sysfs-bus-usb
+++ b/Documentation/ABI/stable/sysfs-bus-usb
@@ -50,8 +50,10 @@ Description:
Tools can use this file and the connected_duration file to
compute the percentage of time that a device has been active.
- For example,
- echo $((100 * `cat active_duration` / `cat connected_duration`))
+ For example::
+
+ echo $((100 * `cat active_duration` / `cat connected_duration`))
+
will give an integer percentage. Note that this does not
account for counter wrap.
Users:
diff --git a/Documentation/ABI/stable/sysfs-class-backlight b/Documentation/ABI/stable/sysfs-class-backlight
index 70302f370e7e..023fb52645f8 100644
--- a/Documentation/ABI/stable/sysfs-class-backlight
+++ b/Documentation/ABI/stable/sysfs-class-backlight
@@ -4,6 +4,7 @@ KernelVersion: 2.6.12
Contact: Richard Purdie <rpurdie@rpsys.net>
Description:
Control BACKLIGHT power, values are FB_BLANK_* from fb.h
+
- FB_BLANK_UNBLANK (0) : power on.
- FB_BLANK_POWERDOWN (4) : power off
Users: HAL
diff --git a/Documentation/ABI/stable/sysfs-class-infiniband b/Documentation/ABI/stable/sysfs-class-infiniband
index 17211ceb9bf4..3f58952a8ccb 100644
--- a/Documentation/ABI/stable/sysfs-class-infiniband
+++ b/Documentation/ABI/stable/sysfs-class-infiniband
@@ -8,12 +8,14 @@ Date: Apr, 2005
KernelVersion: v2.6.12
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ===========================================
node_type: (RO) Node type (CA, RNIC, usNIC, usNIC UDP,
switch or router)
node_guid: (RO) Node GUID
sys_image_guid: (RO) System image GUID
+ =============== ===========================================
What: /sys/class/infiniband/<device>/node_desc
@@ -47,6 +49,7 @@ KernelVersion: v2.6.12
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ===============================================
lid: (RO) Port LID
rate: (RO) Port data rate (active width * active
@@ -66,8 +69,9 @@ Description:
cap_mask: (RO) Port capability mask. 2 bits here are
settable- IsCommunicationManagementSupported
- (set when CM module is loaded) and IsSM (set via
- open of issmN file).
+ (set when CM module is loaded) and IsSM (set
+ via open of issmN file).
+ =============== ===============================================
What: /sys/class/infiniband/<device>/ports/<port-num>/link_layer
@@ -143,7 +147,7 @@ Description:
poor link signal integrity
Data info:
- ---------
+ ----------
port_xmit_data: (RO) Total number of data octets, divided by 4
(lanes), transmitted on all VLs. This is 64 bit counter
@@ -177,7 +181,7 @@ Description:
packets with errors.
Misc info:
- ---------
+ ----------
port_xmit_discards: (RO) Total number of outbound packets
discarded by the port because the port is down or congested.
@@ -244,9 +248,11 @@ Description:
two umad devices and two issm devices, while a switch will have
one device of each type (for switch port 0).
+ ======= =====================================
ibdev: (RO) Show Infiniband (IB) device name
port: (RO) Display port number
+ ======= =====================================
What: /sys/class/infiniband_mad/abi_version
@@ -281,10 +287,12 @@ Date: Sept, 2005
KernelVersion: v2.6.14
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ===========================================
ibdev: (RO) Display Infiniband (IB) device name
abi_version: (RO) Show ABI version of IB device specific
interfaces.
+ =============== ===========================================
What: /sys/class/infiniband_verbs/abi_version
@@ -306,12 +314,14 @@ Date: Apr, 2005
KernelVersion: v2.6.12
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ================================================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Host Channel Adapter type: MT23108, MT25208
(MT23108 compat mode), MT25208 or MT25204
board_id: (RO) Manufacturing board ID
+ =============== ================================================
sysfs interface for Chelsio T3 RDMA Driver (cxgb3)
@@ -324,6 +334,7 @@ Date: Feb, 2007
KernelVersion: v2.6.21
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ==============================================
hw_rev: (RO) Hardware revision number
hca_type: (RO) HCA type. Here it is a driver short name.
@@ -331,6 +342,7 @@ Description:
driver structure (e.g. pci_driver::name).
board_id: (RO) Manufacturing board id
+ =============== ==============================================
sysfs interface for Mellanox ConnectX HCA IB driver (mlx4)
@@ -343,11 +355,13 @@ Date: Sep, 2007
KernelVersion: v2.6.24
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ===============================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Host channel adapter type
board_id: (RO) Manufacturing board ID
+ =============== ===============================
What: /sys/class/infiniband/mlx4_X/iov/ports/<port-num>/gids/<n>
@@ -373,6 +387,7 @@ Description:
example, ports/1/pkeys/10 contains the value at index 10 in port
1's P_Key table.
+ ======================= ==========================================
gids/<n>: (RO) The physical port gids n = 0..127
admin_guids/<n>: (RW) Allows examining or changing the
@@ -401,6 +416,7 @@ Description:
guest, whenever it uses its pkey index
1, will actually be using the real pkey
index 10.
+ ======================= ==========================================
What: /sys/class/infiniband/mlx4_X/iov/<pci-slot-num>/ports/<m>/smi_enabled
@@ -412,12 +428,14 @@ Description:
Enabling QP0 on VFs for selected VF/port. By default, no VFs are
enabled for QP0 operation.
- smi_enabled: (RO) Indicates whether smi is currently enabled
- for the indicated VF/port
+ ================= ==== ===========================================
+ smi_enabled: (RO) Indicates whether smi is currently enabled
+ for the indicated VF/port
- enable_smi_admin:(RW) Used by the admin to request that smi
- capability be enabled or disabled for the
- indicated VF/port. 0 = disable, 1 = enable.
+ enable_smi_admin: (RW) Used by the admin to request that smi
+ capability be enabled or disabled for the
+ indicated VF/port. 0 = disable, 1 = enable.
+ ================= ==== ===========================================
The requested enablement will occur at the next reset of the VF
(e.g. driver restart on the VM which owns the VF).
@@ -433,11 +451,13 @@ Date: Feb, 2008
KernelVersion: v2.6.25
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ==== ==================================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Host Channel Adapter type (NEX020)
board_id: (RO) Manufacturing board id
+ =============== ==== ==================================
sysfs interface for Chelsio T4/T5 RDMA driver (cxgb4)
@@ -451,6 +471,7 @@ KernelVersion: v2.6.35
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== =============================================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Driver short name. Should normally match
@@ -459,6 +480,7 @@ Description:
board_id: (RO) Manufacturing board id. (Vendor + device
information)
+ =============== =============================================
sysfs interface for Intel IB driver qib
@@ -479,6 +501,7 @@ Date: May, 2010
KernelVersion: v2.6.35
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ======================================================
version: (RO) Display version information of installed software
and drivers.
@@ -505,6 +528,7 @@ Description:
chip_reset: (WO) Reset the chip if possible by writing
"reset" to this file. Only allowed if no user
contexts are open that use chip resources.
+ =============== ======================================================
What: /sys/class/infiniband/qibX/ports/N/sl2vl/[0-15]
@@ -524,6 +548,7 @@ Contact: linux-rdma@vger.kernel.org
Description:
Per-port congestion control. Both are binary attributes.
+ =============== ================================================
cc_table_bin: (RO) Congestion control table size followed by
table entries.
@@ -532,6 +557,7 @@ Description:
congestion entries - increase, timer, event log
trigger threshold and the minimum injection rate
delay.
+ =============== ================================================
What: /sys/class/infiniband/qibX/ports/N/linkstate/loopback
What: /sys/class/infiniband/qibX/ports/N/linkstate/led_override
@@ -544,6 +570,7 @@ Contact: linux-rdma@vger.kernel.org
Description:
[to be documented]
+ =============== ===============================================
loopback: (WO)
led_override: (WO)
hrtbt_enable: (RW)
@@ -554,6 +581,7 @@ Description:
errors. Possible states are- "Initted",
"Present", "IB_link_up", "IB_configured" or
"Fatal_Hardware_Error".
+ =============== ===============================================
What: /sys/class/infiniband/qibX/ports/N/diag_counters/rc_resends
What: /sys/class/infiniband/qibX/ports/N/diag_counters/seq_naks
@@ -602,6 +630,7 @@ Contact: Christian Benvenuti <benve@cisco.com>,
linux-rdma@vger.kernel.org
Description:
+ =============== ===============================================
board_id: (RO) Manufacturing board id
config: (RO) Report the configuration for this PF
@@ -614,6 +643,7 @@ Description:
iface: (RO) Shows which network interface this usNIC
entry is associated to (visible with ifconfig).
+ =============== ===============================================
What: /sys/class/infiniband/usnic_X/qpn/summary
What: /sys/class/infiniband/usnic_X/qpn/context
@@ -658,6 +688,7 @@ Date: May, 2016
KernelVersion: v4.6
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ==== ========================================
hw_rev: (RO) Hardware revision number
board_id: (RO) Manufacturing board id
@@ -676,6 +707,7 @@ Description:
available.
tempsense: (RO) Thermal sense information
+ =============== ==== ========================================
What: /sys/class/infiniband/hfi1_X/ports/N/CCMgtA/cc_settings_bin
@@ -687,19 +719,21 @@ Contact: linux-rdma@vger.kernel.org
Description:
Per-port congestion control.
- cc_table_bin: (RO) CCA tables used by PSM2 Congestion control
+ =============== ================================================
+ cc_table_bin (RO) CCA tables used by PSM2 Congestion control
table size followed by table entries. Binary
attribute.
- cc_settings_bin:(RO) Congestion settings: port control, control
+ cc_settings_bin (RO) Congestion settings: port control, control
map and an array of 16 entries for the
congestion entries - increase, timer, event log
trigger threshold and the minimum injection rate
delay. Binary attribute.
- cc_prescan: (RW) enable prescanning for faster BECN
+ cc_prescan (RW) enable prescanning for faster BECN
response. Write "on" to enable and "off" to
disable.
+ =============== ================================================
What: /sys/class/infiniband/hfi1_X/ports/N/sc2vl/[0-31]
What: /sys/class/infiniband/hfi1_X/ports/N/sl2sc/[0-31]
@@ -708,11 +742,13 @@ Date: May, 2016
KernelVersion: v4.6
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ===================================================
sc2vl/: (RO) 32 files (0 - 31) used to translate sl->vl
sl2sc/: (RO) 32 files (0 - 31) used to translate sl->sc
vl2mtu/: (RO) 16 files (0 - 15) used to determine MTU for vl
+ =============== ===================================================
What: /sys/class/infiniband/hfi1_X/sdma_N/cpu_list
@@ -723,26 +759,28 @@ Contact: linux-rdma@vger.kernel.org
Description:
sdma<N>/ contains one directory per sdma engine (0 - 15)
+ =============== ==============================================
cpu_list: (RW) List of cpus for user-process to sdma
engine assignment.
vl: (RO) Displays the virtual lane (vl) the sdma
engine maps to.
+ =============== ==============================================
This interface gives the user control on the affinity settings
for the device. As an example, to set an sdma engine irq
affinity and thread affinity of a user processes to use the
sdma engine, which is "near" in terms of NUMA configuration, or
- physical cpu location, the user will do:
+ physical cpu location, the user will do::
- echo "3" > /proc/irq/<N>/smp_affinity_list
- echo "4-7" > /sys/devices/.../sdma3/cpu_list
- cat /sys/devices/.../sdma3/vl
- 0
- echo "8" > /proc/irq/<M>/smp_affinity_list
- echo "9-12" > /sys/devices/.../sdma4/cpu_list
- cat /sys/devices/.../sdma4/vl
- 1
+ echo "3" > /proc/irq/<N>/smp_affinity_list
+ echo "4-7" > /sys/devices/.../sdma3/cpu_list
+ cat /sys/devices/.../sdma3/vl
+ 0
+ echo "8" > /proc/irq/<M>/smp_affinity_list
+ echo "9-12" > /sys/devices/.../sdma4/cpu_list
+ cat /sys/devices/.../sdma4/vl
+ 1
to make sure that when a process runs on cpus 4,5,6, or 7, and
uses vl=0, then sdma engine 3 is selected by the driver, and
@@ -764,11 +802,13 @@ Date: Jan, 2016
KernelVersion: v4.10
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ==== ========================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Show HCA type (I40IW)
board_id: (RO) I40IW board ID
+ =============== ==== ========================
sysfs interface for QLogic qedr NIC Driver
@@ -781,9 +821,11 @@ KernelVersion: v4.10
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ==== ========================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Display HCA type
+ =============== ==== ========================
sysfs interface for VMware Paravirtual RDMA driver
@@ -797,11 +839,13 @@ KernelVersion: v4.10
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ==== =====================================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Host channel adapter type
board_id: (RO) Display PVRDMA manufacturing board ID
+ =============== ==== =====================================
sysfs interface for Broadcom NetXtreme-E RoCE driver
@@ -813,6 +857,8 @@ Date: Feb, 2017
KernelVersion: v4.11
Contact: linux-rdma@vger.kernel.org
Description:
+ =============== ==== =========================
hw_rev: (RO) Hardware revision number
hca_type: (RO) Host channel adapter type
+ =============== ==== =========================
diff --git a/Documentation/ABI/stable/sysfs-class-rfkill b/Documentation/ABI/stable/sysfs-class-rfkill
index 80151a409d67..370254b6296a 100644
--- a/Documentation/ABI/stable/sysfs-class-rfkill
+++ b/Documentation/ABI/stable/sysfs-class-rfkill
@@ -54,6 +54,7 @@ Description: Current state of the transmitter.
through this interface. There will likely be another attempt to
remove it in the future.
Values: A numeric value.
+
0: RFKILL_STATE_SOFT_BLOCKED
transmitter is turned off by software
1: RFKILL_STATE_UNBLOCKED
@@ -69,6 +70,7 @@ KernelVersion v2.6.34
Contact: linux-wireless@vger.kernel.org
Description: Current hardblock state. This file is read only.
Values: A numeric value.
+
0: inactive
The transmitter is (potentially) active.
1: active
@@ -82,6 +84,7 @@ KernelVersion v2.6.34
Contact: linux-wireless@vger.kernel.org
Description: Current softblock state. This file is read and write.
Values: A numeric value.
+
0: inactive
The transmitter is (potentially) active.
1: active
diff --git a/Documentation/ABI/stable/sysfs-class-tpm b/Documentation/ABI/stable/sysfs-class-tpm
index c0e23830f56a..bbee8899a90e 100644
--- a/Documentation/ABI/stable/sysfs-class-tpm
+++ b/Documentation/ABI/stable/sysfs-class-tpm
@@ -32,11 +32,11 @@ KernelVersion: 2.6.12
Contact: tpmdd-devel@lists.sf.net
Description: The "caps" property contains TPM manufacturer and version info.
- Example output:
+ Example output::
- Manufacturer: 0x53544d20
- TCG version: 1.2
- Firmware version: 8.16
+ Manufacturer: 0x53544d20
+ TCG version: 1.2
+ Firmware version: 8.16
Manufacturer is a hex dump of the 4 byte manufacturer info
space in a TPM. TCG version shows the TCG TPM spec level that
@@ -54,9 +54,9 @@ Description: The "durations" property shows the 3 vendor-specific values
any longer than necessary before starting to poll for a
result.
- Example output:
+ Example output::
- 3015000 4508000 180995000 [original]
+ 3015000 4508000 180995000 [original]
Here the short, medium and long durations are displayed in
usecs. "[original]" indicates that the values are displayed
@@ -92,14 +92,14 @@ Description: The "pcrs" property will dump the current value of all Platform
values may be constantly changing, the output is only valid
for a snapshot in time.
- Example output:
+ Example output::
- PCR-00: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-01: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-02: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-03: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-04: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- ...
+ PCR-00: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-01: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-02: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-03: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-04: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ ...
The number of PCRs and hex bytes needed to represent a PCR
value will vary depending on TPM chip version. For TPM 1.1 and
@@ -119,44 +119,44 @@ Description: The "pubek" property will return the TPM's public endorsement
ated at TPM manufacture time and exists for the life of the
chip.
- Example output:
-
- Algorithm: 00 00 00 01
- Encscheme: 00 03
- Sigscheme: 00 01
- Parameters: 00 00 08 00 00 00 00 02 00 00 00 00
- Modulus length: 256
- Modulus:
- B4 76 41 82 C9 20 2C 10 18 40 BC 8B E5 44 4C 6C
- 3A B2 92 0C A4 9B 2A 83 EB 5C 12 85 04 48 A0 B6
- 1E E4 81 84 CE B2 F2 45 1C F0 85 99 61 02 4D EB
- 86 C4 F7 F3 29 60 52 93 6B B2 E5 AB 8B A9 09 E3
- D7 0E 7D CA 41 BF 43 07 65 86 3C 8C 13 7A D0 8B
- 82 5E 96 0B F8 1F 5F 34 06 DA A2 52 C1 A9 D5 26
- 0F F4 04 4B D9 3F 2D F2 AC 2F 74 64 1F 8B CD 3E
- 1E 30 38 6C 70 63 69 AB E2 50 DF 49 05 2E E1 8D
- 6F 78 44 DA 57 43 69 EE 76 6C 38 8A E9 8E A3 F0
- A7 1F 3C A8 D0 12 15 3E CA 0E BD FA 24 CD 33 C6
- 47 AE A4 18 83 8E 22 39 75 93 86 E6 FD 66 48 B6
- 10 AD 94 14 65 F9 6A 17 78 BD 16 53 84 30 BF 70
- E0 DC 65 FD 3C C6 B0 1E BF B9 C1 B5 6C EF B1 3A
- F8 28 05 83 62 26 11 DC B4 6B 5A 97 FF 32 26 B6
- F7 02 71 CF 15 AE 16 DD D1 C1 8E A8 CF 9B 50 7B
- C3 91 FF 44 1E CF 7C 39 FE 17 77 21 20 BD CE 9B
-
- Possible values:
-
- Algorithm: TPM_ALG_RSA (1)
- Encscheme: TPM_ES_RSAESPKCSv15 (2)
+ Example output::
+
+ Algorithm: 00 00 00 01
+ Encscheme: 00 03
+ Sigscheme: 00 01
+ Parameters: 00 00 08 00 00 00 00 02 00 00 00 00
+ Modulus length: 256
+ Modulus:
+ B4 76 41 82 C9 20 2C 10 18 40 BC 8B E5 44 4C 6C
+ 3A B2 92 0C A4 9B 2A 83 EB 5C 12 85 04 48 A0 B6
+ 1E E4 81 84 CE B2 F2 45 1C F0 85 99 61 02 4D EB
+ 86 C4 F7 F3 29 60 52 93 6B B2 E5 AB 8B A9 09 E3
+ D7 0E 7D CA 41 BF 43 07 65 86 3C 8C 13 7A D0 8B
+ 82 5E 96 0B F8 1F 5F 34 06 DA A2 52 C1 A9 D5 26
+ 0F F4 04 4B D9 3F 2D F2 AC 2F 74 64 1F 8B CD 3E
+ 1E 30 38 6C 70 63 69 AB E2 50 DF 49 05 2E E1 8D
+ 6F 78 44 DA 57 43 69 EE 76 6C 38 8A E9 8E A3 F0
+ A7 1F 3C A8 D0 12 15 3E CA 0E BD FA 24 CD 33 C6
+ 47 AE A4 18 83 8E 22 39 75 93 86 E6 FD 66 48 B6
+ 10 AD 94 14 65 F9 6A 17 78 BD 16 53 84 30 BF 70
+ E0 DC 65 FD 3C C6 B0 1E BF B9 C1 B5 6C EF B1 3A
+ F8 28 05 83 62 26 11 DC B4 6B 5A 97 FF 32 26 B6
+ F7 02 71 CF 15 AE 16 DD D1 C1 8E A8 CF 9B 50 7B
+ C3 91 FF 44 1E CF 7C 39 FE 17 77 21 20 BD CE 9B
+
+ Possible values::
+
+ Algorithm: TPM_ALG_RSA (1)
+ Encscheme: TPM_ES_RSAESPKCSv15 (2)
TPM_ES_RSAESOAEP_SHA1_MGF1 (3)
- Sigscheme: TPM_SS_NONE (1)
- Parameters, a byte string of 3 u32 values:
+ Sigscheme: TPM_SS_NONE (1)
+ Parameters, a byte string of 3 u32 values:
Key Length (bits): 00 00 08 00 (2048)
Num primes: 00 00 00 02 (2)
Exponent Size: 00 00 00 00 (0 means the
default exp)
- Modulus Length: 256 (bytes)
- Modulus: The 256 byte Endorsement Key modulus
+ Modulus Length: 256 (bytes)
+ Modulus: The 256 byte Endorsement Key modulus
What: /sys/class/tpm/tpmX/device/temp_deactivated
Date: April 2006
@@ -176,9 +176,9 @@ Description: The "timeouts" property shows the 4 vendor-specific values
timeouts is defined by the TPM interface spec that the chip
conforms to.
- Example output:
+ Example output::
- 750000 750000 750000 750000 [original]
+ 750000 750000 750000 750000 [original]
The four timeout values are shown in usecs, with a trailing
"[original]" or "[adjusted]" depending on whether the values
diff --git a/Documentation/ABI/stable/sysfs-devices b/Documentation/ABI/stable/sysfs-devices
index 4404bd9b96c1..42bf1eab5677 100644
--- a/Documentation/ABI/stable/sysfs-devices
+++ b/Documentation/ABI/stable/sysfs-devices
@@ -1,5 +1,6 @@
-# Note: This documents additional properties of any device beyond what
-# is documented in Documentation/admin-guide/sysfs-rules.rst
+Note:
+ This documents additional properties of any device beyond what
+ is documented in Documentation/admin-guide/sysfs-rules.rst
What: /sys/devices/*/of_node
Date: February 2015
diff --git a/Documentation/ABI/stable/sysfs-driver-ib_srp b/Documentation/ABI/stable/sysfs-driver-ib_srp
index 7049a2b50359..2d706b4900ce 100644
--- a/Documentation/ABI/stable/sysfs-driver-ib_srp
+++ b/Documentation/ABI/stable/sysfs-driver-ib_srp
@@ -6,6 +6,7 @@ Description: Interface for making ib_srp connect to a new target.
One can request ib_srp to connect to a new target by writing
a comma-separated list of login parameters to this sysfs
attribute. The supported parameters are:
+
* id_ext, a 16-digit hexadecimal number specifying the eight
byte identifier extension in the 16-byte SRP target port
identifier. The target port identifier is sent by ib_srp
diff --git a/Documentation/ABI/stable/sysfs-driver-mlxreg-io b/Documentation/ABI/stable/sysfs-driver-mlxreg-io
index 156319fc5b80..3544968f43cc 100644
--- a/Documentation/ABI/stable/sysfs-driver-mlxreg-io
+++ b/Documentation/ABI/stable/sysfs-driver-mlxreg-io
@@ -1,5 +1,4 @@
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- asic_health
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/asic_health
Date: June 2018
KernelVersion: 4.19
@@ -9,9 +8,8 @@ Description: This file shows ASIC health status. The possible values are:
The files are read only.
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- cpld1_version
- cpld2_version
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/cpld1_version
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/cpld2_version
Date: June 2018
KernelVersion: 4.19
Contact: Vadim Pasternak <vadimpmellanox.com>
@@ -20,8 +18,7 @@ Description: These files show with which CPLD versions have been burned
The files are read only.
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- fan_dir
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/fan_dir
Date: December 2018
KernelVersion: 5.0
@@ -32,8 +29,7 @@ Description: This file shows the system fans direction:
The files are read only.
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- jtag_enable
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/jtag_enable
Date: November 2018
KernelVersion: 5.0
@@ -43,8 +39,7 @@ Description: These files show with which CPLD versions have been burned
The files are read only.
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- jtag_enable
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/jtag_enable
Date: November 2018
KernelVersion: 5.0
@@ -87,16 +82,15 @@ Description: These files allow asserting system power cycling, switching
The files are write only.
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- reset_aux_pwr_or_ref
- reset_asic_thermal
- reset_hotswap_or_halt
- reset_hotswap_or_wd
- reset_fw_reset
- reset_long_pb
- reset_main_pwr_fail
- reset_short_pb
- reset_sw_reset
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_aux_pwr_or_ref
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_asic_thermal
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_hotswap_or_halt
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_hotswap_or_wd
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_fw_reset
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_long_pb
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_main_pwr_fail
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_short_pb
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_sw_reset
Date: June 2018
KernelVersion: 4.19
Contact: Vadim Pasternak <vadimpmellanox.com>
@@ -110,11 +104,10 @@ Description: These files show the system reset cause, as following: power
The files are read only.
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- reset_comex_pwr_fail
- reset_from_comex
- reset_system
- reset_voltmon_upgrade_fail
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_comex_pwr_fail
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_from_comex
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_system
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_voltmon_upgrade_fail
Date: November 2018
KernelVersion: 5.0
diff --git a/Documentation/ABI/stable/sysfs-firmware-efi-vars b/Documentation/ABI/stable/sysfs-firmware-efi-vars
index 5def20b9019e..46ccd233e359 100644
--- a/Documentation/ABI/stable/sysfs-firmware-efi-vars
+++ b/Documentation/ABI/stable/sysfs-firmware-efi-vars
@@ -17,6 +17,7 @@ Description:
directory has a name of the form "<key>-<vendor guid>"
and contains the following files:
+ =============== ========================================
attributes: A read-only text file enumerating the
EFI variable flags. Potential values
include:
@@ -59,12 +60,14 @@ Description:
size: As ASCII representation of the size of
the variable's value.
+ =============== ========================================
In addition, two other magic binary files are provided
in the top-level directory and are used for adding and
removing variables:
+ =============== ========================================
new_var: Takes a "struct efi_variable" and
instructs the EFI firmware to create a
new variable.
@@ -73,3 +76,4 @@ Description:
instructs the EFI firmware to remove any
variable that has a matching vendor GUID
and variable key name.
+ =============== ========================================
diff --git a/Documentation/ABI/stable/sysfs-firmware-opal-dump b/Documentation/ABI/stable/sysfs-firmware-opal-dump
index 32fe7f5c4880..1f74f45327ba 100644
--- a/Documentation/ABI/stable/sysfs-firmware-opal-dump
+++ b/Documentation/ABI/stable/sysfs-firmware-opal-dump
@@ -7,6 +7,7 @@ Description:
This is only for the powerpc/powernv platform.
+ =============== ===============================================
initiate_dump: When '1' is written to it,
we will initiate a dump.
Read this file for supported commands.
@@ -19,8 +20,11 @@ Description:
and ID of the dump, use the id and type files.
Do not rely on any particular size of dump
type or dump id.
+ =============== ===============================================
Each dump has the following files:
+
+ =============== ===============================================
id: An ASCII representation of the dump ID
in hex (e.g. '0x01')
type: An ASCII representation of the type of
@@ -39,3 +43,4 @@ Description:
inaccessible.
Reading this file will get a list of
supported actions.
+ =============== ===============================================
diff --git a/Documentation/ABI/stable/sysfs-firmware-opal-elog b/Documentation/ABI/stable/sysfs-firmware-opal-elog
index 2536434d49d0..7c8a61a2d005 100644
--- a/Documentation/ABI/stable/sysfs-firmware-opal-elog
+++ b/Documentation/ABI/stable/sysfs-firmware-opal-elog
@@ -38,6 +38,7 @@ Description:
For each log entry (directory), there are the following
files:
+ ============== ================================================
id: An ASCII representation of the ID of the
error log, in hex - e.g. "0x01".
@@ -58,3 +59,4 @@ Description:
entry will be removed from sysfs.
Reading this file will list the supported
operations (currently just acknowledge).
+ ============== ================================================
diff --git a/Documentation/ABI/stable/sysfs-hypervisor-xen b/Documentation/ABI/stable/sysfs-hypervisor-xen
index 3cf5cdfcd9a8..01bd46be261e 100644
--- a/Documentation/ABI/stable/sysfs-hypervisor-xen
+++ b/Documentation/ABI/stable/sysfs-hypervisor-xen
@@ -33,6 +33,7 @@ Description: If running under Xen:
Space separated list of supported guest system types. Each type
is in the format: <class>-<major>.<minor>-<arch>
With:
+ ======== ============================================
<class>: "xen" -- x86: paravirtualized, arm: standard
"hvm" -- x86 only: fully virtualized
<major>: major guest interface version
@@ -43,6 +44,7 @@ Description: If running under Xen:
"x86_64": 64 bit x86 guest
"armv7l": 32 bit arm guest
"aarch64": 64 bit arm guest
+ ======== ============================================
What: /sys/hypervisor/properties/changeset
Date: March 2009
diff --git a/Documentation/ABI/stable/vdso b/Documentation/ABI/stable/vdso
index 55406ec8a35a..73ed1240a5c0 100644
--- a/Documentation/ABI/stable/vdso
+++ b/Documentation/ABI/stable/vdso
@@ -23,6 +23,7 @@ Unless otherwise noted, the set of symbols with any given version and the
ABI of those symbols is considered stable. It may vary across architectures,
though.
-(As of this writing, this ABI documentation as been confirmed for x86_64.
+Note:
+ As of this writing, this ABI documentation as been confirmed for x86_64.
The maintainers of the other vDSO-using architectures should confirm
- that it is correct for their architecture.)
+ that it is correct for their architecture.
Thanks,
Mauro
^ permalink raw reply related [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-19 16:37 ` Mauro Carvalho Chehab
@ 2019-06-21 14:27 ` Mauro Carvalho Chehab
2019-06-27 9:48 ` Jani Nikula
0 siblings, 1 reply; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-21 14:27 UTC (permalink / raw)
To: Jani Nikula
Cc: Greg Kroah-Hartman, Linux Doc Mailing List, linux-kernel,
Jonathan Corbet
Em Wed, 19 Jun 2019 13:37:39 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:
> Em Tue, 18 Jun 2019 11:47:32 +0300
> Jani Nikula <jani.nikula@linux.intel.com> escreveu:
>
> > On Mon, 17 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > > Yeah, I guess it should be possible to do that. How a python script
> > > can identify if it was called by Sphinx, or if it was called directly?
> >
> > if __name__ == '__main__':
> > # run on the command-line, not imported
>
> Ok, when I have some spare time, I may try to convert one script
> to python and see how it behaves.
Did a quick test here...
Probably I'm doing something wrong (as I'm a rookie with Python), but,
in order to be able to use the same script as command line and as an Sphinx
extension, everything that it is currently there should be "escaped"
by an:
if __name__ != '__main__':
As event the class definition:
class KernelCmd(Directive):
depends on:
from docutils.parsers.rst import directives, Directive
With is only required when one needs to parse ReST - e. g. only
when the script runs as a Sphinx extension.
If this is right, as we want a script that can run by command line
to parse and search inside ABI files, at the end of the day, it will
be a lot easier to maintain if the parser script is different from the
Sphinx extension.
If so, as the Sphinx extension script will need to call a parsing script
anyway, it doesn't matter the language of the script with will be
doing the ABI file parsing.
See the enclosing file. I didn't add anything from the ABI parsing
script yet. It was just changed in order to not generate an error
when trying to run it from command line.
Thanks,
Mauro
#!/usr/bin/env python3
# coding=utf-8
# SPDX-License-Identifier: GPL-2.0
#
u"""
kernel-abi
~~~~~~~~~~
Implementation of the ``kernel-abi`` reST-directive.
:copyright: Copyright (C) 2016 Markus Heiser
:copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
:maintained-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
:license: GPL Version 2, June 1991 see Linux/COPYING for details.
The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
scripts/get_abi.pl script to parse the Kernel ABI files.
Overview of directive's argument and options.
.. code-block:: rst
.. kernel-abi:: <ABI directory location>
:debug:
The argument ``<ABI directory location>`` is required. It contains the
location of the ABI files to be parsed.
``debug``
Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
what reST is generated.
"""
import codecs
import os
import subprocess
import sys
from os import path
if __name__ != '__main__':
from docutils import nodes, statemachine
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from docutils.utils.error_reporting import ErrorString
#
# AutodocReporter is only good up to Sphinx 1.7
#
import sphinx
Use_SSI = sphinx.__version__[:3] >= '1.7'
if Use_SSI:
from sphinx.util.docutils import switch_source_input
else:
from sphinx.ext.autodoc import AutodocReporter
__version__ = '1.0'
if __name__ != '__main__':
def setup(app):
app.add_directive("kernel-abi", KernelCmd)
return dict(
version = __version__
, parallel_read_safe = True
, parallel_write_safe = True
)
class KernelCmd(Directive):
u"""KernelABI (``kernel-abi``) directive"""
required_arguments = 1
optional_arguments = 2
has_content = False
final_argument_whitespace = True
option_spec = {
"debug" : directives.flag,
"rst" : directives.unchanged
}
def warn(self, message, **replace):
replace["fname"] = self.state.document.current_source
replace["line_no"] = replace.get("line_no", self.lineno)
message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
self.state.document.settings.env.app.warn(message, prefix="")
def run(self):
doc = self.state.document
if not doc.settings.file_insertion_enabled:
raise self.warning("docutils: file insertion disabled")
env = doc.settings.env
cwd = path.dirname(doc.current_source)
fname = self.arguments[0]
cmd = "get_abi.pl rest --dir "
cmd += fname
if 'rst' in self.options:
cmd += " --rst-source"
srctree = path.abspath(os.environ["srctree"])
# extend PATH with $(srctree)/scripts
path_env = os.pathsep.join([
srctree + os.sep + "scripts",
os.environ["PATH"]
])
shell_env = os.environ.copy()
shell_env["PATH"] = path_env
shell_env["srctree"] = srctree
lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
nodeList = self.nestedParse(lines, self.arguments[0])
return nodeList
def runCmd(self, cmd, **kwargs):
u"""Run command ``cmd`` and return it's stdout as unicode."""
try:
proc = subprocess.Popen(
cmd
, stdout = subprocess.PIPE
, stderr = subprocess.PIPE
, **kwargs
)
out, err = proc.communicate()
out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
if proc.returncode != 0:
raise self.severe(
u"command '%s' failed with return code %d"
% (cmd, proc.returncode)
)
except OSError as exc:
raise self.severe(u"problems with '%s' directive: %s."
% (self.name, ErrorString(exc)))
return out
def nestedParse(self, lines, fname):
content = ViewList()
node = nodes.section()
if "debug" in self.options:
code_block = "\n\n.. code-block:: rst\n :linenos:\n"
for l in lines.split("\n"):
code_block += "\n " + l
lines = code_block + "\n\n"
for c, l in enumerate(lines.split("\n")):
content.append(l, fname, c)
buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
if Use_SSI:
with switch_source_input(self.state, content):
self.state.nested_parse(content, 0, node, match_titles=1)
else:
self.state.memo.title_styles = []
self.state.memo.section_level = 0
self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
try:
self.state.nested_parse(content, 0, node, match_titles=1)
finally:
self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
return node.children
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-21 14:27 ` Mauro Carvalho Chehab
@ 2019-06-27 9:48 ` Jani Nikula
2019-06-27 10:52 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 63+ messages in thread
From: Jani Nikula @ 2019-06-27 9:48 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Greg Kroah-Hartman, Linux Doc Mailing List, linux-kernel,
Jonathan Corbet
On Fri, 21 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> Em Wed, 19 Jun 2019 13:37:39 -0300
> Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:
>
>> Em Tue, 18 Jun 2019 11:47:32 +0300
>> Jani Nikula <jani.nikula@linux.intel.com> escreveu:
>>
>> > On Mon, 17 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
>> > > Yeah, I guess it should be possible to do that. How a python script
>> > > can identify if it was called by Sphinx, or if it was called directly?
>> >
>> > if __name__ == '__main__':
>> > # run on the command-line, not imported
>>
>> Ok, when I have some spare time, I may try to convert one script
>> to python and see how it behaves.
>
> Did a quick test here...
>
> Probably I'm doing something wrong (as I'm a rookie with Python), but,
> in order to be able to use the same script as command line and as an Sphinx
> extension, everything that it is currently there should be "escaped"
> by an:
>
> if __name__ != '__main__':
>
> As event the class definition:
>
> class KernelCmd(Directive):
>
> depends on:
>
> from docutils.parsers.rst import directives, Directive
>
> With is only required when one needs to parse ReST - e. g. only
> when the script runs as a Sphinx extension.
>
> If this is right, as we want a script that can run by command line
> to parse and search inside ABI files, at the end of the day, it will
> be a lot easier to maintain if the parser script is different from the
> Sphinx extension.
Split it into two files, one has the nuts and bolts of parsing and has
the "if __name__ == '__main__':" bit to run on the command line, and the
other interfaces with Sphinx and imports the parser.
> If so, as the Sphinx extension script will need to call a parsing script
> anyway, it doesn't matter the language of the script with will be
> doing the ABI file parsing.
Calling the parser using an API will be easier to use, maintain and
extend than using pipes, with all the interleaved sideband information
to adjust line numbers and whatnot.
BR,
Jani.
>
> See the enclosing file. I didn't add anything from the ABI parsing
> script yet. It was just changed in order to not generate an error
> when trying to run it from command line.
>
>
> Thanks,
> Mauro
> #!/usr/bin/env python3
> # coding=utf-8
> # SPDX-License-Identifier: GPL-2.0
> #
> u"""
> kernel-abi
> ~~~~~~~~~~
>
> Implementation of the ``kernel-abi`` reST-directive.
>
> :copyright: Copyright (C) 2016 Markus Heiser
> :copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
> :maintained-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> :license: GPL Version 2, June 1991 see Linux/COPYING for details.
>
> The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
> scripts/get_abi.pl script to parse the Kernel ABI files.
>
> Overview of directive's argument and options.
>
> .. code-block:: rst
>
> .. kernel-abi:: <ABI directory location>
> :debug:
>
> The argument ``<ABI directory location>`` is required. It contains the
> location of the ABI files to be parsed.
>
> ``debug``
> Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
> what reST is generated.
>
> """
>
> import codecs
> import os
> import subprocess
> import sys
>
> from os import path
>
> if __name__ != '__main__':
> from docutils import nodes, statemachine
> from docutils.statemachine import ViewList
> from docutils.parsers.rst import directives, Directive
> from docutils.utils.error_reporting import ErrorString
>
> #
> # AutodocReporter is only good up to Sphinx 1.7
> #
> import sphinx
>
> Use_SSI = sphinx.__version__[:3] >= '1.7'
> if Use_SSI:
> from sphinx.util.docutils import switch_source_input
> else:
> from sphinx.ext.autodoc import AutodocReporter
>
> __version__ = '1.0'
>
> if __name__ != '__main__':
> def setup(app):
>
> app.add_directive("kernel-abi", KernelCmd)
> return dict(
> version = __version__
> , parallel_read_safe = True
> , parallel_write_safe = True
> )
>
> class KernelCmd(Directive):
>
> u"""KernelABI (``kernel-abi``) directive"""
>
> required_arguments = 1
> optional_arguments = 2
> has_content = False
> final_argument_whitespace = True
>
> option_spec = {
> "debug" : directives.flag,
> "rst" : directives.unchanged
> }
>
> def warn(self, message, **replace):
> replace["fname"] = self.state.document.current_source
> replace["line_no"] = replace.get("line_no", self.lineno)
> message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
> self.state.document.settings.env.app.warn(message, prefix="")
>
> def run(self):
>
> doc = self.state.document
> if not doc.settings.file_insertion_enabled:
> raise self.warning("docutils: file insertion disabled")
>
> env = doc.settings.env
> cwd = path.dirname(doc.current_source)
> fname = self.arguments[0]
>
> cmd = "get_abi.pl rest --dir "
> cmd += fname
>
> if 'rst' in self.options:
> cmd += " --rst-source"
>
> srctree = path.abspath(os.environ["srctree"])
>
> # extend PATH with $(srctree)/scripts
> path_env = os.pathsep.join([
> srctree + os.sep + "scripts",
> os.environ["PATH"]
> ])
> shell_env = os.environ.copy()
> shell_env["PATH"] = path_env
> shell_env["srctree"] = srctree
>
> lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
> nodeList = self.nestedParse(lines, self.arguments[0])
> return nodeList
>
> def runCmd(self, cmd, **kwargs):
> u"""Run command ``cmd`` and return it's stdout as unicode."""
>
> try:
> proc = subprocess.Popen(
> cmd
> , stdout = subprocess.PIPE
> , stderr = subprocess.PIPE
> , **kwargs
> )
> out, err = proc.communicate()
>
> out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
>
> if proc.returncode != 0:
> raise self.severe(
> u"command '%s' failed with return code %d"
> % (cmd, proc.returncode)
> )
> except OSError as exc:
> raise self.severe(u"problems with '%s' directive: %s."
> % (self.name, ErrorString(exc)))
> return out
>
> def nestedParse(self, lines, fname):
> content = ViewList()
> node = nodes.section()
>
> if "debug" in self.options:
> code_block = "\n\n.. code-block:: rst\n :linenos:\n"
> for l in lines.split("\n"):
> code_block += "\n " + l
> lines = code_block + "\n\n"
>
> for c, l in enumerate(lines.split("\n")):
> content.append(l, fname, c)
>
> buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
>
> if Use_SSI:
> with switch_source_input(self.state, content):
> self.state.nested_parse(content, 0, node, match_titles=1)
> else:
> self.state.memo.title_styles = []
> self.state.memo.section_level = 0
> self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
> try:
> self.state.nested_parse(content, 0, node, match_titles=1)
> finally:
> self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
>
> return node.children
--
Jani Nikula, Intel Open Source Graphics Center
^ permalink raw reply [flat|nested] 63+ messages in thread
* Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book
2019-06-27 9:48 ` Jani Nikula
@ 2019-06-27 10:52 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 63+ messages in thread
From: Mauro Carvalho Chehab @ 2019-06-27 10:52 UTC (permalink / raw)
To: Jani Nikula
Cc: Greg Kroah-Hartman, Linux Doc Mailing List, linux-kernel,
Jonathan Corbet
Em Thu, 27 Jun 2019 12:48:12 +0300
Jani Nikula <jani.nikula@linux.intel.com> escreveu:
> On Fri, 21 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> > Em Wed, 19 Jun 2019 13:37:39 -0300
> > Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:
> >
> >> Em Tue, 18 Jun 2019 11:47:32 +0300
> >> Jani Nikula <jani.nikula@linux.intel.com> escreveu:
> >>
> >> > On Mon, 17 Jun 2019, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote:
> >> > > Yeah, I guess it should be possible to do that. How a python script
> >> > > can identify if it was called by Sphinx, or if it was called directly?
> >> >
> >> > if __name__ == '__main__':
> >> > # run on the command-line, not imported
> >>
> >> Ok, when I have some spare time, I may try to convert one script
> >> to python and see how it behaves.
> >
> > Did a quick test here...
> >
> > Probably I'm doing something wrong (as I'm a rookie with Python), but,
> > in order to be able to use the same script as command line and as an Sphinx
> > extension, everything that it is currently there should be "escaped"
> > by an:
> >
> > if __name__ != '__main__':
> >
> > As event the class definition:
> >
> > class KernelCmd(Directive):
> >
> > depends on:
> >
> > from docutils.parsers.rst import directives, Directive
> >
> > With is only required when one needs to parse ReST - e. g. only
> > when the script runs as a Sphinx extension.
> >
> > If this is right, as we want a script that can run by command line
> > to parse and search inside ABI files, at the end of the day, it will
> > be a lot easier to maintain if the parser script is different from the
> > Sphinx extension.
>
> Split it into two files, one has the nuts and bolts of parsing and has
> the "if __name__ == '__main__':" bit to run on the command line, and the
> other interfaces with Sphinx and imports the parser.
It seems we have an agreement here: the best is indeed to have two
files, one with the Documentation/ABI parser, and another one with the
Sphinx extension...
>
> > If so, as the Sphinx extension script will need to call a parsing script
> > anyway, it doesn't matter the language of the script with will be
> > doing the ABI file parsing.
>
> Calling the parser using an API will be easier to use, maintain and
> extend than using pipes, with all the interleaved sideband information
> to adjust line numbers and whatnot.
... and here is where we have two different views.
From debug PoV, the Documentation/ABI parser script should be able to
print the results by a command line call. This is also a feature
that it is useful for the users: to be able to seek for a symbol
and output its ABI description. So, the "stdout" output will be
there anyway.
The only extra data for the extension side is the file name where
the information came and the line number.
From maintainership PoV, adding the sideband API for file+line is
one line at the parser script (a print) and two lines at the Sphinx
extension (a regex expression and a match line). That's simple to
maintain.
It is also simple to verify both sides independently, as what
you see when running the parser script is what you get at the
extension.
If we add a new ABI between the parser script and the extension
script, this would require to also maintain the ABI, and would
make harder to identify problems on ABI problems.
-
Another advantage of having those independent is that the
language of the parsing script can be different. Not being
python is a big advantage for me, as perl is a lot more
intuitive and easier to write parser scripts for my eyes.
I can write a perl parsing script in a matter of minutes.
It takes me a lot more time to do the same with python, and then
ensure that it will work with two similar but different languages
(python2 and python3) [1].
[1] btw, with that regards, I still don't know how to teach a
python script that it should "prefer" to run with python3 but would
fall back to python2. Adding this shebang:
# /usr/bin/env python
just do the opposite - at least on Fedora
Thanks,
Mauro
^ permalink raw reply [flat|nested] 63+ messages in thread
end of thread, other threads:[~2019-06-27 10:52 UTC | newest]
Thread overview: 63+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-14 2:04 [PATCH 00/14] Add support to generate ABI documentation at admin-guide Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 01/14] ABI: fix some syntax issues at the ABI database Mauro Carvalho Chehab
2019-06-14 7:20 ` Andrew Donnellan
2019-06-14 16:16 ` Greg Kroah-Hartman
2019-06-14 8:30 ` Rafael J. Wysocki
2019-06-14 2:04 ` [PATCH 02/14] ABI: sysfs-driver-hid: the "What" field doesn't parse fine Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 03/14] ABI: sysfs-class-uwb_rc: remove a duplicated incomplete entry Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 04/14] ABI: better identificate tables Mauro Carvalho Chehab
2019-06-19 12:51 ` Johan Hovold
2019-06-19 13:56 ` Mauro Carvalho Chehab
2019-06-19 15:02 ` Greg Kroah-Hartman
2019-06-19 16:14 ` Mauro Carvalho Chehab
2019-06-20 14:27 ` Mauro Carvalho Chehab
2019-06-21 7:21 ` Greg Kroah-Hartman
2019-06-21 9:49 ` Mauro Carvalho Chehab
2019-06-20 12:01 ` Johan Hovold
2019-06-20 12:54 ` Greg Kroah-Hartman
2019-06-20 14:20 ` Mauro Carvalho Chehab
2019-06-20 16:29 ` Greg Kroah-Hartman
2019-06-20 17:16 ` Mauro Carvalho Chehab
2019-06-20 17:55 ` Greg Kroah-Hartman
2019-06-14 2:04 ` [PATCH 05/14] scripts: add an script to parse the ABI files Mauro Carvalho Chehab
2019-06-14 13:31 ` Jani Nikula
2019-06-14 13:39 ` Greg Kroah-Hartman
2019-06-14 13:58 ` Mauro Carvalho Chehab
2019-06-14 14:00 ` Jani Nikula
2019-06-14 14:14 ` Jonathan Corbet
2019-06-14 16:24 ` Greg Kroah-Hartman
2019-06-14 2:04 ` [PATCH 06/14] scripts/get_abi.pl: parse files with text at beginning Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 07/14] scripts/get_abi.pl: avoid use literal blocks when not needed Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 08/14] scripts/get_abi.pl: split label naming from xref logic Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 09/14] scripts/get_abi.pl: add support for searching for ABI symbols Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 10/14] scripts/get_abi.pl: represent what in tables Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 11/14] scripts/get_abi.pl: fix parse issues with some files Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book Mauro Carvalho Chehab
2019-06-14 13:42 ` Jani Nikula
2019-06-14 14:06 ` Greg Kroah-Hartman
2019-06-14 15:27 ` Mauro Carvalho Chehab
2019-06-17 12:36 ` Jani Nikula
2019-06-17 12:54 ` Greg Kroah-Hartman
2019-06-17 13:48 ` Jonathan Corbet
2019-06-17 13:50 ` Jani Nikula
2019-06-17 13:51 ` Mauro Carvalho Chehab
2019-06-18 8:47 ` Jani Nikula
2019-06-19 16:37 ` Mauro Carvalho Chehab
2019-06-21 14:27 ` Mauro Carvalho Chehab
2019-06-27 9:48 ` Jani Nikula
2019-06-27 10:52 ` Mauro Carvalho Chehab
2019-06-14 14:10 ` Markus Heiser
2019-06-14 14:15 ` Jonathan Corbet
2019-06-16 16:04 ` Markus Heiser
2019-06-17 9:11 ` Mauro Carvalho Chehab
2019-06-17 16:31 ` Markus Heiser
2019-06-14 2:04 ` [PATCH 13/14] sphinx/kernel_abi.py: make it compatible with Sphinx 1.7+ Mauro Carvalho Chehab
2019-06-14 2:04 ` [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support Mauro Carvalho Chehab
2019-06-14 16:18 ` Greg Kroah-Hartman
2019-06-14 16:25 ` Mauro Carvalho Chehab
2019-06-15 6:16 ` Greg Kroah-Hartman
2019-06-16 15:43 ` Markus Heiser
2019-06-17 9:16 ` Mauro Carvalho Chehab
2019-06-17 13:56 ` Jonathan Corbet
2019-06-17 15:55 ` Mauro Carvalho Chehab
2019-06-14 16:20 ` [PATCH 00/14] Add support to generate ABI documentation at admin-guide Greg Kroah-Hartman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).