[libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Erik Schilling]

When releasing the 0.2.0 version of the libgpiod crate, I did not
realize that there were C lib features that were not released yet.
Helpfully, vhost-device's CI went up in flames and revealed this [1].

This suggests a way to handle that and sketches how further updates can
be handled.

I acknowledge that this may be very strange to C developers...
Traditionally, one would just use whatever your distro provided and the
distro would make sure that dependencies update in lock-step.

However, in Rust the default way to consume libraries is to pull them
from crates.io. This is a balancing act for -sys crates which link to
distro-provided libraries on the system. Since crates.io does not wait
for distros to update their libraries, crates will need to support a
wider range of system libraries.

This sets up / sketches the infrastructure for that.

Only the first commit is intended to be merged. The second one just
sketches how a release will look like once it happens.

[1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479

To: Bartosz Golaszewski <brgl@bgdev.pl>
To: Linux-GPIO <linux-gpio@vger.kernel.org>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Cc: Kent Gibson <warthog618@gmail.com>

Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
---
Erik Schilling (2):
      bindings: rust: feature gate unreleased features
      DONOTMERGE: bindings: rust: simulate v2.1 release

 bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
 bindings/rust/libgpiod/Cargo.toml            |  4 ++++
 bindings/rust/libgpiod/Makefile.am           |  2 +-
 bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
 bindings/rust/libgpiod/src/line_request.rs   |  2 ++
 bindings/rust/libgpiod/tests/line_request.rs |  1 +
 6 files changed, 29 insertions(+), 3 deletions(-)
---
base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
change-id: 20231006-b4-bindings-old-version-fix-789973703b77

Best regards,
-- 
Erik Schilling <erik.schilling@linaro.org>

[libgpiod][PATCH 1/2] bindings: rust: feature gate unreleased features

[Erik Schilling]

`gpiod_line_request_get_chip_name()` is not released yet. Still, libgpiod-sys
will just happily generate bindings for it if it sees the definition in the
header file.

This guards the unreleased features behind an optional feature `vnext`.

To sketch the process of what happens once these features get into an
assumed "2.1" release:

libgpiod-sys will get updated with a `v2_1` feature. That feature would
then raise the minimum version that is attempted to query from pkg-
config. An identical feature will then be introduced on the `libgpiod`
crate and `vnext` guards will be changed to `v2_1` guards. The `vnext`
feature will then be updated to require the new `v2_1` feature.

Eventually, we will probably raise the minimum supported version for the
rust bindings and drop all the version gates before that.

Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
---
 bindings/rust/libgpiod/Cargo.toml            |  3 +++
 bindings/rust/libgpiod/Makefile.am           |  2 +-
 bindings/rust/libgpiod/README.md             | 13 +++++++++++++
 bindings/rust/libgpiod/src/line_request.rs   |  2 ++
 bindings/rust/libgpiod/tests/line_request.rs |  1 +
 5 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/bindings/rust/libgpiod/Cargo.toml b/bindings/rust/libgpiod/Cargo.toml
index 3be4aa0..3fd1d74 100644
--- a/bindings/rust/libgpiod/Cargo.toml
+++ b/bindings/rust/libgpiod/Cargo.toml
@@ -18,6 +18,9 @@ exclude = [
     "Makefile.am",
 ]
 
+[features]
+vnext = []
+
 [dependencies]
 errno = "0.2.8"
 intmap = "2.0.0"
diff --git a/bindings/rust/libgpiod/Makefile.am b/bindings/rust/libgpiod/Makefile.am
index 92edbfc..619e36c 100644
--- a/bindings/rust/libgpiod/Makefile.am
+++ b/bindings/rust/libgpiod/Makefile.am
@@ -8,7 +8,7 @@ command = SYSTEM_DEPS_LIBGPIOD_NO_PKG_CONFIG=1 \
 		SYSTEM_DEPS_LIBGPIOD_SEARCH_NATIVE="${PWD}/../../../lib/.libs/" \
 		SYSTEM_DEPS_LIBGPIOD_LIB=gpiod \
 		SYSTEM_DEPS_LIBGPIOD_INCLUDE="${PWD}/../../../include/"  \
-		cargo build --release --lib
+		cargo build --features=vnext --release --lib
 
 if WITH_TESTS
 command += --tests
diff --git a/bindings/rust/libgpiod/README.md b/bindings/rust/libgpiod/README.md
index 8d514e7..c86b06e 100644
--- a/bindings/rust/libgpiod/README.md
+++ b/bindings/rust/libgpiod/README.md
@@ -17,6 +17,19 @@ By default, `libgpiod-sys` builds against the libgpiod version identified via
 `pkg-config`. See the `README.md` of `libgpiod-sys` for options to override
 that.
 
+Currently at least libgpiod 2.0 is required with the default feature set.
+
+## Features
+
+The Rust bindings will usually be built against whatever libgpiod version a
+system provides. Hence, only the functionality of the oldest supported libgpiod
+C library will be exposed by default.
+
+Setting flags allows to increase the base version and export features of newer
+versions:
+
+- `vnext`: The upcoming, still unreleased version of the C lib
+
 ## License
 
 This project is licensed under either of
diff --git a/bindings/rust/libgpiod/src/line_request.rs b/bindings/rust/libgpiod/src/line_request.rs
index 64ef05d..a7fe6d0 100644
--- a/bindings/rust/libgpiod/src/line_request.rs
+++ b/bindings/rust/libgpiod/src/line_request.rs
@@ -2,6 +2,7 @@
 // SPDX-FileCopyrightText: 2022 Linaro Ltd.
 // SPDX-FileCopyrightText: 2022 Viresh Kumar <viresh.kumar@linaro.org>
 
+#[cfg(feature = "vnext")]
 use std::ffi::CStr;
 use std::os::unix::prelude::AsRawFd;
 use std::time::Duration;
@@ -31,6 +32,7 @@ impl Request {
     }
 
     /// Get the name of the chip this request was made on.
+    #[cfg(feature = "vnext")]
     pub fn chip_name(&self) -> Result<&str> {
         // SAFETY: The `gpiod_line_request` is guaranteed to be live as long
         // as `&self`
diff --git a/bindings/rust/libgpiod/tests/line_request.rs b/bindings/rust/libgpiod/tests/line_request.rs
index e0ae200..a936a1b 100644
--- a/bindings/rust/libgpiod/tests/line_request.rs
+++ b/bindings/rust/libgpiod/tests/line_request.rs
@@ -60,6 +60,7 @@ mod line_request {
         use super::*;
 
         #[test]
+        #[cfg(feature = "vnext")]
         fn chip_name() {
             const GPIO: Offset = 2;
             let mut config = TestConfig::new(NGPIO).unwrap();

-- 
2.41.0

[libgpiod][PATCH 2/2] DONOTMERGE: bindings: rust: simulate v2.1 release

[Erik Schilling]

This is just a demo of a release bump to better illustrate what was
sketched in the previous commit.

Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
---
 bindings/rust/libgpiod-sys/Cargo.toml        | 9 +++++++--
 bindings/rust/libgpiod/Cargo.toml            | 3 ++-
 bindings/rust/libgpiod/README.md             | 1 +
 bindings/rust/libgpiod/src/line_request.rs   | 4 ++--
 bindings/rust/libgpiod/tests/line_request.rs | 2 +-
 5 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/bindings/rust/libgpiod-sys/Cargo.toml b/bindings/rust/libgpiod-sys/Cargo.toml
index b4d26e9..f90615d 100644
--- a/bindings/rust/libgpiod-sys/Cargo.toml
+++ b/bindings/rust/libgpiod-sys/Cargo.toml
@@ -18,11 +18,16 @@ exclude = [
     "Makefile.am",
 ]
 
+[features]
+v2_1 = []
+
 [dependencies]
 
 [build-dependencies]
 bindgen = "0.63"
 system-deps = "2.0"
 
-[package.metadata.system-deps]
-libgpiod = "2"
+[package.metadata.system-deps.libgpiod]
+name = "libgpiod"
+version = "2"
+v2_1 = { version = "2.1" }
diff --git a/bindings/rust/libgpiod/Cargo.toml b/bindings/rust/libgpiod/Cargo.toml
index 3fd1d74..0d8745d 100644
--- a/bindings/rust/libgpiod/Cargo.toml
+++ b/bindings/rust/libgpiod/Cargo.toml
@@ -19,7 +19,8 @@ exclude = [
 ]
 
 [features]
-vnext = []
+v2_1 = ["libgpiod-sys/v2_1"]
+vnext = ["v2_1"]
 
 [dependencies]
 errno = "0.2.8"
diff --git a/bindings/rust/libgpiod/README.md b/bindings/rust/libgpiod/README.md
index c86b06e..1ef3743 100644
--- a/bindings/rust/libgpiod/README.md
+++ b/bindings/rust/libgpiod/README.md
@@ -28,6 +28,7 @@ C library will be exposed by default.
 Setting flags allows to increase the base version and export features of newer
 versions:
 
+- `v2_1`: Minimum version of `2.1.x`
 - `vnext`: The upcoming, still unreleased version of the C lib
 
 ## License
diff --git a/bindings/rust/libgpiod/src/line_request.rs b/bindings/rust/libgpiod/src/line_request.rs
index a7fe6d0..7ec6508 100644
--- a/bindings/rust/libgpiod/src/line_request.rs
+++ b/bindings/rust/libgpiod/src/line_request.rs
@@ -2,7 +2,7 @@
 // SPDX-FileCopyrightText: 2022 Linaro Ltd.
 // SPDX-FileCopyrightText: 2022 Viresh Kumar <viresh.kumar@linaro.org>
 
-#[cfg(feature = "vnext")]
+#[cfg(feature = "v2_1")]
 use std::ffi::CStr;
 use std::os::unix::prelude::AsRawFd;
 use std::time::Duration;
@@ -32,7 +32,7 @@ impl Request {
     }
 
     /// Get the name of the chip this request was made on.
-    #[cfg(feature = "vnext")]
+    #[cfg(feature = "v2_1")]
     pub fn chip_name(&self) -> Result<&str> {
         // SAFETY: The `gpiod_line_request` is guaranteed to be live as long
         // as `&self`
diff --git a/bindings/rust/libgpiod/tests/line_request.rs b/bindings/rust/libgpiod/tests/line_request.rs
index a936a1b..4e095a4 100644
--- a/bindings/rust/libgpiod/tests/line_request.rs
+++ b/bindings/rust/libgpiod/tests/line_request.rs
@@ -60,7 +60,7 @@ mod line_request {
         use super::*;
 
         #[test]
-        #[cfg(feature = "vnext")]
+        #[cfg(feature = "v2_1")]
         fn chip_name() {
             const GPIO: Offset = 2;
             let mut config = TestConfig::new(NGPIO).unwrap();

-- 
2.41.0

Re: [libgpiod][PATCH 1/2] bindings: rust: feature gate unreleased features

[Viresh Kumar]

On 06-10-23, 09:24, Erik Schilling wrote:
> `gpiod_line_request_get_chip_name()` is not released yet. Still, libgpiod-sys
> will just happily generate bindings for it if it sees the definition in the
> header file.
> 
> This guards the unreleased features behind an optional feature `vnext`.
> 
> To sketch the process of what happens once these features get into an
> assumed "2.1" release:
> 
> libgpiod-sys will get updated with a `v2_1` feature. That feature would
> then raise the minimum version that is attempted to query from pkg-
> config. An identical feature will then be introduced on the `libgpiod`
> crate and `vnext` guards will be changed to `v2_1` guards. The `vnext`
> feature will then be updated to require the new `v2_1` feature.
> 
> Eventually, we will probably raise the minimum supported version for the
> rust bindings and drop all the version gates before that.
> 
> Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
> ---
>  bindings/rust/libgpiod/Cargo.toml            |  3 +++
>  bindings/rust/libgpiod/Makefile.am           |  2 +-
>  bindings/rust/libgpiod/README.md             | 13 +++++++++++++
>  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
>  bindings/rust/libgpiod/tests/line_request.rs |  1 +
>  5 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/bindings/rust/libgpiod/Cargo.toml b/bindings/rust/libgpiod/Cargo.toml
> index 3be4aa0..3fd1d74 100644
> --- a/bindings/rust/libgpiod/Cargo.toml
> +++ b/bindings/rust/libgpiod/Cargo.toml
> @@ -18,6 +18,9 @@ exclude = [
>      "Makefile.am",
>  ]
>  
> +[features]
> +vnext = []
> +
>  [dependencies]
>  errno = "0.2.8"
>  intmap = "2.0.0"
> diff --git a/bindings/rust/libgpiod/Makefile.am b/bindings/rust/libgpiod/Makefile.am
> index 92edbfc..619e36c 100644
> --- a/bindings/rust/libgpiod/Makefile.am
> +++ b/bindings/rust/libgpiod/Makefile.am
> @@ -8,7 +8,7 @@ command = SYSTEM_DEPS_LIBGPIOD_NO_PKG_CONFIG=1 \
>  		SYSTEM_DEPS_LIBGPIOD_SEARCH_NATIVE="${PWD}/../../../lib/.libs/" \
>  		SYSTEM_DEPS_LIBGPIOD_LIB=gpiod \
>  		SYSTEM_DEPS_LIBGPIOD_INCLUDE="${PWD}/../../../include/"  \
> -		cargo build --release --lib
> +		cargo build --features=vnext --release --lib
>  
>  if WITH_TESTS
>  command += --tests
> diff --git a/bindings/rust/libgpiod/README.md b/bindings/rust/libgpiod/README.md
> index 8d514e7..c86b06e 100644
> --- a/bindings/rust/libgpiod/README.md
> +++ b/bindings/rust/libgpiod/README.md
> @@ -17,6 +17,19 @@ By default, `libgpiod-sys` builds against the libgpiod version identified via
>  `pkg-config`. See the `README.md` of `libgpiod-sys` for options to override
>  that.
>  
> +Currently at least libgpiod 2.0 is required with the default feature set.
> +
> +## Features
> +
> +The Rust bindings will usually be built against whatever libgpiod version a
> +system provides. Hence, only the functionality of the oldest supported libgpiod
> +C library will be exposed by default.
> +
> +Setting flags allows to increase the base version and export features of newer
> +versions:
> +
> +- `vnext`: The upcoming, still unreleased version of the C lib
> +
>  ## License
>  
>  This project is licensed under either of
> diff --git a/bindings/rust/libgpiod/src/line_request.rs b/bindings/rust/libgpiod/src/line_request.rs
> index 64ef05d..a7fe6d0 100644
> --- a/bindings/rust/libgpiod/src/line_request.rs
> +++ b/bindings/rust/libgpiod/src/line_request.rs
> @@ -2,6 +2,7 @@
>  // SPDX-FileCopyrightText: 2022 Linaro Ltd.
>  // SPDX-FileCopyrightText: 2022 Viresh Kumar <viresh.kumar@linaro.org>
>  
> +#[cfg(feature = "vnext")]
>  use std::ffi::CStr;
>  use std::os::unix::prelude::AsRawFd;
>  use std::time::Duration;
> @@ -31,6 +32,7 @@ impl Request {
>      }
>  
>      /// Get the name of the chip this request was made on.
> +    #[cfg(feature = "vnext")]
>      pub fn chip_name(&self) -> Result<&str> {
>          // SAFETY: The `gpiod_line_request` is guaranteed to be live as long
>          // as `&self`
> diff --git a/bindings/rust/libgpiod/tests/line_request.rs b/bindings/rust/libgpiod/tests/line_request.rs
> index e0ae200..a936a1b 100644
> --- a/bindings/rust/libgpiod/tests/line_request.rs
> +++ b/bindings/rust/libgpiod/tests/line_request.rs
> @@ -60,6 +60,7 @@ mod line_request {
>          use super::*;
>  
>          #[test]
> +        #[cfg(feature = "vnext")]
>          fn chip_name() {
>              const GPIO: Offset = 2;
>              let mut config = TestConfig::new(NGPIO).unwrap();

Acked-by: Viresh Kumar <viresh.kumar@linaro.org>

-- 
viresh

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Bartosz Golaszewski]

On Fri, Oct 6, 2023 at 9:24 AM Erik Schilling <erik.schilling@linaro.org> wrote:
>
> When releasing the 0.2.0 version of the libgpiod crate, I did not
> realize that there were C lib features that were not released yet.
> Helpfully, vhost-device's CI went up in flames and revealed this [1].
>
> This suggests a way to handle that and sketches how further updates can
> be handled.
>
> I acknowledge that this may be very strange to C developers...
> Traditionally, one would just use whatever your distro provided and the
> distro would make sure that dependencies update in lock-step.
>
> However, in Rust the default way to consume libraries is to pull them
> from crates.io. This is a balancing act for -sys crates which link to
> distro-provided libraries on the system. Since crates.io does not wait
> for distros to update their libraries, crates will need to support a
> wider range of system libraries.
>
> This sets up / sketches the infrastructure for that.
>
> Only the first commit is intended to be merged. The second one just
> sketches how a release will look like once it happens.
>
> [1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479
>
> To: Bartosz Golaszewski <brgl@bgdev.pl>
> To: Linux-GPIO <linux-gpio@vger.kernel.org>
> Cc: Viresh Kumar <viresh.kumar@linaro.org>
> Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> Cc: Kent Gibson <warthog618@gmail.com>
>
> Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
> ---
> Erik Schilling (2):
>       bindings: rust: feature gate unreleased features
>       DONOTMERGE: bindings: rust: simulate v2.1 release
>
>  bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
>  bindings/rust/libgpiod/Cargo.toml            |  4 ++++
>  bindings/rust/libgpiod/Makefile.am           |  2 +-
>  bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
>  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
>  bindings/rust/libgpiod/tests/line_request.rs |  1 +
>  6 files changed, 29 insertions(+), 3 deletions(-)
> ---
> base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
> change-id: 20231006-b4-bindings-old-version-fix-789973703b77
>
> Best regards,
> --
> Erik Schilling <erik.schilling@linaro.org>
>

I'm currently discussing a similar problem with a developer who
offered to help make Python bindings released on PyPi more
user-friendly. His suggestion for python bindings is what many PyPi
packages that provide bindings to C libraries do: just compile the
library statically and make it part of the C extension module bundled
together with python code.

Given that rust programs are statically linked, maybe this is a
solution we could use?

Bart

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Erik Schilling]

On Mon Oct 9, 2023 at 10:58 AM CEST, Bartosz Golaszewski wrote:
> On Fri, Oct 6, 2023 at 9:24 AM Erik Schilling <erik.schilling@linaro.org> wrote:
> >
> > When releasing the 0.2.0 version of the libgpiod crate, I did not
> > realize that there were C lib features that were not released yet.
> > Helpfully, vhost-device's CI went up in flames and revealed this [1].
> >
> > This suggests a way to handle that and sketches how further updates can
> > be handled.
> >
> > I acknowledge that this may be very strange to C developers...
> > Traditionally, one would just use whatever your distro provided and the
> > distro would make sure that dependencies update in lock-step.
> >
> > However, in Rust the default way to consume libraries is to pull them
> > from crates.io. This is a balancing act for -sys crates which link to
> > distro-provided libraries on the system. Since crates.io does not wait
> > for distros to update their libraries, crates will need to support a
> > wider range of system libraries.
> >
> > This sets up / sketches the infrastructure for that.
> >
> > Only the first commit is intended to be merged. The second one just
> > sketches how a release will look like once it happens.
> >
> > [1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479
> >
> > To: Bartosz Golaszewski <brgl@bgdev.pl>
> > To: Linux-GPIO <linux-gpio@vger.kernel.org>
> > Cc: Viresh Kumar <viresh.kumar@linaro.org>
> > Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> > Cc: Kent Gibson <warthog618@gmail.com>
> >
> > Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
> > ---
> > Erik Schilling (2):
> >       bindings: rust: feature gate unreleased features
> >       DONOTMERGE: bindings: rust: simulate v2.1 release
> >
> >  bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
> >  bindings/rust/libgpiod/Cargo.toml            |  4 ++++
> >  bindings/rust/libgpiod/Makefile.am           |  2 +-
> >  bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
> >  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
> >  bindings/rust/libgpiod/tests/line_request.rs |  1 +
> >  6 files changed, 29 insertions(+), 3 deletions(-)
> > ---
> > base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
> > change-id: 20231006-b4-bindings-old-version-fix-789973703b77
> >
> > Best regards,
> > --
> > Erik Schilling <erik.schilling@linaro.org>
> >
>
> I'm currently discussing a similar problem with a developer who
> offered to help make Python bindings released on PyPi more
> user-friendly. His suggestion for python bindings is what many PyPi
> packages that provide bindings to C libraries do: just compile the
> library statically and make it part of the C extension module bundled
> together with python code.
>
> Given that rust programs are statically linked, maybe this is a
> solution we could use?

It may be one option that we could provide, but I still see need for
linking against system libs.

Rust is already a bit special - at least compared to traditional C
projects - in that it pulls all the code of all dependencies and builds
the entire bunch into a static lib. This is still upsetting some people
who prefer each dependency to be its own self-contained dynamic library
that can be updated without updating the rest of it. Rust can make
that work by being a mono-culture in term of build tools. Everything on
crates.io builds through cargo in the same way, so tooling can assist
with auditing and updating crates.

While distros - to various degrees - are starting to accept that this
may be a viable approach, I imagine that they would still feel uneasy
about some Rust code statically linking a C lib. They spend a lot of
effort making sure that the C lib is built according to their packaging
guidelines and do not want to repeat that effort for each Rust binary
that ends up including libgpiod.

Hence, I think that we would still need to provide a way to build and
link against the system libs.

However, that does not necessarily means
that we need to feature gate pending features like suggested here.
We _could_ just tie the Rust bindings to the libgpiod version 1:1 and
make bindings only support one version of the C lib.

That would come with some implications for the release  though. We would
effectively need to manage bugfixes to the bindings through libgpiod's
stable branches. For the version numbers to be non-confusing, we would
probably also need to always do the releases together. So there may
be needs to do a new patch release for libgpiod even if only a fix was
needed in the bindings.

Then, we may end up in a situation where SemVer changes may be
conflicting between bindings and the core lib. With the Rust bindings
being new, my recent fix of a soundness issue required changes to types
that required a bump of the major version. The C version was entirely
unaffected. Should we have bumped its major for this too?

I outlined these kind of issues in my initial thread around publishing
to crates.io [1]. We could of course just document somewhere which
version of the Rust crate pulls in which version of the C lib.

But is it worth it? We would still need to maintain the nested build
next to building against system libs. libgpiod is of course fairly
boring to build. But we would still need to tell users to install the
few build dependencies, C tools may update and the latest crates.io
release may fail building.

I totally see that static builds may be helpful in some cases. But I
would assume that the maintenance overhead of nesting the build is less
controllable compared to what I suggest in this patch. The C lib already
follows the good practices around versioning and API compatibility. So
I would assume that maintaining the conditional compiles will be fairly
simple.

Once another major bump happens in the C lib, we can just follow the
bump by removing all existing conditionals and bumping the required
floor version for the Rust bindings.

All in all, I only see minor advantages for users while seeing a lot of
maintenance burden.

[1] https://lore.kernel.org/all/CT0CRWOTJIEO.20BGIDMLFM0E8@fedora/

- Erik

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Bartosz Golaszewski]

On Mon, Oct 9, 2023 at 1:38 PM Erik Schilling <erik.schilling@linaro.org> wrote:
>
> On Mon Oct 9, 2023 at 10:58 AM CEST, Bartosz Golaszewski wrote:
> > On Fri, Oct 6, 2023 at 9:24 AM Erik Schilling <erik.schilling@linaro.org> wrote:
> > >
> > > When releasing the 0.2.0 version of the libgpiod crate, I did not
> > > realize that there were C lib features that were not released yet.
> > > Helpfully, vhost-device's CI went up in flames and revealed this [1].
> > >
> > > This suggests a way to handle that and sketches how further updates can
> > > be handled.
> > >
> > > I acknowledge that this may be very strange to C developers...
> > > Traditionally, one would just use whatever your distro provided and the
> > > distro would make sure that dependencies update in lock-step.
> > >
> > > However, in Rust the default way to consume libraries is to pull them
> > > from crates.io. This is a balancing act for -sys crates which link to
> > > distro-provided libraries on the system. Since crates.io does not wait
> > > for distros to update their libraries, crates will need to support a
> > > wider range of system libraries.
> > >
> > > This sets up / sketches the infrastructure for that.
> > >
> > > Only the first commit is intended to be merged. The second one just
> > > sketches how a release will look like once it happens.
> > >
> > > [1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479
> > >
> > > To: Bartosz Golaszewski <brgl@bgdev.pl>
> > > To: Linux-GPIO <linux-gpio@vger.kernel.org>
> > > Cc: Viresh Kumar <viresh.kumar@linaro.org>
> > > Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> > > Cc: Kent Gibson <warthog618@gmail.com>
> > >
> > > Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
> > > ---
> > > Erik Schilling (2):
> > >       bindings: rust: feature gate unreleased features
> > >       DONOTMERGE: bindings: rust: simulate v2.1 release
> > >
> > >  bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
> > >  bindings/rust/libgpiod/Cargo.toml            |  4 ++++
> > >  bindings/rust/libgpiod/Makefile.am           |  2 +-
> > >  bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
> > >  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
> > >  bindings/rust/libgpiod/tests/line_request.rs |  1 +
> > >  6 files changed, 29 insertions(+), 3 deletions(-)
> > > ---
> > > base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
> > > change-id: 20231006-b4-bindings-old-version-fix-789973703b77
> > >
> > > Best regards,
> > > --
> > > Erik Schilling <erik.schilling@linaro.org>
> > >
> >
> > I'm currently discussing a similar problem with a developer who
> > offered to help make Python bindings released on PyPi more
> > user-friendly. His suggestion for python bindings is what many PyPi
> > packages that provide bindings to C libraries do: just compile the
> > library statically and make it part of the C extension module bundled
> > together with python code.
> >
> > Given that rust programs are statically linked, maybe this is a
> > solution we could use?
>
> It may be one option that we could provide, but I still see need for
> linking against system libs.
>

Sure, this is what the python bindings will get. A switch to link
against the system libgpiod if desired.

> Rust is already a bit special - at least compared to traditional C
> projects - in that it pulls all the code of all dependencies and builds
> the entire bunch into a static lib. This is still upsetting some people
> who prefer each dependency to be its own self-contained dynamic library
> that can be updated without updating the rest of it. Rust can make
> that work by being a mono-culture in term of build tools. Everything on
> crates.io builds through cargo in the same way, so tooling can assist
> with auditing and updating crates.
>
> While distros - to various degrees - are starting to accept that this
> may be a viable approach, I imagine that they would still feel uneasy
> about some Rust code statically linking a C lib. They spend a lot of
> effort making sure that the C lib is built according to their packaging
> guidelines and do not want to repeat that effort for each Rust binary
> that ends up including libgpiod.
>
> Hence, I think that we would still need to provide a way to build and
> link against the system libs.
>
> However, that does not necessarily means
> that we need to feature gate pending features like suggested here.
> We _could_ just tie the Rust bindings to the libgpiod version 1:1 and
> make bindings only support one version of the C lib.
>
> That would come with some implications for the release  though. We would
> effectively need to manage bugfixes to the bindings through libgpiod's
> stable branches. For the version numbers to be non-confusing, we would
> probably also need to always do the releases together. So there may
> be needs to do a new patch release for libgpiod even if only a fix was
> needed in the bindings.
>
> Then, we may end up in a situation where SemVer changes may be
> conflicting between bindings and the core lib. With the Rust bindings
> being new, my recent fix of a soundness issue required changes to types
> that required a bump of the major version. The C version was entirely
> unaffected. Should we have bumped its major for this too?
>

Absolutely not! I expect rust bindings to still be quite volatile in
terms of API stability but libgpiod v2 is pretty mature. Distros
already seem to be slow adopting the new release so I definitely don't
want to confuse them even more by bumping libgpiod major version
because of a change in rust bindings.

We decided to fully decouple rust from libgpiod and let's keep it that way.

> I outlined these kind of issues in my initial thread around publishing
> to crates.io [1]. We could of course just document somewhere which
> version of the Rust crate pulls in which version of the C lib.
>
> But is it worth it? We would still need to maintain the nested build
> next to building against system libs. libgpiod is of course fairly
> boring to build. But we would still need to tell users to install the
> few build dependencies, C tools may update and the latest crates.io
> release may fail building.

The core libgpiod library requires a sane toolchain and standard
library with GNU extensions period. That will not change and I'm quite
convinced that if an environment has cargo and rustcc and it will have
a C compiler as well.

Bart

>
> I totally see that static builds may be helpful in some cases. But I
> would assume that the maintenance overhead of nesting the build is less
> controllable compared to what I suggest in this patch. The C lib already
> follows the good practices around versioning and API compatibility. So
> I would assume that maintaining the conditional compiles will be fairly
> simple.
>
> Once another major bump happens in the C lib, we can just follow the
> bump by removing all existing conditionals and bumping the required
> floor version for the Rust bindings.
>
> All in all, I only see minor advantages for users while seeing a lot of
> maintenance burden.
>
> [1] https://lore.kernel.org/all/CT0CRWOTJIEO.20BGIDMLFM0E8@fedora/
>
> - Erik
>

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Erik Schilling]

On Mon Oct 9, 2023 at 2:43 PM CEST, Bartosz Golaszewski wrote:
> On Mon, Oct 9, 2023 at 1:38 PM Erik Schilling <erik.schilling@linaro.org> wrote:
> >
> > On Mon Oct 9, 2023 at 10:58 AM CEST, Bartosz Golaszewski wrote:
> > > On Fri, Oct 6, 2023 at 9:24 AM Erik Schilling <erik.schilling@linaro.org> wrote:
> > > >
> > > > When releasing the 0.2.0 version of the libgpiod crate, I did not
> > > > realize that there were C lib features that were not released yet.
> > > > Helpfully, vhost-device's CI went up in flames and revealed this [1].
> > > >
> > > > This suggests a way to handle that and sketches how further updates can
> > > > be handled.
> > > >
> > > > I acknowledge that this may be very strange to C developers...
> > > > Traditionally, one would just use whatever your distro provided and the
> > > > distro would make sure that dependencies update in lock-step.
> > > >
> > > > However, in Rust the default way to consume libraries is to pull them
> > > > from crates.io. This is a balancing act for -sys crates which link to
> > > > distro-provided libraries on the system. Since crates.io does not wait
> > > > for distros to update their libraries, crates will need to support a
> > > > wider range of system libraries.
> > > >
> > > > This sets up / sketches the infrastructure for that.
> > > >
> > > > Only the first commit is intended to be merged. The second one just
> > > > sketches how a release will look like once it happens.
> > > >
> > > > [1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479
> > > >
> > > > To: Bartosz Golaszewski <brgl@bgdev.pl>
> > > > To: Linux-GPIO <linux-gpio@vger.kernel.org>
> > > > Cc: Viresh Kumar <viresh.kumar@linaro.org>
> > > > Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> > > > Cc: Kent Gibson <warthog618@gmail.com>
> > > >
> > > > Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
> > > > ---
> > > > Erik Schilling (2):
> > > >       bindings: rust: feature gate unreleased features
> > > >       DONOTMERGE: bindings: rust: simulate v2.1 release
> > > >
> > > >  bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
> > > >  bindings/rust/libgpiod/Cargo.toml            |  4 ++++
> > > >  bindings/rust/libgpiod/Makefile.am           |  2 +-
> > > >  bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
> > > >  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
> > > >  bindings/rust/libgpiod/tests/line_request.rs |  1 +
> > > >  6 files changed, 29 insertions(+), 3 deletions(-)
> > > > ---
> > > > base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
> > > > change-id: 20231006-b4-bindings-old-version-fix-789973703b77
> > > >
> > > > Best regards,
> > > > --
> > > > Erik Schilling <erik.schilling@linaro.org>
> > > >
> > >
> > > I'm currently discussing a similar problem with a developer who
> > > offered to help make Python bindings released on PyPi more
> > > user-friendly. His suggestion for python bindings is what many PyPi
> > > packages that provide bindings to C libraries do: just compile the
> > > library statically and make it part of the C extension module bundled
> > > together with python code.
> > >
> > > Given that rust programs are statically linked, maybe this is a
> > > solution we could use?
> >
> > It may be one option that we could provide, but I still see need for
> > linking against system libs.
> >
>
> Sure, this is what the python bindings will get. A switch to link
> against the system libgpiod if desired.

How does this switch work with python? Will it be some environment
variable or does pip have something similar to the Rust --feature flags?
>
> > Rust is already a bit special - at least compared to traditional C
> > projects - in that it pulls all the code of all dependencies and builds
> > the entire bunch into a static lib. This is still upsetting some people
> > who prefer each dependency to be its own self-contained dynamic library
> > that can be updated without updating the rest of it. Rust can make
> > that work by being a mono-culture in term of build tools. Everything on
> > crates.io builds through cargo in the same way, so tooling can assist
> > with auditing and updating crates.
> >
> > While distros - to various degrees - are starting to accept that this
> > may be a viable approach, I imagine that they would still feel uneasy
> > about some Rust code statically linking a C lib. They spend a lot of
> > effort making sure that the C lib is built according to their packaging
> > guidelines and do not want to repeat that effort for each Rust binary
> > that ends up including libgpiod.
> >
> > Hence, I think that we would still need to provide a way to build and
> > link against the system libs.
> >
> > However, that does not necessarily means
> > that we need to feature gate pending features like suggested here.
> > We _could_ just tie the Rust bindings to the libgpiod version 1:1 and
> > make bindings only support one version of the C lib.
> >
> > That would come with some implications for the release  though. We would
> > effectively need to manage bugfixes to the bindings through libgpiod's
> > stable branches. For the version numbers to be non-confusing, we would
> > probably also need to always do the releases together. So there may
> > be needs to do a new patch release for libgpiod even if only a fix was
> > needed in the bindings.
> >
> > Then, we may end up in a situation where SemVer changes may be
> > conflicting between bindings and the core lib. With the Rust bindings
> > being new, my recent fix of a soundness issue required changes to types
> > that required a bump of the major version. The C version was entirely
> > unaffected. Should we have bumped its major for this too?
> >
>
> Absolutely not! I expect rust bindings to still be quite volatile in
> terms of API stability but libgpiod v2 is pretty mature. Distros
> already seem to be slow adopting the new release so I definitely don't
> want to confuse them even more by bumping libgpiod major version
> because of a change in rust bindings.
>
> We decided to fully decouple rust from libgpiod and let's keep it that way.

Sure. I brought it up as rethorical question to demonstrate the kind of
problems. I fully agree that separate version numbers are the way to go.

>
> > I outlined these kind of issues in my initial thread around publishing
> > to crates.io [1]. We could of course just document somewhere which
> > version of the Rust crate pulls in which version of the C lib.
> >
> > But is it worth it? We would still need to maintain the nested build
> > next to building against system libs. libgpiod is of course fairly
> > boring to build. But we would still need to tell users to install the
> > few build dependencies, C tools may update and the latest crates.io
> > release may fail building.
>
> The core libgpiod library requires a sane toolchain and standard
> library with GNU extensions period. That will not change and I'm quite
> convinced that if an environment has cargo and rustcc and it will have
> a C compiler as well.

Yeah. I think making sure people got the dependencies installed will be
an easy part. That said... yesterday I had to extend my aarch64 SDK
with autoconf-archive for doing a build :).

Anyway, I am mostly concerned about being "odd" and entering less well
tested paths. We are already a "special" Rust crate since we link to a
C lib outside of the Rust ecosystem. But at least we can rely on
somebody already having done the work of providing the library. Now, I
wonder in which ways an autoconf, configure and compilation automation
from a build tool that is optimized to build Rust code can fail.

I am not aware of any popular Rust crate that does this kind of static
linking to C libs by default. So I am not really counting on all the
tooling around this being very well tested across all environments and
targets.

But maybe I am just overly pessimistic :->.

Do you dislike the conditional compilations here? Alternatively, we
could also just backport the fixes that I did to the stable branch and
do a bindings release from there. Then the offending access will be gone
and we could just bump the required C lib version for the Rust bindings
on new releases from the master branch.

That would not lock us into any future and would give us more time to
consider whats best.

- Erik

>
> Bart
>
> >
> > I totally see that static builds may be helpful in some cases. But I
> > would assume that the maintenance overhead of nesting the build is less
> > controllable compared to what I suggest in this patch. The C lib already
> > follows the good practices around versioning and API compatibility. So
> > I would assume that maintaining the conditional compiles will be fairly
> > simple.
> >
> > Once another major bump happens in the C lib, we can just follow the
> > bump by removing all existing conditionals and bumping the required
> > floor version for the Rust bindings.
> >
> > All in all, I only see minor advantages for users while seeing a lot of
> > maintenance burden.
> >
> > [1] https://lore.kernel.org/all/CT0CRWOTJIEO.20BGIDMLFM0E8@fedora/
> >
> > - Erik
> >

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Bartosz Golaszewski]

On Mon, Oct 9, 2023 at 4:16 PM Erik Schilling <erik.schilling@linaro.org> wrote:
>
> On Mon Oct 9, 2023 at 2:43 PM CEST, Bartosz Golaszewski wrote:
> > On Mon, Oct 9, 2023 at 1:38 PM Erik Schilling <erik.schilling@linaro.org> wrote:
> > >
> > > On Mon Oct 9, 2023 at 10:58 AM CEST, Bartosz Golaszewski wrote:
> > > > On Fri, Oct 6, 2023 at 9:24 AM Erik Schilling <erik.schilling@linaro.org> wrote:
> > > > >
> > > > > When releasing the 0.2.0 version of the libgpiod crate, I did not
> > > > > realize that there were C lib features that were not released yet.
> > > > > Helpfully, vhost-device's CI went up in flames and revealed this [1].
> > > > >
> > > > > This suggests a way to handle that and sketches how further updates can
> > > > > be handled.
> > > > >
> > > > > I acknowledge that this may be very strange to C developers...
> > > > > Traditionally, one would just use whatever your distro provided and the
> > > > > distro would make sure that dependencies update in lock-step.
> > > > >
> > > > > However, in Rust the default way to consume libraries is to pull them
> > > > > from crates.io. This is a balancing act for -sys crates which link to
> > > > > distro-provided libraries on the system. Since crates.io does not wait
> > > > > for distros to update their libraries, crates will need to support a
> > > > > wider range of system libraries.
> > > > >
> > > > > This sets up / sketches the infrastructure for that.
> > > > >
> > > > > Only the first commit is intended to be merged. The second one just
> > > > > sketches how a release will look like once it happens.
> > > > >
> > > > > [1] https://buildkite.com/rust-vmm/vhost-device-ci/builds/1746#018b0110-b9d3-468a-973c-c3bbc27cd479
> > > > >
> > > > > To: Bartosz Golaszewski <brgl@bgdev.pl>
> > > > > To: Linux-GPIO <linux-gpio@vger.kernel.org>
> > > > > Cc: Viresh Kumar <viresh.kumar@linaro.org>
> > > > > Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> > > > > Cc: Kent Gibson <warthog618@gmail.com>
> > > > >
> > > > > Signed-off-by: Erik Schilling <erik.schilling@linaro.org>
> > > > > ---
> > > > > Erik Schilling (2):
> > > > >       bindings: rust: feature gate unreleased features
> > > > >       DONOTMERGE: bindings: rust: simulate v2.1 release
> > > > >
> > > > >  bindings/rust/libgpiod-sys/Cargo.toml        |  9 +++++++--
> > > > >  bindings/rust/libgpiod/Cargo.toml            |  4 ++++
> > > > >  bindings/rust/libgpiod/Makefile.am           |  2 +-
> > > > >  bindings/rust/libgpiod/README.md             | 14 ++++++++++++++
> > > > >  bindings/rust/libgpiod/src/line_request.rs   |  2 ++
> > > > >  bindings/rust/libgpiod/tests/line_request.rs |  1 +
> > > > >  6 files changed, 29 insertions(+), 3 deletions(-)
> > > > > ---
> > > > > base-commit: e7b02c2259d97c77107c77b68e3bc1664e6703c1
> > > > > change-id: 20231006-b4-bindings-old-version-fix-789973703b77
> > > > >
> > > > > Best regards,
> > > > > --
> > > > > Erik Schilling <erik.schilling@linaro.org>
> > > > >
> > > >
> > > > I'm currently discussing a similar problem with a developer who
> > > > offered to help make Python bindings released on PyPi more
> > > > user-friendly. His suggestion for python bindings is what many PyPi
> > > > packages that provide bindings to C libraries do: just compile the
> > > > library statically and make it part of the C extension module bundled
> > > > together with python code.
> > > >
> > > > Given that rust programs are statically linked, maybe this is a
> > > > solution we could use?
> > >
> > > It may be one option that we could provide, but I still see need for
> > > linking against system libs.
> > >
> >
> > Sure, this is what the python bindings will get. A switch to link
> > against the system libgpiod if desired.
>
> How does this switch work with python? Will it be some environment
> variable or does pip have something similar to the Rust --feature flags?
> >
> > > Rust is already a bit special - at least compared to traditional C
> > > projects - in that it pulls all the code of all dependencies and builds
> > > the entire bunch into a static lib. This is still upsetting some people
> > > who prefer each dependency to be its own self-contained dynamic library
> > > that can be updated without updating the rest of it. Rust can make
> > > that work by being a mono-culture in term of build tools. Everything on
> > > crates.io builds through cargo in the same way, so tooling can assist
> > > with auditing and updating crates.
> > >
> > > While distros - to various degrees - are starting to accept that this
> > > may be a viable approach, I imagine that they would still feel uneasy
> > > about some Rust code statically linking a C lib. They spend a lot of
> > > effort making sure that the C lib is built according to their packaging
> > > guidelines and do not want to repeat that effort for each Rust binary
> > > that ends up including libgpiod.
> > >
> > > Hence, I think that we would still need to provide a way to build and
> > > link against the system libs.
> > >
> > > However, that does not necessarily means
> > > that we need to feature gate pending features like suggested here.
> > > We _could_ just tie the Rust bindings to the libgpiod version 1:1 and
> > > make bindings only support one version of the C lib.
> > >
> > > That would come with some implications for the release  though. We would
> > > effectively need to manage bugfixes to the bindings through libgpiod's
> > > stable branches. For the version numbers to be non-confusing, we would
> > > probably also need to always do the releases together. So there may
> > > be needs to do a new patch release for libgpiod even if only a fix was
> > > needed in the bindings.
> > >
> > > Then, we may end up in a situation where SemVer changes may be
> > > conflicting between bindings and the core lib. With the Rust bindings
> > > being new, my recent fix of a soundness issue required changes to types
> > > that required a bump of the major version. The C version was entirely
> > > unaffected. Should we have bumped its major for this too?
> > >
> >
> > Absolutely not! I expect rust bindings to still be quite volatile in
> > terms of API stability but libgpiod v2 is pretty mature. Distros
> > already seem to be slow adopting the new release so I definitely don't
> > want to confuse them even more by bumping libgpiod major version
> > because of a change in rust bindings.
> >
> > We decided to fully decouple rust from libgpiod and let's keep it that way.
>
> Sure. I brought it up as rethorical question to demonstrate the kind of
> problems. I fully agree that separate version numbers are the way to go.
>
> >
> > > I outlined these kind of issues in my initial thread around publishing
> > > to crates.io [1]. We could of course just document somewhere which
> > > version of the Rust crate pulls in which version of the C lib.
> > >
> > > But is it worth it? We would still need to maintain the nested build
> > > next to building against system libs. libgpiod is of course fairly
> > > boring to build. But we would still need to tell users to install the
> > > few build dependencies, C tools may update and the latest crates.io
> > > release may fail building.
> >
> > The core libgpiod library requires a sane toolchain and standard
> > library with GNU extensions period. That will not change and I'm quite
> > convinced that if an environment has cargo and rustcc and it will have
> > a C compiler as well.
>
> Yeah. I think making sure people got the dependencies installed will be
> an easy part. That said... yesterday I had to extend my aarch64 SDK
> with autoconf-archive for doing a build :).
>
> Anyway, I am mostly concerned about being "odd" and entering less well
> tested paths. We are already a "special" Rust crate since we link to a
> C lib outside of the Rust ecosystem. But at least we can rely on
> somebody already having done the work of providing the library. Now, I
> wonder in which ways an autoconf, configure and compilation automation
> from a build tool that is optimized to build Rust code can fail.
>

I'm Cc'ing Phil Howard who's the developer behind the Python bindings work.

In Phil's WiP branch[1] that should soon be posted to this list the
autotools flow is entirely omitted and building of the libgpiod C
sources happens in setup.py directly. Can cargo compile C sources like
that?

> I am not aware of any popular Rust crate that does this kind of static
> linking to C libs by default. So I am not really counting on all the
> tooling around this being very well tested across all environments and
> targets.
>
> But maybe I am just overly pessimistic :->.
>
> Do you dislike the conditional compilations here? Alternatively, we
> could also just backport the fixes that I did to the stable branch and
> do a bindings release from there. Then the offending access will be gone
> and we could just bump the required C lib version for the Rust bindings
> on new releases from the master branch.
>

I'm not sure how that would work honestly. The stable branches in
libgpiod are per libgpiod minor release. This doesn't map onto rust
releases anymore with decoupled versioning. Maybe rust should get its
own tags in the repo (on the master branch for major and minor
releases) and its own stable branches?

> That would not lock us into any future and would give us more time to
> consider whats best.
>

Bart

[1] https://github.com/Gadgetoid/libgpiod/tree/python-bindings-2

> - Erik
>
> >
> > Bart
> >
> > >
> > > I totally see that static builds may be helpful in some cases. But I
> > > would assume that the maintenance overhead of nesting the build is less
> > > controllable compared to what I suggest in this patch. The C lib already
> > > follows the good practices around versioning and API compatibility. So
> > > I would assume that maintaining the conditional compiles will be fairly
> > > simple.
> > >
> > > Once another major bump happens in the C lib, we can just follow the
> > > bump by removing all existing conditionals and bumping the required
> > > floor version for the Rust bindings.
> > >
> > > All in all, I only see minor advantages for users while seeing a lot of
> > > maintenance burden.
> > >
> > > [1] https://lore.kernel.org/all/CT0CRWOTJIEO.20BGIDMLFM0E8@fedora/
> > >
> > > - Erik
> > >
>

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Manos Pitsidianakis]

On Mon, 09 Oct 2023 17:32, Bartosz Golaszewski <brgl@bgdev.pl> wrote:
>I'm Cc'ing Phil Howard who's the developer behind the Python bindings 
>work.
>
>In Phil's WiP branch[1] that should soon be posted to this list the
>autotools flow is entirely omitted and building of the libgpiod C
>sources happens in setup.py directly. Can cargo compile C sources like
>that?

The rust compiler team maintains a library for that:

https://crates.io/crates/cc

You can find examples of it in use in many popular rust crates, like 
when building the openssl crate https://docs.rs/openssl/latest/openssl/ 
with the `vendored` feature, it uses the following build-time dependency 
to build the static librarie:

https://github.com/alexcrichton/openssl-src-rs/tree/main

There is no general need to put the vendoring code in a build-time 
dependency by the way, it can be done in in the bindings crate's 
build.rs as well.

>
>I'm not sure how that would work honestly. The stable branches in
>libgpiod are per libgpiod minor release. This doesn't map onto rust
>releases anymore with decoupled versioning. Maybe rust should get its
>own tags in the repo (on the master branch for major and minor
>releases) and its own stable branches?

In cases Rust crates want to support multiple releases, the usual route 
is to expose different bindings per release exposed via feature flags.

I can't say if that makes sense for libgpiod though, because I'm not 
familiar that much.

Manos

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Erik Schilling]

On Mon Oct 9, 2023 at 4:39 PM CEST, Manos Pitsidianakis wrote:
> On Mon, 09 Oct 2023 17:32, Bartosz Golaszewski <brgl@bgdev.pl> wrote:
> >I'm Cc'ing Phil Howard who's the developer behind the Python bindings 
> >work.

Hi!

> >
> >In Phil's WiP branch[1] that should soon be posted to this list the
> >autotools flow is entirely omitted and building of the libgpiod C
> >sources happens in setup.py directly. Can cargo compile C sources like
> >that?
>
> The rust compiler team maintains a library for that:
>
> https://crates.io/crates/cc
>
> You can find examples of it in use in many popular rust crates, like 
> when building the openssl crate https://docs.rs/openssl/latest/openssl/ 
> with the `vendored` feature, it uses the following build-time dependency 
> to build the static librarie:
>
> https://github.com/alexcrichton/openssl-src-rs/tree/main
>
> There is no general need to put the vendoring code in a build-time 
> dependency by the way, it can be done in in the bindings crate's 
> build.rs as well.

Right. One can use cc, there also seems to be a somewhat popular crate
that allows calling autotools: https://crates.io/crates/autotools.

That said. I am not sure if I like listing all the sources and defining
the build process manually again. It feels like we duplicate what the
existing build system already does for us and no longer have a single
source of truth...

Taking a look at the openssl build code [1] I also see all the target
and environment specific hacks that I feared about... I guess it won't
be as bad for libgpiod, but I guess it might be a painful way to figure
out whether that is true.

I have seen similar things happening when cmake projects attempted to
vendor in external dependencies and are not a huge fan of marrying to
different worlds together like this.

That said. I am completely supportive to the idea of exploring static
linking for the Rust bindings. I am just sceptical that doing that by
default will make things more simple for consumers on the long run.

[1] https://github.com/alexcrichton/openssl-src-rs/blob/main/src/lib.rs

>
> >
> >I'm not sure how that would work honestly. The stable branches in
> >libgpiod are per libgpiod minor release. This doesn't map onto rust
> >releases anymore with decoupled versioning. Maybe rust should get its
> >own tags in the repo (on the master branch for major and minor
> >releases) and its own stable branches?
>
> In cases Rust crates want to support multiple releases, the usual route 
> is to expose different bindings per release exposed via feature flags.
>
> I can't say if that makes sense for libgpiod though, because I'm not 
> familiar that much.

Thats true for attempting to support different versions of the C lib
(and it is what I suggest in this series). However, the recent release
became necessary to a bug in the Rust bindings, not in the C lib. So a
stable branch could still make sense. But hopefully, we would only need
it increasingly rarely in the future.

Overall I still think what I suggest in this patch + maybe exploring
optional static linking is the simplest path. I mostly suggested this as
an alternative since I felt resistance to my suggestion :)

- Erik

Re: [libgpiod][PATCH 0/2] bindings: rust: feature gate unreleased features

[Bartosz Golaszewski]

On Mon, Oct 9, 2023 at 5:21 PM Erik Schilling <erik.schilling@linaro.org> wrote:
>
> On Mon Oct 9, 2023 at 4:39 PM CEST, Manos Pitsidianakis wrote:
> > On Mon, 09 Oct 2023 17:32, Bartosz Golaszewski <brgl@bgdev.pl> wrote:
> > >I'm Cc'ing Phil Howard who's the developer behind the Python bindings
> > >work.
>
> Hi!
>
> > >
> > >In Phil's WiP branch[1] that should soon be posted to this list the
> > >autotools flow is entirely omitted and building of the libgpiod C
> > >sources happens in setup.py directly. Can cargo compile C sources like
> > >that?
> >
> > The rust compiler team maintains a library for that:
> >
> > https://crates.io/crates/cc
> >
> > You can find examples of it in use in many popular rust crates, like
> > when building the openssl crate https://docs.rs/openssl/latest/openssl/
> > with the `vendored` feature, it uses the following build-time dependency
> > to build the static librarie:
> >
> > https://github.com/alexcrichton/openssl-src-rs/tree/main
> >
> > There is no general need to put the vendoring code in a build-time
> > dependency by the way, it can be done in in the bindings crate's
> > build.rs as well.
>
> Right. One can use cc, there also seems to be a somewhat popular crate
> that allows calling autotools: https://crates.io/crates/autotools.
>
> That said. I am not sure if I like listing all the sources and defining
> the build process manually again. It feels like we duplicate what the
> existing build system already does for us and no longer have a single
> source of truth...
>
> Taking a look at the openssl build code [1] I also see all the target
> and environment specific hacks that I feared about... I guess it won't
> be as bad for libgpiod, but I guess it might be a painful way to figure
> out whether that is true.
>
> I have seen similar things happening when cmake projects attempted to
> vendor in external dependencies and are not a huge fan of marrying to
> different worlds together like this.
>
> That said. I am completely supportive to the idea of exploring static
> linking for the Rust bindings. I am just sceptical that doing that by
> default will make things more simple for consumers on the long run.
>
> [1] https://github.com/alexcrichton/openssl-src-rs/blob/main/src/lib.rs
>
> >
> > >
> > >I'm not sure how that would work honestly. The stable branches in
> > >libgpiod are per libgpiod minor release. This doesn't map onto rust
> > >releases anymore with decoupled versioning. Maybe rust should get its
> > >own tags in the repo (on the master branch for major and minor
> > >releases) and its own stable branches?
> >
> > In cases Rust crates want to support multiple releases, the usual route
> > is to expose different bindings per release exposed via feature flags.
> >
> > I can't say if that makes sense for libgpiod though, because I'm not
> > familiar that much.
>
> Thats true for attempting to support different versions of the C lib
> (and it is what I suggest in this series). However, the recent release
> became necessary to a bug in the Rust bindings, not in the C lib. So a
> stable branch could still make sense. But hopefully, we would only need
> it increasingly rarely in the future.
>
> Overall I still think what I suggest in this patch + maybe exploring
> optional static linking is the simplest path. I mostly suggested this as
> an alternative since I felt resistance to my suggestion :)
>
> - Erik
>

Fair enough, I applied patch 1/2.

Bart