Linux GPIO internals

Linux GPIO internals

[Michael Welling]

All,

For years now EMAC has provided an out-of-tree series of class drivers
for accessing various devices. The EMAC GPIO class and character
interfaces predate the introduction of the gpiolib interface and have
been ported across several kernel versions.

http://wiki.emacinc.com/wiki/Using_the_EMAC_GPIO_Class

Recently we have come to the conclusion that continuing to provide
support for these drivers is getting out of hand. It was agreed that we
move away from our non-standard drivers and use mainstream drivers for our
newest products.

That being said, we would like to be able to provide the capabilities
of our old drivers but it is not the case with the current gpiolib
implementation.

Here are the major concerns that we have with the gpiolib implementation:
- There is no mechanism to provide simultaneous access to multiple
  GPIOs from userspace. 
- The sysfs interface seems to vastly slower than the character
  interface and it is far more cumbersome to handle access from a
  userspace C program.

It seems that the first concern was attempted to be addressed by the
following patch:
https://lkml.org/lkml/2012/12/23/66

It seems this effort dropped off the radar in January of 2013.
What happened to this patch?

As for the second issue, I am not sure how to resolve this and am open
to ideas. I have seen similar concerns in other subsystem that use the
sysfs interface.

IIO example: 
http://www.spinics.net/lists/linux-iio/msg15344.html

Suggestions?

Re: Linux GPIO internals

[Alexandre Courbot]

Hi Michael, sorry for the delayed reply on this interesting issue.

On Wed, Dec 24, 2014 at 10:53 PM, Michael Welling <mwelling@emacinc.com> wrote:
> All,
>
> For years now EMAC has provided an out-of-tree series of class drivers
> for accessing various devices. The EMAC GPIO class and character
> interfaces predate the introduction of the gpiolib interface and have
> been ported across several kernel versions.
>
> http://wiki.emacinc.com/wiki/Using_the_EMAC_GPIO_Class
>
> Recently we have come to the conclusion that continuing to provide
> support for these drivers is getting out of hand. It was agreed that we
> move away from our non-standard drivers and use mainstream drivers for our
> newest products.
>
> That being said, we would like to be able to provide the capabilities
> of our old drivers but it is not the case with the current gpiolib
> implementation.
>
> Here are the major concerns that we have with the gpiolib implementation:
> - There is no mechanism to provide simultaneous access to multiple
>   GPIOs from userspace.
> - The sysfs interface seems to vastly slower than the character
>   interface and it is far more cumbersome to handle access from a
>   userspace C program.
>
> It seems that the first concern was attempted to be addressed by the
> following patch:
> https://lkml.org/lkml/2012/12/23/66
>
> It seems this effort dropped off the radar in January of 2013.
> What happened to this patch?
>
> As for the second issue, I am not sure how to resolve this and am open
> to ideas. I have seen similar concerns in other subsystem that use the
> sysfs interface.
>
> IIO example:
> http://www.spinics.net/lists/linux-iio/msg15344.html
>
> Suggestions?

It seems like your mail is coming just at the right time. We have
recently merged a patch that allows setting several GPIOs at the same
time, if the hardware supports it:

http://patchwork.ozlabs.org/patch/406666/

It lacks sysfs support, but we are also increasingly feeling the
pressure of the old sysfs interface and are planning to develop a new
one:

http://lists.infradead.org/pipermail/linux-arm-kernel/2014-December/311158.html

Our main gripe with the old sysfs interface is its use of the
(potentially varying) integer GPIO namespace, and we would like one
that relies on the new gpiod_ functions. But while we are at it, we
could just as well support GPIO groups and try to make things faster,
although I am not sure why sysfs would be slower than a character
device.

On the top of my head, I think we could have an export interface that
allows to export a group of GPIOs under a given name, and to
manipulate them using gpiod_set/get_array(). Single GPIOs could be
exported as a group of 1.

I have looked at the EMAC GPIO class (very low level, and clearly tied
to a given subset of GPIO hardware), and anything that makes it
upstream would need to feature considerably more abstraction, but we
can certainly try to work something out.

Considering the constraints that we have (no GPIO integers for
exporting, sysfs-based, uses gpiod_*array()), do you think we could
satisfy your goals as well?

Re: Linux GPIO internals

[Michael Welling]

On Sat, Jan 03, 2015 at 02:12:06PM +0100, Alexandre Courbot wrote:
> Hi Michael, sorry for the delayed reply on this interesting issue.
> 
> On Wed, Dec 24, 2014 at 10:53 PM, Michael Welling <mwelling@emacinc.com> wrote:
> > All,
> >
> > For years now EMAC has provided an out-of-tree series of class drivers
> > for accessing various devices. The EMAC GPIO class and character
> > interfaces predate the introduction of the gpiolib interface and have
> > been ported across several kernel versions.
> >
> > http://wiki.emacinc.com/wiki/Using_the_EMAC_GPIO_Class
> >
> > Recently we have come to the conclusion that continuing to provide
> > support for these drivers is getting out of hand. It was agreed that we
> > move away from our non-standard drivers and use mainstream drivers for our
> > newest products.
> >
> > That being said, we would like to be able to provide the capabilities
> > of our old drivers but it is not the case with the current gpiolib
> > implementation.
> >
> > Here are the major concerns that we have with the gpiolib implementation:
> > - There is no mechanism to provide simultaneous access to multiple
> >   GPIOs from userspace.
> > - The sysfs interface seems to vastly slower than the character
> >   interface and it is far more cumbersome to handle access from a
> >   userspace C program.
> >
> > It seems that the first concern was attempted to be addressed by the
> > following patch:
> > https://lkml.org/lkml/2012/12/23/66
> >
> > It seems this effort dropped off the radar in January of 2013.
> > What happened to this patch?
> >
> > As for the second issue, I am not sure how to resolve this and am open
> > to ideas. I have seen similar concerns in other subsystem that use the
> > sysfs interface.
> >
> > IIO example:
> > http://www.spinics.net/lists/linux-iio/msg15344.html
> >
> > Suggestions?
> 
> It seems like your mail is coming just at the right time. We have
> recently merged a patch that allows setting several GPIOs at the same
> time, if the hardware supports it:
> 
> http://patchwork.ozlabs.org/patch/406666/
>

This looks like a good starting point. Being able to access to multiple
GPIOs simultaneously on the same controller bank is great.

What is nice about the EMAC class is that the GPIOs do not necessarily
have to be on the same bank (or controller) to be grouped together. The
system call overhead of accessing a single IO is the same as multiple
IOs in the same group.

Still wondering what happened to the gpioblock patch.

> It lacks sysfs support, but we are also increasingly feeling the
> pressure of the old sysfs interface and are planning to develop a new
> one:
> 
> http://lists.infradead.org/pipermail/linux-arm-kernel/2014-December/311158.html
> 
> Our main gripe with the old sysfs interface is its use of the
> (potentially varying) integer GPIO namespace, and we would like one
> that relies on the new gpiod_ functions. But while we are at it, we
> could just as well support GPIO groups and try to make things faster,
> although I am not sure why sysfs would be slower than a character
> device.

The with the migration from the legacy GPIO namespace it only seems
natural for the sysfs interface to follow. Adding groups in the process
would provide most of what is needed.

The sysfs interface is great for command line and scripting languages
but it has more overhead. It requires string conversion at both the
kernel and userspace. More system calls are typically required for
similar transactions.

> 
> On the top of my head, I think we could have an export interface that
> allows to export a group of GPIOs under a given name, and to
> manipulate them using gpiod_set/get_array(). Single GPIOs could be
> exported as a group of 1.
>

This sounds good. Currently we group the IOs with devicetree
registrations.

http://git.emacinc.com/source/linux-at91.git/blob/linux-3.10-at91/arch/arm/boot/dts/ipac-9x25.dts#L226

It would be nice if it were possible to create the groups from
userspace.

> I have looked at the EMAC GPIO class (very low level, and clearly tied
> to a given subset of GPIO hardware), and anything that makes it
> upstream would need to feature considerably more abstraction, but we
> can certainly try to work something out.

I am more worried about standardizing EMAC's code base than pushing our
old code into the mainline.

> 
> Considering the constraints that we have (no GPIO integers for
> exporting, sysfs-based, uses gpiod_*array()), do you think we could
> satisfy your goals as well?

This should be satisfactory for most use cases. I will try to support
efforts toward modernizing the sysfs interface.

Would a character interface to gpiolib ever be considered? 

Re: Linux GPIO internals

[Linus Walleij]

On Mon, Jan 5, 2015 at 6:47 PM, Michael Welling <mwelling-ei31t81uNw5BDgjK7y7TUQ@public.gmane.org> wrote:
> On Sat, Jan 03, 2015 at 02:12:06PM +0100, Alexandre Courbot wrote:

>> It seems like your mail is coming just at the right time. We have
>> recently merged a patch that allows setting several GPIOs at the same
>> time, if the hardware supports it:
>>
>> http://patchwork.ozlabs.org/patch/406666/
>
> This looks like a good starting point. Being able to access to multiple
> GPIOs simultaneously on the same controller bank is great.

It's intended to handle the case when several GPIOs can be
switched at the same time by a single register write. It implies
being on the same controller.

> What is nice about the EMAC class is that the GPIOs do not necessarily
> have to be on the same bank (or controller) to be grouped together. The
> system call overhead of accessing a single IO is the same as multiple
> IOs in the same group.

You're comparing pears and apples now I think.

A userspace ABI making it possible to switch several GPIOs
on several controllers is perfectly doable both with and without
the above interface.

In this latter case you're more worried about the latency
incurred by the userspace/kernelspace switch, whereas
the former is about the delays incurred by several register
writes.

Userspace/kernelspace switch delay is a few magnitudes
larger than the delay between sequenced register writes I
suspect.

> Still wondering what happened to the gpioblock patch.

Roland?

> The sysfs interface is great for command line and scripting languages
> but it has more overhead. It requires string conversion at both the
> kernel and userspace. More system calls are typically required for
> similar transactions.

Agree. I have a problem with it too.

>> Considering the constraints that we have (no GPIO integers for
>> exporting, sysfs-based, uses gpiod_*array()), do you think we could
>> satisfy your goals as well?
>
> This should be satisfactory for most use cases. I will try to support
> efforts toward modernizing the sysfs interface.
>
> Would a character interface to gpiolib ever be considered?

I like the character interface idea actually.

/dev/gpiochip0
/dev/gpiochip1
(...)

Then ioctl() operations to do all the magic to figure out what GPIOs
are there and how to read/write them etc. To me this reflects the
system properly and gives all kind of freedom to manipulate the
GPIOs with efficient context switches. Plus we can deliberately
avoid including any GPIO numbers anywhere, just allow names
and nothing else.

But I'm no userspace/kernelspace expert, let's hear what
the linux-api mailing list has to say.

Yours,
Linus Walleij

Re: Linux GPIO internals

[Linus Walleij]

On Mon, Jan 5, 2015 at 6:47 PM, Michael Welling <mwelling@emacinc.com> wrote:
> On Sat, Jan 03, 2015 at 02:12:06PM +0100, Alexandre Courbot wrote:

>> It seems like your mail is coming just at the right time. We have
>> recently merged a patch that allows setting several GPIOs at the same
>> time, if the hardware supports it:
>>
>> http://patchwork.ozlabs.org/patch/406666/
>
> This looks like a good starting point. Being able to access to multiple
> GPIOs simultaneously on the same controller bank is great.

It's intended to handle the case when several GPIOs can be
switched at the same time by a single register write. It implies
being on the same controller.

> What is nice about the EMAC class is that the GPIOs do not necessarily
> have to be on the same bank (or controller) to be grouped together. The
> system call overhead of accessing a single IO is the same as multiple
> IOs in the same group.

You're comparing pears and apples now I think.

A userspace ABI making it possible to switch several GPIOs
on several controllers is perfectly doable both with and without
the above interface.

In this latter case you're more worried about the latency
incurred by the userspace/kernelspace switch, whereas
the former is about the delays incurred by several register
writes.

Userspace/kernelspace switch delay is a few magnitudes
larger than the delay between sequenced register writes I
suspect.

> Still wondering what happened to the gpioblock patch.

Roland?

> The sysfs interface is great for command line and scripting languages
> but it has more overhead. It requires string conversion at both the
> kernel and userspace. More system calls are typically required for
> similar transactions.

Agree. I have a problem with it too.

>> Considering the constraints that we have (no GPIO integers for
>> exporting, sysfs-based, uses gpiod_*array()), do you think we could
>> satisfy your goals as well?
>
> This should be satisfactory for most use cases. I will try to support
> efforts toward modernizing the sysfs interface.
>
> Would a character interface to gpiolib ever be considered?

I like the character interface idea actually.

/dev/gpiochip0
/dev/gpiochip1
(...)

Then ioctl() operations to do all the magic to figure out what GPIOs
are there and how to read/write them etc. To me this reflects the
system properly and gives all kind of freedom to manipulate the
GPIOs with efficient context switches. Plus we can deliberately
avoid including any GPIO numbers anywhere, just allow names
and nothing else.

But I'm no userspace/kernelspace expert, let's hear what
the linux-api mailing list has to say.

Yours,
Linus Walleij