Looking for good references for ARM driver development

Looking for good references for ARM driver development

[Mason]

Hello everyone,

I've been using several Linux distributions, and writing user-space programs, for 15 years.
I recently seized an opportunity to move into kernel development, mainly writing drivers
for an ARM SoC, and I'm finding the transition harder than I expected.

I'm having a hard time finding good reference material to bring me up to speed on the ins
and outs of driver development. Sure, I found Documentation/driver-model (eventually) but
I'm still missing the high-level overview that ties everything together, and makes my
brain click and "get it".

Many years ago, I read LDD2 and it was an eye-opener. We do have LDD3 at work, but it is
now almost 10 years old, and it doesn't document things like the platform bus, or the "new"
driver model, or hwmon, or cpufreq, or device tree, to name a few things on my plate.
(I'm writing drivers for the 3.14 kernel.)

http://www.xml.com/ldd/chapter/book/
http://www.makelinux.net/ldd3/

Are there more recent technical references, as good as LDD3, that cover "modern" aspects
of kernel development?

I've read good things about:

"Writing Linux Device Drivers: a guide with exercises"
by Jerry Cooperstein (2009)
originally written for 2.6.31, updated for 3.3
http://www.coopj.com/

What do kernel devs recommend as the worthy successor to LDD3?


To wrap up this (long) message, I'd like to post some of my questions, to highlight some
of my confusion. I'm writing a driver for a temperature sensor, which is supposed to work
within the hwmon/lm-sensors framework.

The sensor's API consists of 3 memory-mapped registers, which are accessible over the
SoC's memory bus. I first wrote a user-space proof-of-concept to interact with the
sensors, accessing them through /dev/mem O_SYNC + mmap. That works as expected, and
I now have the code to read the temperature when needed.

Next, I looked at the driver glue required to use these sensors from lm-sensors.
That's where my troubles started.

1) Which bus should I be using for this driver? Is the platform bus appropriate?

2) platform.txt states

> Some drivers are not fully converted to the driver model, because they take
> on a non-driver role:  the driver registers its platform device, rather than
> leaving that for system infrastructure.  Such drivers can't be hotplugged
> or coldplugged, since those mechanisms require device creation to be in a
> different system component than the driver.

How do I "leave device registration for the system infrastructure"?
Where should I put that code?
Is it a good idea to separate device registration and driver registration
in the case of a SoC, where the device is embedded in the SoC and is not
"hot-plugged" (or anything-plugged for that matter, it's just "there").

3) Why is the function used to "destroy a platform device" named
platform_device_put? Why put?
Put on a list of things to destroy at a later time?

4) Can I use platform_driver_probe, instead of platform_driver_register?
The comments in platform.c state:
> /**
>  * platform_driver_probe - register driver for non-hotpluggable device
>  * @drv: platform driver structure
>  * @probe: the driver probe routine, probably from an __init section
>  *
>  * Use this instead of platform_driver_register() when you know the device
>  * is not hotpluggable and has already been registered, and you want to
>  * remove its run-once probe() infrastructure from memory after the driver
>  * has bound to the device.
>  *
>  * One typical use for this would be with drivers for controllers integrated
>  * into system-on-chip processors, where the controller devices have been
>  * configured as part of board setup.
>  *
>  * Note that this is incompatible with deferred probing.
>  *
>  * Returns zero if the driver registered and bound to a device, else returns
>  * a negative error code and with the driver not registered.
>  */

AFAICT, I need to answer question 2 before I can answer this one.
How do I get "devices have been configured as part of board setup."?

5) Very hwmon-specific, how often are temperature sensors probed?
Does the frequency vary? (Through /sys knobs? Or something else?)


That's all I have on my mind at the moment (well, aside from some ARM TLB
shenanigans, but that's another story for another time).

I shall be eternally grateful to anyone who can enlighten me through advice,
pointers, references, documentation, voodoo, etc.

Regards.

Re: Looking for good references for ARM driver development

[Andreas Färber]

Hi,

Am 19.11.2014 um 13:50 schrieb Mason:
> [...] I'm writing a driver for a temperature sensor, which is
> supposed to work
> within the hwmon/lm-sensors framework.
> 
> The sensor's API consists of 3 memory-mapped registers, which are
> accessible over the
> SoC's memory bus. [...]
> 
> 1) Which bus should I be using for this driver? Is the platform bus
> appropriate?

Probably.

> 2) platform.txt states
> 
>> Some drivers are not fully converted to the driver model, because they
>> take
>> on a non-driver role:  the driver registers its platform device,
>> rather than
>> leaving that for system infrastructure.  Such drivers can't be hotplugged
>> or coldplugged, since those mechanisms require device creation to be in a
>> different system component than the driver.
> 
> How do I "leave device registration for the system infrastructure"?
> Where should I put that code?
> Is it a good idea to separate device registration and driver registration
> in the case of a SoC, where the device is embedded in the SoC and is not
> "hot-plugged" (or anything-plugged for that matter, it's just "there").

Since this appears to be about an ARM SoC according to your To list, in
general, you create a device tree binding, that binding is registered
within your platform/... driver code and referenced in the device tree
for SoC or board, and then your driver will automatically be probed.

> 4) Can I use platform_driver_probe, instead of platform_driver_register?

Most likely you do not need to call either yourself.

Just compare other platform drivers on the one hand, and temparature
sensor drivers on the other (such as I2C based gmt,g781 / LM90).
Did you already check whether there is a driver that is both?

Regards,
Andreas

-- 
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 21284 AG Nürnberg

Re: Looking for good references for ARM driver development

[Mason]

Hello Andreas,

On 19/11/2014 16:02, Andreas Färber wrote:

> Am 19.11.2014 um 13:50 schrieb Mason:
>
>> [...] I'm writing a driver for a temperature sensor, which is
>> supposed to work within the hwmon/lm-sensors framework.
>>
>> The sensor's API consists of 3 memory-mapped registers, which are
>> accessible over the SoC's memory bus. [...]
>>
>> 1) Which bus should I be using for this driver?
>> Is the platform bus appropriate?
>
> Probably.

Is there an exhaustive list of available buses (on the ARM platform)
and an overview of when/where each one is appropriate?

>> 2) platform.txt states
>>
>>> Some drivers are not fully converted to the driver model, because
>>> they take on a non-driver role:  the driver registers its
>>> platform device, rather than leaving that for system
>>> infrastructure.  Such drivers can't be hotplugged or coldplugged,
>>> since those mechanisms require device creation to be in a
>>> different system component than the driver.
>>
>> How do I "leave device registration for the system
>> infrastructure"? Where should I put that code? Is it a good idea to
>> separate device registration and driver registration in the case of
>> a SoC, where the device is embedded in the SoC and is not
>> "hot-plugged" (or anything-plugged for that matter, it's just
>> "there").
>
> Since this appears to be about an ARM SoC according to your To list,
> in general, you create a device tree binding, that binding is
> registered within your platform/... driver code and referenced in the
> device tree for SoC or board, and then your driver will automatically
> be probed.

I know nothing about DT (aside from the Wikipedia entry).
I'll take a closer look at Documentation/devicetree.
Will that explain what platform/... is?

I see a drivers/platform folder, but nothing ARM-specific there?

>> 4) Can I use platform_driver_probe, instead of
>> platform_driver_register?
>
> Most likely you do not need to call either yourself.

Hmmm, color me even more confused. I had really come to believe that
driver registration was a mandatory part of the driver, something
that wasn't left to "infrastructure code".

> Just compare other platform drivers on the one hand, and temperature
> sensor drivers on the other (such as I2C based gmt,g781 / LM90). Did
> you already check whether there is a driver that is both?

Please excuse my naive question: what are platform drivers, and where
are they stored in the kernel source tree?

Regards.

Re: Looking for good references for ARM driver development

[Victor Ascroft]

On 11/19/2014 06:20 PM, Mason wrote:
> Hello everyone,
> 
> I've been using several Linux distributions, and writing user-space programs, for 15 years.
> I recently seized an opportunity to move into kernel development, mainly writing drivers
> for an ARM SoC, and I'm finding the transition harder than I expected.
> 
> I'm having a hard time finding good reference material to bring me up to speed on the ins
> and outs of driver development. Sure, I found Documentation/driver-model (eventually) but
> I'm still missing the high-level overview that ties everything together, and makes my
> brain click and "get it".
> 
> Many years ago, I read LDD2 and it was an eye-opener. We do have LDD3 at work, but it is
> now almost 10 years old, and it doesn't document things like the platform bus, or the "new"
> driver model, or hwmon, or cpufreq, or device tree, to name a few things on my plate.
> (I'm writing drivers for the 3.14 kernel.)
> 
> http://www.xml.com/ldd/chapter/book/
> http://www.makelinux.net/ldd3/
> 
> Are there more recent technical references, as good as LDD3, that cover "modern" aspects
> of kernel development?

The LDD3 is one of the best there is. A fourth edition is suppose to come out sometime
next year. 
http://www.amazon.com/Linux-Device-Drivers-Jessica-McKellar/dp/1449371612

> 
> I've read good things about:
> 
> "Writing Linux Device Drivers: a guide with exercises"
> by Jerry Cooperstein (2009)
> originally written for 2.6.31, updated for 3.3
> http://www.coopj.com/
> 
> What do kernel devs recommend as the worthy successor to LDD3?
> 
> 
> To wrap up this (long) message, I'd like to post some of my questions, to highlight some
> of my confusion. I'm writing a driver for a temperature sensor, which is supposed to work
> within the hwmon/lm-sensors framework.
> 
> The sensor's API consists of 3 memory-mapped registers, which are accessible over the
> SoC's memory bus. I first wrote a user-space proof-of-concept to interact with the
> sensors, accessing them through /dev/mem O_SYNC + mmap. That works as expected, and
> I now have the code to read the temperature when needed.
> 
> Next, I looked at the driver glue required to use these sensors from lm-sensors.
> That's where my troubles started.
> 
> 1) Which bus should I be using for this driver? Is the platform bus appropriate?
> 
> 2) platform.txt states
> 
>> Some drivers are not fully converted to the driver model, because they take
>> on a non-driver role:  the driver registers its platform device, rather than
>> leaving that for system infrastructure.  Such drivers can't be hotplugged
>> or coldplugged, since those mechanisms require device creation to be in a
>> different system component than the driver.
> 
> How do I "leave device registration for the system infrastructure"?
> Where should I put that code?
> Is it a good idea to separate device registration and driver registration
> in the case of a SoC, where the device is embedded in the SoC and is not
> "hot-plugged" (or anything-plugged for that matter, it's just "there").

For understanding this you need to understand how the platform infrastructure
works and why is it used.
http://lwn.net/Articles/448499/
http://lwn.net/Articles/448502/
and go through some of the relevant drivers. 
> 
> 3) Why is the function used to "destroy a platform device" named
> platform_device_put? Why put?
> Put on a list of things to destroy at a later time?

This should be more clear once you go through LDD3.

> 
> 4) Can I use platform_driver_probe, instead of platform_driver_register?
> The comments in platform.c state:
>> /**
>>  * platform_driver_probe - register driver for non-hotpluggable device
>>  * @drv: platform driver structure
>>  * @probe: the driver probe routine, probably from an __init section
>>  *
>>  * Use this instead of platform_driver_register() when you know the device
>>  * is not hotpluggable and has already been registered, and you want to
>>  * remove its run-once probe() infrastructure from memory after the driver
>>  * has bound to the device.
>>  *
>>  * One typical use for this would be with drivers for controllers integrated
>>  * into system-on-chip processors, where the controller devices have been
>>  * configured as part of board setup.
>>  *
>>  * Note that this is incompatible with deferred probing.
>>  *
>>  * Returns zero if the driver registered and bound to a device, else returns
>>  * a negative error code and with the driver not registered.
>>  */
> 
> AFAICT, I need to answer question 2 before I can answer this one.
> How do I get "devices have been configured as part of board setup."?

This actually depends on the kernel you are using. Do you have relatively
new kernel or an old one? Depending on that, either you will get that
information in a board file or else in the device tree in arch/arm/boot/dts.
> 
> 5) Very hwmon-specific, how often are temperature sensors probed?
> Does the frequency vary? (Through /sys knobs? Or something else?)
> 
> 
> That's all I have on my mind at the moment (well, aside from some ARM TLB
> shenanigans, but that's another story for another time).
> 
> I shall be eternally grateful to anyone who can enlighten me through advice,
> pointers, references, documentation, voodoo, etc.

This depends. If you have driver compiled in with the kernel this will happen
on kernel boot up. If you have the driver as a loadable module, the probe hook
will be called on modprobe or insmod.

My blog: http://coherentmusings.wordpress.com/

Some practical stuff on there which might be useful to you.

--Regards,
Sanchayan.

> 
> Regards.
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Looking for good references for ARM driver development

[Mason]

On 19/11/2014 17:57, Victor Ascroft wrote:

> This actually depends on the kernel you are using. Do you have relatively
> new kernel or an old one? Depending on that, either you will get that
> information in a board file or else in the device tree in arch/arm/boot/dts.

I'll reply more thoroughly later, but I wanted to address this question.
We're targeting kernel 3.14.x

> This depends. If you have driver compiled in with the kernel this will happen
> on kernel boot up. If you have the driver as a loadable module, the probe hook
> will be called on modprobe or insmod.

Is this an answer to question 5?
I think there is some confusion with the "probe" terminology.

I meant "probe" as in "read the current value".
I think you mean "probe" as in "the driver probes for the device through
the probe method."

I meant to ask how often the hwmon framework "polls" the temperature sensor.

Regards.

Re: Looking for good references for ARM driver development

[Victor Ascroft]

On 11/19/2014 10:49 PM, Mason wrote:
> On 19/11/2014 17:57, Victor Ascroft wrote:
> 
>> This actually depends on the kernel you are using. Do you have relatively
>> new kernel or an old one? Depending on that, either you will get that
>> information in a board file or else in the device tree in arch/arm/boot/dts.
> 
> I'll reply more thoroughly later, but I wanted to address this question.
> We're targeting kernel 3.14.x

OK. Has to be a device tree then.

> 
>> This depends. If you have driver compiled in with the kernel this will happen
>> on kernel boot up. If you have the driver as a loadable module, the probe hook
>> will be called on modprobe or insmod.
> 
> Is this an answer to question 5?
> I think there is some confusion with the "probe" terminology.
> 
> I meant "probe" as in "read the current value".
> I think you mean "probe" as in "the driver probes for the device through
> the probe method."
> 
> I meant to ask how often the hwmon framework "polls" the temperature sensor.

Oh. My understanding was you meant the "probe" of the driver. The hwmon
will be exposing the functionality of reading the temperature sensor through
a sysfs interface. I am not sure as to whether a "poll" functionality is
provided by hwmon (probably not), but, when you cat or read the sysfs path 
you will have the temperature readout. The lm75.c is good place to start.

http://lxr.free-electrons.com/source/Documentation/hwmon/
http://lxr.free-electrons.com/source/Documentation/hwmon/sysfs-interface

-- Sanchayan
> 
> Regards.
> 

Re: Looking for good references for ARM driver development

[Viresh Kumar]

On Wed, Nov 19, 2014 at 9:35 PM, Mason <mpeg.blue@free.fr> wrote:
> Is there an exhaustive list of available buses (on the ARM platform)
> and an overview of when/where each one is appropriate?

Not sure if its mentioned that clearly anywhere. BUT bus is normally
bound by the way you need to access registers of a device.

For example, if a device's registers are accessible via SPI, then it has
to register on SPI. Similarly for I2C, USB, UART, PCI, etc..

The only special cases are Platform and AMBA. Both these are for
the devices which are directly accessible from Soc bus.

AMBA is specifically for the IPs designed by ARM. All these IPs have
their vendor/version information at a predefined offset and so the AMBA
bus core reads that value to make device/driver bindings. i.e. It doesn't
depend on matching .name string in device/driver structures.

Whereas platform is for all other devices accessible over Soc bus. Device
and drivers are matches here based on the .name string as their is no
ID to read from hardware.

Re: Looking for good references for ARM driver development

[Antony Pavlov]

On Wed, 19 Nov 2014 17:05:00 +0100
Mason <mpeg.blue@free.fr> wrote:

> Hello Andreas,
> 
> On 19/11/2014 16:02, Andreas Färber wrote:
> 
> > Am 19.11.2014 um 13:50 schrieb Mason:
> >
...
> > Since this appears to be about an ARM SoC according to your To list,
> > in general, you create a device tree binding, that binding is
> > registered within your platform/... driver code and referenced in the
> > device tree for SoC or board, and then your driver will automatically
> > be probed.
> 
> I know nothing about DT (aside from the Wikipedia entry).
> I'll take a closer look at Documentation/devicetree.
> Will that explain what platform/... is?
> 
> I see a drivers/platform folder, but nothing ARM-specific there?

There are very good presentations from Free Electrons:

  * http://free-electrons.com/pub/conferences/2013/elce/petazzoni-device-tree-dummies/petazzoni-device-tree-dummies.pdf
     ** see also http://www.youtube.com/watch?v=m_NyYEBxfn8
  * https://archive.fosdem.org/2013/schedule/event/arm_in_the_linux_kernel/attachments/slides/273/export/events/attachments/arm_in_the_linux_kernel/slides/273/arm_support_kernel.pdf

-- 
Best regards,
  Antony Pavlov

Re: Looking for good references for ARM driver development

[Mason]

On 19/11/2014 17:57, Victor Ascroft wrote:

> On 11/19/2014 06:20 PM, Mason wrote:
> 
>> Are there more recent technical references, as good as LDD3, that
>> cover "modern" aspects of kernel development?
> 
> The LDD3 is one of the best there is. A fourth edition is supposed
> to come out sometime next year.
> http://www.amazon.com/Linux-Device-Drivers-Jessica-McKellar/dp/1449371612

That's the best news I've heard this week!
Hope Jonathan gets well and lives happily ever after.

>> How do I "leave device registration for the system infrastructure"?
>> Where should I put that code?
>> Is it a good idea to separate device registration and driver registration
>> in the case of a SoC, where the device is embedded in the SoC and is not
>> "hot-plugged" (or anything-plugged for that matter, it's just "there").
> 
> For understanding this you need to understand how the platform infrastructure
> works and why is it used.
> http://lwn.net/Articles/448499/
> http://lwn.net/Articles/448502/
> and go through some of the relevant drivers.

More of Jon's (great) work ;-)
Will definitely read these articles carefully.

>> 3) Why is the function used to "destroy a platform device" named
>> platform_device_put? Why put?
>> Put on a list of things to destroy at a later time?
> 
> This should be more clear once you go through LDD3.

I'm coming up empty.
AFAICT, it's unrelated to the _put_user* family of functions.
I see that it has something to do with ref count decrement.
[more reading...]
Something about kref.
https://lwn.net/Articles/336224/

They mention "put" but not *why* it's called "put".
Is it related to Dijkstra's P and V for semaphores?
https://cs.nyu.edu/~yap/classes/os/resources/origin_of_PV.html
Still confused...

Regards.

Re: Looking for good references for ARM driver development

[Victor Ascroft]

On Friday 21 November 2014 03:11 AM, Mason wrote:
> On 19/11/2014 17:57, Victor Ascroft wrote:
>
>> On 11/19/2014 06:20 PM, Mason wrote:
>>
>>> Are there more recent technical references, as good as LDD3, that
>>> cover "modern" aspects of kernel development?
>> The LDD3 is one of the best there is. A fourth edition is supposed
>> to come out sometime next year.
>> http://www.amazon.com/Linux-Device-Drivers-Jessica-McKellar/dp/1449371612
> That's the best news I've heard this week!
> Hope Jonathan gets well and lives happily ever after.
>
>>> How do I "leave device registration for the system infrastructure"?
>>> Where should I put that code?
>>> Is it a good idea to separate device registration and driver registration
>>> in the case of a SoC, where the device is embedded in the SoC and is not
>>> "hot-plugged" (or anything-plugged for that matter, it's just "there").
>> For understanding this you need to understand how the platform infrastructure
>> works and why is it used.
>> http://lwn.net/Articles/448499/
>> http://lwn.net/Articles/448502/
>> and go through some of the relevant drivers.
> More of Jon's (great) work ;-)
> Will definitely read these articles carefully.
>
>>> 3) Why is the function used to "destroy a platform device" named
>>> platform_device_put? Why put?
>>> Put on a list of things to destroy at a later time?
>> This should be more clear once you go through LDD3.
> I'm coming up empty.
> AFAICT, it's unrelated to the _put_user* family of functions.
> I see that it has something to do with ref count decrement.
> [more reading...]
> Something about kref.
> https://lwn.net/Articles/336224/
>
> They mention "put" but not *why* it's called "put".
> Is it related to Dijkstra's P and V for semaphores?
> https://cs.nyu.edu/~yap/classes/os/resources/origin_of_PV.html
> Still confused...
It has been quiet a while since I read LDD3, but, I remember this information
being somewhere in it or may be I am wrong. Not aware if these are related to
the P and V concept for semaphores.

Hmm.. You surprised me, I was not aware of put_user etc. myself. Never seen
them anywhere. I have only used and seen copy_(to/from)_user or may be it's
just that I have not worked, where these are used. I guess these _put_user family
of functions, do not carry out any kind of verification.

Yes, you are right. The platform_device_put has got to with the reference count.
platform_device_put in turn calls put_device and which in turn calls kobject_put.
The reference count for the kobject is decremented. When the reference count
reaches 0 for the kobject, kobject_cleanup is called to free the resources.

You will find good explanation of all this, the Chapter 14 Linux Device Model  of
LDD3. That is one of the best chapters to read in that book.

-Regards

>
> Regards.