Raw data block input: is IIO a good choice?

Raw data block input: is IIO a good choice?

[Pavel Roskin]

Hello,

I'm writing a driver for an FPGA device that collects information from
several sensors. The data is collected every 20 milliseconds (i.e. 50
times a second). The data is a 256 byte block that is put to a ring
buffer in the hardware. The software should read the data quickly to
avoid it being overwritten in the ring buffer. It is very important
not to lose any data; previous values are just as important as the
current ones. No parsing needs to be done in the kernel; the userspace
will take care of it.

Is IIO a good choice for such device? I see that the existing IIO
drivers are careful to describe all data they collect. I'm not going
to do that in the kernel (OK, maybe later to allow other utilities to
get the parsed data, but it can be done in userspace too).

I would really appreciate is somebody could point me to a driver that
implements similar functionality.

-- 
Regards,
Pavel Roskin

Re: Raw data block input: is IIO a good choice?

[Jonah Petri]

Hi Pavel,

I=E2=80=99m fairly new to IIO, but I am working on a somewhat similar =
project.  One important question is how you are intending to transport =
your data from the FPGA to your CPU?  Do you have a DMA engine of some =
kind, with a linux driver?  Or perhaps a wire protocol, such as SPI or =
I2C?  What=E2=80=99s your realtime requirements?  That is, if your data =
rate is 12.5k/sec, so how much buffer space do you have on the hardware? =
 Typically you would aim to drain the hardware when it is 50% full, so =
that would give you an idea of how frequently you=E2=80=99d need to =
service your hardware buffer.

Stock Linux isn=E2=80=99t going to be your friend if your realtime =
requirements are tight.  Don=E2=80=99t rely on interrupt latencies less =
than 10s of milliseconds, or even more on busy systems.

Jonah


> On Apr 7, 2017, at 5:48 PM, Pavel Roskin <plroskin@gmail.com> wrote:
>=20
> Hello,
>=20
> I'm writing a driver for an FPGA device that collects information from
> several sensors. The data is collected every 20 milliseconds (i.e. 50
> times a second). The data is a 256 byte block that is put to a ring
> buffer in the hardware. The software should read the data quickly to
> avoid it being overwritten in the ring buffer. It is very important
> not to lose any data; previous values are just as important as the
> current ones. No parsing needs to be done in the kernel; the userspace
> will take care of it.
>=20
> Is IIO a good choice for such device? I see that the existing IIO
> drivers are careful to describe all data they collect. I'm not going
> to do that in the kernel (OK, maybe later to allow other utilities to
> get the parsed data, but it can be done in userspace too).
>=20
> I would really appreciate is somebody could point me to a driver that
> implements similar functionality.
>=20
> --=20
> Regards,
> Pavel Roskin
> --
> To unsubscribe from this list: send the line "unsubscribe linux-iio" =
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Raw data block input: is IIO a good choice?

[Pavel Roskin]

Hi Jonah,

Thank you for reply! There is no DMA. The ring buffer is implemented
in FPGA. It is mapped to the CPU memory. It has space for 64 messages.
That is, if the userspace doesn't read data for 64/50 = 1.28 seconds,
there will be lost data. It is possible to set up an interrupt as a
certain watermark, e.g. when half of the buffers are utilized. But I
expect that the userspace utility will be polling the driver for new
messages and will read them before the interrupt is triggered.

I'm not sure if a buffer in the kernel space would be needed. It would
be nice to save the data in case if the userspace software is slow.
But I don't see that as a hard requirement. If the userspace utility
sleeps for over a second, I would be in trouble, with or without
kernel buffering.

I don't want to do any processing in the kernel space. So the driver
would have one "channel" spitting out 256 byte long opaque packets.

I'm sure there are tons of ways to get that data to the userspace, I
just want to select a subsystem that would cause less frustration and
"impedance mismatch".

-- 
Regards,
Pavel Roskin

Re: Raw data block input: is IIO a good choice?

[Jonah Petri]

Hi Pavel,

To me, IIO seems really great if you want to have generic, structured =
access to your sensor's data from userspace, with a single driver =
supporting a wide variety of use cases.  Your system sounds very =
bespoke, and I=E2=80=99m guessing that you probably have a single, =
tightly-coupled client for your data.

In my opinion, IIO might be a bit much for your situation.  If I were =
you, I would just implement a /dev/ node which returns the queued data =
from the FPGA on read.  Should be a simple fops table and not much else, =
I=E2=80=99d think.  Just my 2=C2=A2!

Best,
Jonah

> On Apr 7, 2017, at 9:47 PM, Pavel Roskin <plroskin@gmail.com> wrote:
>=20
> Hi Jonah,
>=20
> Thank you for reply! There is no DMA. The ring buffer is implemented
> in FPGA. It is mapped to the CPU memory. It has space for 64 messages.
> That is, if the userspace doesn't read data for 64/50 =3D 1.28 =
seconds,
> there will be lost data. It is possible to set up an interrupt as a
> certain watermark, e.g. when half of the buffers are utilized. But I
> expect that the userspace utility will be polling the driver for new
> messages and will read them before the interrupt is triggered.
>=20
> I'm not sure if a buffer in the kernel space would be needed. It would
> be nice to save the data in case if the userspace software is slow.
> But I don't see that as a hard requirement. If the userspace utility
> sleeps for over a second, I would be in trouble, with or without
> kernel buffering.
>=20
> I don't want to do any processing in the kernel space. So the driver
> would have one "channel" spitting out 256 byte long opaque packets.
>=20
> I'm sure there are tons of ways to get that data to the userspace, I
> just want to select a subsystem that would cause less frustration and
> "impedance mismatch".
>=20
> --=20
> Regards,
> Pavel Roskin

Re: Raw data block input: is IIO a good choice?

[Pavel Roskin]

Jonah, I appreciate your advice.

I'm still considering using IIO, as I would need some ways to
configure the hardware (set interrupt watermark, set some other
configuration registers). But it's good to understand why the
resulting driver would be unlike all others.

Pavel

Re: Raw data block input: is IIO a good choice?

[Jonathan Cameron]

On 08/04/17 07:13, Pavel Roskin wrote:
> Jonah, I appreciate your advice.
> 
> I'm still considering using IIO, as I would need some ways to
> configure the hardware (set interrupt watermark, set some other
> configuration registers). But it's good to understand why the
> resulting driver would be unlike all others.
The primary reason for the existence of a subsystem is to
provide a consistent userspace interface. In some ways it doesn't
matter what goes on within the kernel at all.

Now it may be possible to do this for your device, but that would
only make sense if we were to describe the data in your ring buffer.
It could be implemented as a hardware ring buffer with no software
buffer.

Whilst we fairly recently move the sca3000 driver away from this,
as in that case an intermediate software buffer made more sense,
that driver used to do this.
Jumping back an arbitrary distance:
https://git.kernel.org/pub/scm/linux/kernel/git/jic23/iio.git/tree/drivers/staging/iio/accel/sca3000_ring.c?h=iio-for-4.7a&id=486294f184c05cff116160bb731cbb679f047621
I believe there are drivers in analog devices out of mainline tree
that do it as well. One example there is the spi offload engine
support which uses the hardware buffer stuff, but that is somewhat
cryptic so the sca3000 driver above is probably a better starting
point.

Then we come to the question of how to describe the data. If you
don't do this of course you can use IIO out of mainline, but there
will be little interest in merging it.

If you describe the data using normal IIO channels it is up to you
whether you then use your own userspace stack or something more
general.  What we are interested in from a subsystem point of view
is ensuring you 'could' use standard userspace code.

So what is actually in this magic data dump?  Sensor data by the
sound of it, so it could be described?  Without that description
I'm not going to be keen to have it mainline IIO.

Jonathan

p.s. When replying to a thread, please keep all relevant previous
emails in there! Makes it a lot easier for those of us running
a bit behind to catch up.

> 
> Pavel
> --
> To unsubscribe from this list: send the line "unsubscribe linux-iio" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Raw data block input: is IIO a good choice?

[Pavel Roskin]

Hi Jonathan,

I appreciate your response.

On Sat, Apr 8, 2017 at 7:36 AM, Jonathan Cameron <jic23@kernel.org> wrote:
> On 08/04/17 07:13, Pavel Roskin wrote:
>> Jonah, I appreciate your advice.
>>
>> I'm still considering using IIO, as I would need some ways to
>> configure the hardware (set interrupt watermark, set some other
>> configuration registers). But it's good to understand why the
>> resulting driver would be unlike all others.
> The primary reason for the existence of a subsystem is to
> provide a consistent userspace interface. In some ways it doesn't
> matter what goes on within the kernel at all.
>
> Now it may be possible to do this for your device, but that would
> only make sense if we were to describe the data in your ring buffer.
> It could be implemented as a hardware ring buffer with no software
> buffer.

I can describe the data.

My concern is that I don't see way to read unparsed blocks of data.
realbits and storagebits are described by u8, which limits them to 255
bits or 63 bytes.

The main userspace consumer of the data will need whole blocks of data
captures at the same time.

> Whilst we fairly recently move the sca3000 driver away from this,
> as in that case an intermediate software buffer made more sense,
> that driver used to do this.
> Jumping back an arbitrary distance:
> https://git.kernel.org/pub/scm/linux/kernel/git/jic23/iio.git/tree/drivers/staging/iio/accel/sca3000_ring.c?h=iio-for-4.7a&id=486294f184c05cff116160bb731cbb679f047621
> I believe there are drivers in analog devices out of mainline tree
> that do it as well. One example there is the spi offload engine
> support which uses the hardware buffer stuff, but that is somewhat
> cryptic so the sca3000 driver above is probably a better starting
> point.

Thank you for the example! The driver is quite large, it will take me
a while to process.

Do you have any documentation about buffers? It's good to have
iio_simple_dummy_buffer.c, but it doesn't have a hardware buffer mode.

> Then we come to the question of how to describe the data. If you
> don't do this of course you can use IIO out of mainline, but there
> will be little interest in merging it.
>
> If you describe the data using normal IIO channels it is up to you
> whether you then use your own userspace stack or something more
> general.  What we are interested in from a subsystem point of view
> is ensuring you 'could' use standard userspace code.
>
> So what is actually in this magic data dump?  Sensor data by the
> sound of it, so it could be described?  Without that description
> I'm not going to be keen to have it mainline IIO.

Yes, it can be described. It's sensor data, nothing unusual.

> p.s. When replying to a thread, please keep all relevant previous
> emails in there! Makes it a lot easier for those of us running
> a bit behind to catch up.

My bad, I just realized gmail can do the right thing. Jonathan, please
ignore the personal reply, my email setup is new to me.

Pavel

Re: Raw data block input: is IIO a good choice?

[Pavel Roskin]

Hi Jonathan,

I tried to describe all channels, and it turns out many of them cannot
be adequately described by the types in iio_chan_type. How do I
describe the message sequence number? How about the control register
value at the time when the data was captured? That data needs to be
passed to the userspace.

I checked sca3000 driver. I see that it uses
sca3000_read_first_n_hw_rb() to read the data from the buffer. Can
that data be accessed by the userspace as is, without any processing?

Pavel

Re: Raw data block input: is IIO a good choice?

[Jonathan Cameron]

On 12 April 2017 01:05:17 BST, Pavel Roskin <plroskin@gmail.com> wrote:
>Hi Jonathan,
>
>I tried to describe all channels, and it turns out many of them cannot
>be adequately described by the types in iio_chan_type. How do I
>describe the message sequence number? 
That wouldn't normally be of interest to userspace as driver would guarantee order.

How about the control register
>value at the time when the data was captured? That data needs to be
>passed to the userspace.

Is it changing on its own? If not normally that is represented via out of band reads of sysfs.

>
>I checked sca3000 driver. I see that it uses
>sca3000_read_first_n_hw_rb() to read the data from the buffer. Can
>that data be accessed by the userspace as is, without any processing?

It is supplied raw to userspace. You can do what you like.

J
>
>Pavel

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.