Some thoughts on metadata and iio

["Medenblik, H.J.W. (Henk)" <henk.medenblik@tno.nl>]

Dear IIO supporters,

I have been struggling for a while with the lack of metadata support
inside the IIO framework. I really appreciate all the tremendous work
that has been done to extend the iio framework to a very useful
subsystem which easily can be used to acquire fast data streams inside
Linux.

I have been able to implement complete FPGA systems with high datarate
devices connected and store that info to Solid State Drives or transfer
that data over the network at high speeds (thanks Analog Devices).
However, what I have been missing so far is some feature which allows me
to add metadata to these streams. I was mostly focused on what is
currently available right now and tried to understand and adapt to these
thought and 'common' practices in order to see if I was something
missing how to handle this issue of using metadata inside the IIO
framework or if someone already designed that before within the framework.

I believe there are many real world application examples where somebody
need adding metadata into the 'standard' datastreams such as ADC or DAC
data samples. The metadata that I needed to have inserted was also very
timing critical and also changing in a time critical manner, which I
believe that it could only be inserted and triggered within my HDL
hardware design. So i did.....

My solution therefore was to define a customized metadata header (custom
size) generated inside my HDL design. I also decided to make the size of
this header a multiple of the elementary data sample size such that I
would not get any alignment problems when acquiring data from my
backend. Whenever I now retrieve data from my fpga backend with custom
iio device I can get all the data that I want including the meta header.

With this 'solution' I was wondering if it would make sense if the
current framework would be extended with additional iio information
describing the iio device. Currently there are so many properties
defining the samples inside iio such as if it is voltage or light or
whatever.. Why not add something which indicates if the device has
metadata support or not by defining a dedicated property for it. A
second parameter could describe the size of the metadata header. The
combination of the two parameter extensions can then be used to help the
user software to identify positions of where the real data samples start
inside the stream.

I am wondering if this makes sense.

With kind regards,

Henk


This message may contain information that is not intended for you. If you are not the addressee or if this message was sent to you by mistake, you are requested to inform the sender and delete the message. TNO accepts no liability for the content of this e-mail, for the manner in which you use it and for damage of any kind resulting from the risks inherent to the electronic transmission of messages.