[BRAINSTORM] Controls, matrices and properties

[BRAINSTORM] Controls, matrices and properties

[Hans Verkuil]

Hi all,

I have been working on support for passing matrices to/from drivers using a
new matrix API. See this earlier thread for more background information:

http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/66200

The basic feedback is that, yes, matrices are useful, but it is yet another
control-like API.

My problem with using the control API for things like this is that the control
API has been designed for use with GUIs: e.g. the controls are elements that
the end-user can modify through a GUI. Things like a matrix or some really
low-level driver property are either hard to model in a GUI or too advanced
and obscure for an end-user.

On the other hand, the control framework has all the desirable properties that
you would want: atomicity (as far as is allowed by the hardware), the ability
to get/set multiple controls in one ioctl, efficient, inheritance of subdev
controls in bridge devices, events, etc.

I'm wondering whether we cannot tweak the control API a bit to make it possible
to use it for matrices and general 'properties' as well. The main requirement
for me is that when applications enumerate over controls such properties should
never turn up in the enumerations: only controls suitable for a GUI should
appear. After all, an application would have no idea what to do with a matrix
of e.g. 200x300 elements.

While it is possible to extend queryctrl to e.g. enumerate only properties
instead of controls, it is probably better to create a new VIDIOC_QUERYPROP
ioctl. Also because the v4l2_queryctrl is pretty full and has no support to set
the minimum/maximum values of a 64 bit value. In addition, the name field is not
needed for a property, I think. Currently the name is there for the GUI, not
for identification purposes.

For setting/getting controls the existing extended control API can be used,
although I would be inclined to go for VIDIOC_G/S/TRY_PROPS ioctls as well.
For example, when I set a matrix property it is very desirable to pass only
a subset of the matrix along instead of a full matrix. In my original matrix
proposal I had a v4l2_rect struct that defined that. But there is no space
in struct v4l2_ext_control to store such information.

In general, implementing properties requires more variation since the GUI
restriction has been lifted. Also, properties can be assigned to specific
internal objects (e.g. buffer specific properties), so you need fields to
tell the kernel with which object the property is associated.

However, although the public API is different from the control API, there
is no reason not to use the internal control framework for both.

Internally controls and properties work pretty much the same way and can all
be handled by the control framework. Only supporting e.g. per-buffer controls
would take work since that is currently not implemented.

At the moment this is just an idea and I don't want to spend time on creating
a detailed RFC if people don't like it. So comments are welcome!

Regards,

	Hans

Re: [BRAINSTORM] Controls, matrices and properties

[Sakari Ailus]

Hi Hans,

Hans Verkuil wrote:
> Hi all,
>
> I have been working on support for passing matrices to/from drivers using a
> new matrix API. See this earlier thread for more background information:
>
> http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/66200
>
> The basic feedback is that, yes, matrices are useful, but it is yet another
> control-like API.
>
> My problem with using the control API for things like this is that the control
> API has been designed for use with GUIs: e.g. the controls are elements that
> the end-user can modify through a GUI. Things like a matrix or some really
> low-level driver property are either hard to model in a GUI or too advanced
> and obscure for an end-user.

We also have a lot of low level controls.

GUI implementations can always choose not to show matrix controls. I 
think a low level control flag has been proposed earlier on, but AFAIR 
the conclusion that time around was that it's sometimes difficult to 
define what is actually a low level control and what isn't.

IMHO (and according to Unix principles, too), APIs should provide means, 
not policy. Saying that controls should be used for something that can 
(or should) be displayed by a GUI, and what isn't displayed in a GUI 
isn't a control, definitely falls into this category.

> On the other hand, the control framework has all the desirable properties that
> you would want: atomicity (as far as is allowed by the hardware), the ability
> to get/set multiple controls in one ioctl, efficient, inheritance of subdev
> controls in bridge devices, events, etc.
>
> I'm wondering whether we cannot tweak the control API a bit to make it possible
> to use it for matrices and general 'properties' as well. The main requirement
> for me is that when applications enumerate over controls such properties should
> never turn up in the enumerations: only controls suitable for a GUI should
> appear. After all, an application would have no idea what to do with a matrix
> of e.g. 200x300 elements.

This sounds like the low-level control flag. I'm certainly not against 
it. I have to admit I'm not someone who'd ever access controls through a 
GUI, and if it helps, then why not.

Alternatively... how about just ignoring control types that aren't 
supported in GUI? That'd be probably even easier for GUIs than checking 
a flag --- just ignore what you don't know about.

> While it is possible to extend queryctrl to e.g. enumerate only properties
> instead of controls, it is probably better to create a new VIDIOC_QUERYPROP
> ioctl. Also because the v4l2_queryctrl is pretty full and has no support to set
> the minimum/maximum values of a 64 bit value. In addition, the name field is not
> needed for a property, I think. Currently the name is there for the GUI, not
> for identification purposes.

The are drivers that use private controls the ID of which is only 
defined as a macro in the driver. I wonder if user space programs expect 
controls under certain names.

Alternatively we could require that macro definitions exists for all new 
controls.

Would you intend VIDIOC_QUERYPROP to replace VIDIOC_QUERYCTRL or not? I 
might just create an extended version of QUERYCTRL which would fix the 
limits for 64-bit controls, too... it'd be easy to add a wrapper in 
v4l2-ioctl.c to implement the original VIDIOV_QUERYCTRL for drivers that 
implement the extended version (like we've done a bunch of time already).

> For setting/getting controls the existing extended control API can be used,
> although I would be inclined to go for VIDIOC_G/S/TRY_PROPS ioctls as well.
> For example, when I set a matrix property it is very desirable to pass only
> a subset of the matrix along instead of a full matrix. In my original matrix
> proposal I had a v4l2_rect struct that defined that. But there is no space
> in struct v4l2_ext_control to store such information.

Doe you have a use case for this?

An unrelated thing, which is out of scope for now, but something to 
think about: when passing around large amounts of (configuration) data 
the number of times the data is copied really counts. Especially on 
embedded systems.

Memory mapping helps avoiding problems --- what would happen is that the 
driver would access memory mapped to the device directly and the driver 
would then get the address to pass to the device as the configuration. 
Like video buffers, but for control, not data.

This requires a new RFC (one or more). For later, definitely.

> In general, implementing properties requires more variation since the GUI
> restriction has been lifted. Also, properties can be assigned to specific
> internal objects (e.g. buffer specific properties), so you need fields to
> tell the kernel with which object the property is associated.

Interesting idea. Definitely.

> However, although the public API is different from the control API, there
> is no reason not to use the internal control framework for both.

This could be extended ^ 2 controls. For existing controls the scope 
would always be either the video node or the subdev node.

What do you think? I think that functionality-wise it'd be a superset, 
so implementing the existing extended controls could be done as a 
backwards compatibility measure, both for drivers and applications.

> Internally controls and properties work pretty much the same way and can all
> be handled by the control framework. Only supporting e.g. per-buffer controls
> would take work since that is currently not implemented.
>
> At the moment this is just an idea and I don't want to spend time on creating
> a detailed RFC if people don't like it. So comments are welcome!

I definitely like these ideas, and I wrote down some of my own above. 
:-) I think that at this point there are so many ideas on how to improve 
what we currently have, and your matrix RFC answered to a subset of 
these aspirations.

I think we can do more without introducing a bunch of new IOCTLs every 
time a bit of new functionality is introduced, at least it without being 
a full superset of something that exists.

-- 
Kind regards,

Sakari Ailus
sakari.ailus@iki.fi

Re: [BRAINSTORM] Controls, matrices and properties

[Hans Verkuil]

Hi Sakari,

Thanks for looking at this! Much appreciated.

On Wed 10 July 2013 23:30:21 Sakari Ailus wrote:
> Hi Hans,
> 
> Hans Verkuil wrote:
> > Hi all,
> >
> > I have been working on support for passing matrices to/from drivers using a
> > new matrix API. See this earlier thread for more background information:
> >
> > http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/66200
> >
> > The basic feedback is that, yes, matrices are useful, but it is yet another
> > control-like API.
> >
> > My problem with using the control API for things like this is that the control
> > API has been designed for use with GUIs: e.g. the controls are elements that
> > the end-user can modify through a GUI. Things like a matrix or some really
> > low-level driver property are either hard to model in a GUI or too advanced
> > and obscure for an end-user.
> 
> We also have a lot of low level controls.
> 
> GUI implementations can always choose not to show matrix controls. I 
> think a low level control flag has been proposed earlier on, but AFAIR 
> the conclusion that time around was that it's sometimes difficult to 
> define what is actually a low level control and what isn't.

That's correct. Another reason was that existing applications don't check for
such a flag, so they will show everything anyway.

> IMHO (and according to Unix principles, too), APIs should provide means, 
> not policy. Saying that controls should be used for something that can 
> (or should) be displayed by a GUI, and what isn't displayed in a GUI 
> isn't a control, definitely falls into this category.

Well, yes, but tell that to the original API designers :-)

It is what it is, and the whole control API *has* been designed with a GUI
in mind.

> > On the other hand, the control framework has all the desirable properties that
> > you would want: atomicity (as far as is allowed by the hardware), the ability
> > to get/set multiple controls in one ioctl, efficient, inheritance of subdev
> > controls in bridge devices, events, etc.
> >
> > I'm wondering whether we cannot tweak the control API a bit to make it possible
> > to use it for matrices and general 'properties' as well. The main requirement
> > for me is that when applications enumerate over controls such properties should
> > never turn up in the enumerations: only controls suitable for a GUI should
> > appear. After all, an application would have no idea what to do with a matrix
> > of e.g. 200x300 elements.
> 
> This sounds like the low-level control flag. I'm certainly not against 
> it. I have to admit I'm not someone who'd ever access controls through a 
> GUI, and if it helps, then why not.
> 
> Alternatively... how about just ignoring control types that aren't 
> supported in GUI? That'd be probably even easier for GUIs than checking 
> a flag --- just ignore what you don't know about.

It runs into the same problems: properties that you don't want to show up
in a GUI and that have a valid control type would then appear.

> > While it is possible to extend queryctrl to e.g. enumerate only properties
> > instead of controls, it is probably better to create a new VIDIOC_QUERYPROP
> > ioctl. Also because the v4l2_queryctrl is pretty full and has no support to set
> > the minimum/maximum values of a 64 bit value. In addition, the name field is not
> > needed for a property, I think. Currently the name is there for the GUI, not
> > for identification purposes.
> 
> The are drivers that use private controls the ID of which is only 
> defined as a macro in the driver. I wonder if user space programs expect 
> controls under certain names.

Occasionally, yes. But in that case the control ID has to be available in a
public header.

> Alternatively we could require that macro definitions exists for all new 
> controls.
> 
> Would you intend VIDIOC_QUERYPROP to replace VIDIOC_QUERYCTRL or not? I 
> might just create an extended version of QUERYCTRL which would fix the 
> limits for 64-bit controls, too... it'd be easy to add a wrapper in 
> v4l2-ioctl.c to implement the original VIDIOV_QUERYCTRL for drivers that 
> implement the extended version (like we've done a bunch of time already).

I haven't decided yet.

> > For setting/getting controls the existing extended control API can be used,
> > although I would be inclined to go for VIDIOC_G/S/TRY_PROPS ioctls as well.
> > For example, when I set a matrix property it is very desirable to pass only
> > a subset of the matrix along instead of a full matrix. In my original matrix
> > proposal I had a v4l2_rect struct that defined that. But there is no space
> > in struct v4l2_ext_control to store such information.
> 
> Doe you have a use case for this?

If you have a large matrix (e.g. motion detection sensitivity for 16x16 pixel
squares in an image) and you only want to update a subset, then this is actually
quite useful. No need to completely exchange the full matrix if you only are
interested in specific parts.

> An unrelated thing, which is out of scope for now, but something to 
> think about: when passing around large amounts of (configuration) data 
> the number of times the data is copied really counts. Especially on 
> embedded systems.
> 
> Memory mapping helps avoiding problems --- what would happen is that the 
> driver would access memory mapped to the device directly and the driver 
> would then get the address to pass to the device as the configuration. 
> Like video buffers, but for control, not data.
> 
> This requires a new RFC (one or more). For later, definitely.
> 
> > In general, implementing properties requires more variation since the GUI
> > restriction has been lifted. Also, properties can be assigned to specific
> > internal objects (e.g. buffer specific properties), so you need fields to
> > tell the kernel with which object the property is associated.
> 
> Interesting idea. Definitely.
> 
> > However, although the public API is different from the control API, there
> > is no reason not to use the internal control framework for both.
> 
> This could be extended ^ 2 controls. For existing controls the scope 
> would always be either the video node or the subdev node.

I'm not sure about this. It might work, though. As long as the existing
control API ignores any controls not part of the video or subdev node scope,
that I see no problems with it.

It might be an interesting method to associate controls/properties to e.g.
a particular buffer (like exposure, flash) that become activated when the
buffer is activated by the DMA engine.

> What do you think? I think that functionality-wise it'd be a superset, 
> so implementing the existing extended controls could be done as a 
> backwards compatibility measure, both for drivers and applications.

I probably agree :-)

Some details would have to be worked out, but I don't see any technical
reason at least why it couldn't be done.

> > Internally controls and properties work pretty much the same way and can all
> > be handled by the control framework. Only supporting e.g. per-buffer controls
> > would take work since that is currently not implemented.
> >
> > At the moment this is just an idea and I don't want to spend time on creating
> > a detailed RFC if people don't like it. So comments are welcome!
> 
> I definitely like these ideas, and I wrote down some of my own above. 
> :-) I think that at this point there are so many ideas on how to improve 
> what we currently have, and your matrix RFC answered to a subset of 
> these aspirations.
> 
> I think we can do more without introducing a bunch of new IOCTLs every 
> time a bit of new functionality is introduced, at least it without being 
> a full superset of something that exists.

Right.

If possible I would like to discuss this during the New Orleans conference.
I know Laurent and Mauro will be there.

Regards,

	Hans

Re: [BRAINSTORM] Controls, matrices and properties

[Sakari Ailus]

Hans Verkuil wrote:
> If possible I would like to discuss this during the New Orleans conference.
> I know Laurent and Mauro will be there.

I won't be able to attend that one, but I'll be in Edinburgh in October.

-- 
Sakari Ailus
sakari.ailus@iki.fi

Re: [BRAINSTORM] Controls, matrices and properties

[Laurent Pinchart]

Hi Hans,

Thank you for the proposal.

On Monday 08 July 2013 13:06:56 Hans Verkuil wrote:
> Hi all,
> 
> I have been working on support for passing matrices to/from drivers using a
> new matrix API. See this earlier thread for more background information:
> 
> http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/662
> 00
> 
> The basic feedback is that, yes, matrices are useful, but it is yet another
> control-like API.
> 
> My problem with using the control API for things like this is that the
> control API has been designed for use with GUIs: e.g. the controls are
> elements that the end-user can modify through a GUI. Things like a matrix or
> some really low-level driver property are either hard to model in a GUI or
> too advanced and obscure for an end-user.

That's true for some controls as well. I think the controls API has evolved 
over time to accommodate the needs of both user-facing, GUI-friendly controls 
and low-level controls (especially in the embedded world). The controls APIs 
(both the userspace API and the in-kernel control framework) were suitable for 
both (after a bit of tweaking in some cases), so we just haven't realized how 
conceptually different controls were. There's not much more in common between 
a brightness level and a flash strobe time then between a contrast level and a 
gamma table or motion detection matrix.

> On the other hand, the control framework has all the desirable properties
> that you would want: atomicity (as far as is allowed by the hardware), the
> ability to get/set multiple controls in one ioctl, efficient, inheritance
> of subdev controls in bridge devices, events, etc.
> 
> I'm wondering whether we cannot tweak the control API a bit to make it
> possible to use it for matrices and general 'properties' as well. The main
> requirement for me is that when applications enumerate over controls such
> properties should never turn up in the enumerations: only controls suitable
> for a GUI should appear. After all, an application would have no idea what
> to do with a matrix of e.g. 200x300 elements.
> 
> While it is possible to extend queryctrl to e.g. enumerate only properties
> instead of controls, it is probably better to create a new VIDIOC_QUERYPROP
> ioctl. Also because the v4l2_queryctrl is pretty full and has no support to
> set the minimum/maximum values of a 64 bit value. In addition, the name
> field is not needed for a property, I think. Currently the name is there
> for the GUI, not for identification purposes.

Names indeed feel a bit out of place. Even for GUI-related controls the 
controls API doesn't support localization (not that it should!), so handling 
control names in a central location in userspace would make sense (names can 
still be provided by the kernel as hints though, to simplify basic command 
line applications).

Moreover, I've never been really convinced by the GUI-related flags we 
currently have in the controls API. They feel like a policy decision to me, 
and should ideally (at least in my ideal view of V4L2) be part of userspace. 
If an application wants to render controls as knobs instead of sliders I don't 
see a reason why the kernel should hint otherwise. Maybe we could rethink, 
while developing the properties API, how userspace should use properties 
globally, from a userspace point of view. What is it that intrinsically make 
some of the controls we have today be displayed as sliders for instance ? Is 
it really a property of the control ? What influences our idea of how controls 
should be rendered ?

At the end of the day, whether we should present a control to a user will 
depend on the user and his/her use cases. There's no one-size-fits-them-all 
rule to decide whether all individual controls should or should not be 
displayed in a panel application. I haven't really thought this topic through, 
but a binary decision made in the kernel doesn't look like the right solution 
to me. As Sakari noted, user-facing panel applications could just whitelist 
controls they want to display. This would work for the most simple cases, but 
would require updating all those userspace applications when we add a user-
facing control, which adds an additional burden to developers and make 
extensions slower to roll out. One could however argue that such controls are 
very rarely added. I have no clear answer to this question.

> For setting/getting controls the existing extended control API can be used,
> although I would be inclined to go for VIDIOC_G/S/TRY_PROPS ioctls as well.
> For example, when I set a matrix property it is very desirable to pass only
> a subset of the matrix along instead of a full matrix. In my original matrix
> proposal I had a v4l2_rect struct that defined that. But there is no space
> in struct v4l2_ext_control to store such information.
> 
> In general, implementing properties requires more variation since the GUI
> restriction has been lifted. Also, properties can be assigned to specific
> internal objects (e.g. buffer specific properties), so you need fields to
> tell the kernel with which object the property is associated.
> 
> However, although the public API is different from the control API, there
> is no reason not to use the internal control framework for both.
> 
> Internally controls and properties work pretty much the same way and can all
> be handled by the control framework. Only supporting e.g. per-buffer
> controls would take work since that is currently not implemented.

We'll probably find out along the way that we'll need more extensions to the 
control framework, but I'm not worried about that. We have a solid code base, 
and it's a pure in-kernel API.

> At the moment this is just an idea and I don't want to spend time on
> creating a detailed RFC if people don't like it. So comments are welcome!

Please count me in. I'm definitely in favour of discussing the idea further. 
Face-to-face discussions are needed, but we should first brainstorm ideas on 
the list to let them sink through in everyone's mind at least for a couple of 
weeks before a meeting.

-- 
Regards,

Laurent Pinchart

Re: [BRAINSTORM] Controls, matrices and properties

[Laurent Pinchart]

Hi Sakari,

Just a small question below, I've addressed all the rest in a reply to Hans' 
original e-mail.

On Thursday 11 July 2013 00:30:21 Sakari Ailus wrote:
> Hans Verkuil wrote:
> > Hi all,
> > 
> > I have been working on support for passing matrices to/from drivers using
> > a new matrix API. See this earlier thread for more background information:
> > 
> > http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/6
> > 6200
> > 
> > The basic feedback is that, yes, matrices are useful, but it is yet
> > another control-like API.
> > 
> > My problem with using the control API for things like this is that the
> > control API has been designed for use with GUIs: e.g. the controls are
> > elements that the end-user can modify through a GUI. Things like a matrix
> > or some really low-level driver property are either hard to model in a
> > GUI or too advanced and obscure for an end-user.
> 
> We also have a lot of low level controls.
> 
> GUI implementations can always choose not to show matrix controls. I
> think a low level control flag has been proposed earlier on, but AFAIR
> the conclusion that time around was that it's sometimes difficult to
> define what is actually a low level control and what isn't.
> 
> IMHO (and according to Unix principles, too), APIs should provide means,
> not policy. Saying that controls should be used for something that can
> (or should) be displayed by a GUI, and what isn't displayed in a GUI
> isn't a control, definitely falls into this category.
> 
> > On the other hand, the control framework has all the desirable properties
> > that you would want: atomicity (as far as is allowed by the hardware),
> > the ability to get/set multiple controls in one ioctl, efficient,
> > inheritance of subdev controls in bridge devices, events, etc.
> > 
> > I'm wondering whether we cannot tweak the control API a bit to make it
> > possible to use it for matrices and general 'properties' as well. The
> > main requirement for me is that when applications enumerate over controls
> > such properties should never turn up in the enumerations: only controls
> > suitable for a GUI should appear. After all, an application would have no
> > idea what to do with a matrix of e.g. 200x300 elements.
> 
> This sounds like the low-level control flag. I'm certainly not against
> it. I have to admit I'm not someone who'd ever access controls through a
> GUI, and if it helps, then why not.
> 
> Alternatively... how about just ignoring control types that aren't
> supported in GUI? That'd be probably even easier for GUIs than checking
> a flag --- just ignore what you don't know about.
> 
> > While it is possible to extend queryctrl to e.g. enumerate only properties
> > instead of controls, it is probably better to create a new
> > VIDIOC_QUERYPROP ioctl. Also because the v4l2_queryctrl is pretty full and
> > has no support to set the minimum/maximum values of a 64 bit value. In
> > addition, the name field is not needed for a property, I think. Currently
> > the name is there for the GUI, not for identification purposes.
> 
> The are drivers that use private controls the ID of which is only
> defined as a macro in the driver. I wonder if user space programs expect
> controls under certain names.
> 
> Alternatively we could require that macro definitions exists for all new
> controls.
> 
> Would you intend VIDIOC_QUERYPROP to replace VIDIOC_QUERYCTRL or not? I
> might just create an extended version of QUERYCTRL which would fix the
> limits for 64-bit controls, too... it'd be easy to add a wrapper in
> v4l2-ioctl.c to implement the original VIDIOV_QUERYCTRL for drivers that
> implement the extended version (like we've done a bunch of time already).
> 
> > For setting/getting controls the existing extended control API can be
> > used, although I would be inclined to go for VIDIOC_G/S/TRY_PROPS ioctls
> > as well. For example, when I set a matrix property it is very desirable to
> > pass only a subset of the matrix along instead of a full matrix. In my
> > original matrix proposal I had a v4l2_rect struct that defined that. But
> > there is no space in struct v4l2_ext_control to store such information.
> 
> Doe you have a use case for this?
> 
> An unrelated thing, which is out of scope for now, but something to think
> about: when passing around large amounts of (configuration) data the number
> of times the data is copied really counts. Especially on embedded systems.
> 
> Memory mapping helps avoiding problems --- what would happen is that the
> driver would access memory mapped to the device directly and the driver
> would then get the address to pass to the device as the configuration. Like
> video buffers, but for control, not data.

Would you map that memory to userspace as well ?

> This requires a new RFC (one or more). For later, definitely.
> 
> > In general, implementing properties requires more variation since the GUI
> > restriction has been lifted. Also, properties can be assigned to specific
> > internal objects (e.g. buffer specific properties), so you need fields to
> > tell the kernel with which object the property is associated.
> 
> Interesting idea. Definitely.
> 
> > However, although the public API is different from the control API, there
> > is no reason not to use the internal control framework for both.
> 
> This could be extended ^ 2 controls. For existing controls the scope
> would always be either the video node or the subdev node.
> 
> What do you think? I think that functionality-wise it'd be a superset,
> so implementing the existing extended controls could be done as a
> backwards compatibility measure, both for drivers and applications.
> 
> > Internally controls and properties work pretty much the same way and can
> > all be handled by the control framework. Only supporting e.g. per-buffer
> > controls would take work since that is currently not implemented.
> > 
> > At the moment this is just an idea and I don't want to spend time on
> > creating a detailed RFC if people don't like it. So comments are welcome!
> 
> I definitely like these ideas, and I wrote down some of my own above.
> 
> :-) I think that at this point there are so many ideas on how to improve
> 
> what we currently have, and your matrix RFC answered to a subset of
> these aspirations.
> 
> I think we can do more without introducing a bunch of new IOCTLs every
> time a bit of new functionality is introduced, at least it without being
> a full superset of something that exists.

-- 
Regards,

Laurent Pinchart

Re: [BRAINSTORM] Controls, matrices and properties

[Sakari Ailus]

Hi Laurent,

On Thu, Jul 18, 2013 at 03:07:26AM +0200, Laurent Pinchart wrote:
...
> > An unrelated thing, which is out of scope for now, but something to think
> > about: when passing around large amounts of (configuration) data the number
> > of times the data is copied really counts. Especially on embedded systems.
> > 
> > Memory mapping helps avoiding problems --- what would happen is that the
> > driver would access memory mapped to the device directly and the driver
> > would then get the address to pass to the device as the configuration. Like
> > video buffers, but for control, not data.
> 
> Would you map that memory to userspace as well ?

Yes --- that's the very intent. This way all unnecessary memory copies in
the kernel can be avoided. Say, if you're providing (and copying) the device
a new LSC table of 128 kiB for every frame while recording video, it will
show up in the energy consumption of the system.

It does let the user space to do wrong things, too. But the same is equally
true for video buffers as well.

-- 
Cheers,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk