Idea of a v4l -> fb interface driver

Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

This message has been earlier sent to the V4L mailing list

Linux Media Mailing List <linux-media@vger.kernel.org>

When replying, please add it to CC. This is a discussion proposal for the 
V4L mini summit, that is going to take place in June in Helsinki, Finland. 
Any opinions very welcome.

		V4L(2) video output vs. framebuffer.

Problem: Currently the standard way to provide graphical output on various 
(embedded) displays like LCDs is to use a framebuffer driver. The 
interface is well supported and widely adopted in the user-space, many 
applications, including the X-server, various libraries like directfb, 
gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a 
number of problems. It is unmaintained. The infrastructure is not being 
further developed, every specific hardware driver is being supported by 
the respective architecture community. But as video output hardware 
evolves, more complex displays and buses appear and have to be supported, 
the subsystem shows its aging. For example, there is currently no way to 
write reusable across multiple platforms display drivers.

OTOH V4L2 has a standard video output driver support, it is not very 
widely used, in the userspace I know only of gstreamer, that somehow 
supports video-output v4l2 devices in latest versions. But, being a part 
of the v4l2 subsystem, these drivers already now can take a full advantage 
of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.

So, how can we help graphics driver developers on the one hand by 
providing them with a capable driver framework (v4l2) and on the other 
hand by simplifying the task of interfacing to the user-space?

How about a v4l2-output - fbdev translation layer? You write a v4l2-output 
driver and get a framebuffer device free of charge... TBH, I haven't given 
this too much of a thought, but so far I don't see anything that would 
make this impossible in principle. The video buffer management is quite 
different between the two systems, but maybe we can teach video-output 
drivers to work with just one buffer too? Anyway, feel free to tell me why 
this is an absolutely impossible / impractical idea;)

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Jaya Kumar]

On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> Problem: Currently the standard way to provide graphical output on various
> (embedded) displays like LCDs is to use a framebuffer driver. The
> interface is well supported and widely adopted in the user-space, many
> applications, including the X-server, various libraries like directfb,
> gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
> number of problems. It is unmaintained. The infrastructure is not being
> further developed, every specific hardware driver is being supported by
> the respective architecture community. But as video output hardware

I understand the issue you are raising, but to be clear there are
several developers, Geert, Krzysztof, and others who are helping with
the role of fbdev maintainer while Tony is away. If you meant that it
has no specific currently active maintainer person, when you wrote
"unmaintained", then I agree that is correct. We're not sure where
Tony is and we hope he's okay and that he'll be back soon. But if you
meant that it is not maintained as in bugs aren't being fixed, then
I'd have to slightly disagree. Maybe not as fast as commercial
organizations seem to think should come for free, but still they are
being worked on.

> evolves, more complex displays and buses appear and have to be supported,
> the subsystem shows its aging. For example, there is currently no way to
> write reusable across multiple platforms display drivers.

At first I misread your point as talking about multi-headed displays
which you're correct is not so great in fbdev. But "write reusable
across multi-platform display driver", I did not understand fully. I
maintain a fbdev driver, broadsheetfb, that we're using on arm and x86
without problems and my presumption is other fbdev drivers are also
capable of this unless the author made it explicitly platform
specific. In my experience with adding defio to the fbdev infra, the
fbdev community seemed quite good and I did not notice any aging
problems. I realize there's probably issues that you're encountering
where fbdev might be weak, this is good, and if you raise them
specifically, I think the community can work together to address the
issues.

>
> OTOH V4L2 has a standard video output driver support, it is not very
> widely used, in the userspace I know only of gstreamer, that somehow
> supports video-output v4l2 devices in latest versions. But, being a part
> of the v4l2 subsystem, these drivers already now can take a full advantage
> of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
>
> So, how can we help graphics driver developers on the one hand by
> providing them with a capable driver framework (v4l2) and on the other
> hand by simplifying the task of interfacing to the user-space?

I think it would help if there were more specific elaborations on the
functionality that you'd want the fbdev community to improve and how
it could reuse v4l2 for this.

>
> How about a v4l2-output - fbdev translation layer? You write a v4l2-output
> driver and get a framebuffer device free of charge... TBH, I haven't given
> this too much of a thought, but so far I don't see anything that would
> make this impossible in principle. The video buffer management is quite
> different between the two systems, but maybe we can teach video-output
> drivers to work with just one buffer too? Anyway, feel free to tell me why
> this is an absolutely impossible / impractical idea;)

Sounds interesting. I think that idea seems viable but I'm not sure
we'd want every usb webcam to register an fbdev interface if that is
part of the thoughts you're having. If you could elaborate on the
benefits, that'd be great.

Thanks,
jaya

RE: Idea of a v4l -> fb interface driver

[Hiremath, Vaibhav]

> -----Original Message-----
> From: linux-fbdev-owner@vger.kernel.org [mailto:linux-fbdev-
> owner@vger.kernel.org] On Behalf Of Guennadi Liakhovetski
> Sent: Wednesday, May 26, 2010 7:40 PM
> To: linux-fbdev@vger.kernel.org
> Subject: Idea of a v4l -> fb interface driver
> 
> This message has been earlier sent to the V4L mailing list
> 
> Linux Media Mailing List <linux-media@vger.kernel.org>
I replied to Linux-Media list, but saw this thread again. So pasting my response here and adding linux-media to CC.


> 
> When replying, please add it to CC. This is a discussion proposal for the
> V4L mini summit, that is going to take place in June in Helsinki, Finland.
> Any opinions very welcome.
> 
> 		V4L(2) video output vs. framebuffer.
> 
> Problem: Currently the standard way to provide graphical output on various
> (embedded) displays like LCDs is to use a framebuffer driver. The
> interface is well supported and widely adopted in the user-space, many
> applications, including the X-server, various libraries like directfb,
> gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
> number of problems. It is unmaintained. The infrastructure is not being
> further developed, every specific hardware driver is being supported by
> the respective architecture community. But as video output hardware
> evolves, more complex displays and buses appear and have to be supported,
> the subsystem shows its aging. For example, there is currently no way to
> write reusable across multiple platforms display drivers.
> 
[Hiremath, Vaibhav] Up to certain extent yes you are correct.

> OTOH V4L2 has a standard video output driver support, it is not very
> widely used, in the userspace I know only of gstreamer, that somehow
> supports video-output v4l2 devices in latest versions. But, being a part
> of the v4l2 subsystem, these drivers already now can take a full advantage
> of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
> 
> So, how can we help graphics driver developers on the one hand by
> providing them with a capable driver framework (v4l2) and on the other
> hand by simplifying the task of interfacing to the user-space?
> 
[Hiremath, Vaibhav] I think this is really complex question which requires healthy discussion over list.

> How about a v4l2-output - fbdev translation layer? You write a v4l2-output
> driver and get a framebuffer device free of charge... TBH, I haven't given
> this too much of a thought, but so far I don't see anything that would
> make this impossible in principle. The video buffer management is quite
> different between the two systems, but maybe we can teach video-output
> drivers to work with just one buffer too? 
[Hiremath, Vaibhav] I believe V4L2 buf won't limit you to do this. Atleast in case of OMAP v4L2 display driver we are sticking to last buffer if application fails to queue one. So for me this is single buffer keeps on displaying unless application queue next buffer.

> Anyway, feel free to tell me why
> this is an absolutely impossible / impractical idea;)
> 
[Hiremath, Vaibhav] If I understanding correctly you are trying to propose something like,

Without changing Fbdev interface to user space application, create translation layers which will allow driver developer to write driver under V4L2 framework providing /dev/fbx but using V4L2 API/framework.

Also as mentioned by Jaya, it would be great if you put benefits we are targeting would be helpful.

Thanks,
Vaibhav

> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

(adding V4L ML to CC and preserving the complete reply for V4L readers)

On Thu, 27 May 2010, Jaya Kumar wrote:

> On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski
> <g.liakhovetski@gmx.de> wrote:
> > Problem: Currently the standard way to provide graphical output on various
> > (embedded) displays like LCDs is to use a framebuffer driver. The
> > interface is well supported and widely adopted in the user-space, many
> > applications, including the X-server, various libraries like directfb,
> > gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
> > number of problems. It is unmaintained. The infrastructure is not being
> > further developed, every specific hardware driver is being supported by
> > the respective architecture community. But as video output hardware
> 
> I understand the issue you are raising, but to be clear there are
> several developers, Geert, Krzysztof, and others who are helping with
> the role of fbdev maintainer while Tony is away. If you meant that it
> has no specific currently active maintainer person, when you wrote
> "unmaintained", then I agree that is correct.

Exactly, I just interpreted this excerpt from MAINTAINERS:

FRAMEBUFFER LAYER
L:	linux-fbdev@vger.kernel.org
W:	http://linux-fbdev.sourceforge.net/
S:	Orphan

> We're not sure where
> Tony is and we hope he's okay and that he'll be back soon. But if you
> meant that it is not maintained as in bugs aren't being fixed, then
> I'd have to slightly disagree. Maybe not as fast as commercial
> organizations seem to think should come for free, but still they are
> being worked on.
> 
> > evolves, more complex displays and buses appear and have to be supported,
> > the subsystem shows its aging. For example, there is currently no way to
> > write reusable across multiple platforms display drivers.
> 
> At first I misread your point as talking about multi-headed displays
> which you're correct is not so great in fbdev. But "write reusable
> across multi-platform display driver", I did not understand fully. I
> maintain a fbdev driver, broadsheetfb, that we're using on arm and x86
> without problems and my presumption is other fbdev drivers are also
> capable of this unless the author made it explicitly platform
> specific.

Ok, let me explain what exactly I meant. Above I referred to "display 
drivers," which is not the same as a "framebuffer controller driver" or 
whatever you would call it. By framebuffer controller driver I mean a 
driver for the actual graphics engine on a certain graphics card or an 
SoC. This is the part, that reads data from the actual framebuffer and 
outputs it to some hardware interface to a display device. Now this 
interface can be a VGA or a DVI connector, it can be one of several bus 
types, used with various LCD displays. In many cases this is all you have 
to do to get the output to your display. But in some cases the actual 
display on the other side of this bus also requires a driver. That can be 
some kind of a smart display, it can be a panel with an attached display 
controller, that must be at least configured, say, over SPI, it can be a 
display, attached to the host over the MIPI DSI bus, and implementing some 
proprietary commands. In each of these cases you will have to write a 
display driver for this specific display or controller type, and your 
framebuffer driver will have to interface with that display driver. Now, 
obviously, those displays can be connected to a variety of host systems, 
in which case you will want to reuse that display driver. This means, 
there has to be a standard fb-driver - display-driver API. AFAICS, this is 
currently not implemented in fbdev, please, correct me if I am wrong.

> In my experience with adding defio to the fbdev infra, the
> fbdev community seemed quite good and I did not notice any aging
> problems. I realize there's probably issues that you're encountering
> where fbdev might be weak, this is good, and if you raise them
> specifically, I think the community can work together to address the
> issues.
> 
> >
> > OTOH V4L2 has a standard video output driver support, it is not very
> > widely used, in the userspace I know only of gstreamer, that somehow
> > supports video-output v4l2 devices in latest versions. But, being a part
> > of the v4l2 subsystem, these drivers already now can take a full advantage
> > of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
> >
> > So, how can we help graphics driver developers on the one hand by
> > providing them with a capable driver framework (v4l2) and on the other
> > hand by simplifying the task of interfacing to the user-space?
> 
> I think it would help if there were more specific elaborations on the
> functionality that you'd want the fbdev community to improve and how
> it could reuse v4l2 for this.

Since some time the V4L2 kernel driver API includes a v4l2-subdev API, 
which is used to interface between drivers for various single components. 
Typical examples are USB camera bridges and camera sensors in webcams, 
TV-decoders and PCI framegrabber controllers, any of the above video 
signal sources (sensors or decoders) and a camera interface controller on 
a SoC. In all of the above cases the v4l2-subdev API is used, which allows 
you to use the same video sensor driver in a webcam configuration or with 
an SoC.

> > How about a v4l2-output - fbdev translation layer? You write a v4l2-output
> > driver and get a framebuffer device free of charge... TBH, I haven't given
> > this too much of a thought, but so far I don't see anything that would
> > make this impossible in principle. The video buffer management is quite
> > different between the two systems, but maybe we can teach video-output
> > drivers to work with just one buffer too? Anyway, feel free to tell me why
> > this is an absolutely impossible / impractical idea;)
> 
> Sounds interesting. I think that idea seems viable but I'm not sure
> we'd want every usb webcam to register an fbdev interface if that is
> part of the thoughts you're having. If you could elaborate on the
> benefits, that'd be great.

No, sorry, webcam is a video-input (capture) device, whereas I am talking 
about video output devices like TV encoders. My example is the sh-mobile 
Video Output Unit (VOU) driver, that I've written recently. For it first 
of all we had to decide which subsystem to select - V4L2 output or fbdev. 
On the one hand, if we used fbdev, using the driver would be dead simple - 
just put a console on it, or the X-server or any other user-space 
software, capable of talking to an fbdev. OTOH we would have to come up 
with some (proprietary) API to interface to tv-encoders. So, we chose to 
use v4l2, which gave us a standard API, but the only user-space solution 
we found to interface to v4l2 output drivers was gstreamer, which also 
just recently acquired this capability and is not yet very well developed 
in that respect. That's why we thought about a v4l2-output - fbdev 
interface driver.

> Thanks,
> jaya

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

(adding V4L ML to CC and preserving the complete reply for V4L readers)

On Thu, 27 May 2010, Jaya Kumar wrote:

> On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski
> <g.liakhovetski@gmx.de> wrote:
> > Problem: Currently the standard way to provide graphical output on various
> > (embedded) displays like LCDs is to use a framebuffer driver. The
> > interface is well supported and widely adopted in the user-space, many
> > applications, including the X-server, various libraries like directfb,
> > gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
> > number of problems. It is unmaintained. The infrastructure is not being
> > further developed, every specific hardware driver is being supported by
> > the respective architecture community. But as video output hardware
> 
> I understand the issue you are raising, but to be clear there are
> several developers, Geert, Krzysztof, and others who are helping with
> the role of fbdev maintainer while Tony is away. If you meant that it
> has no specific currently active maintainer person, when you wrote
> "unmaintained", then I agree that is correct.

Exactly, I just interpreted this excerpt from MAINTAINERS:

FRAMEBUFFER LAYER
L:	linux-fbdev@vger.kernel.org
W:	http://linux-fbdev.sourceforge.net/
S:	Orphan

> We're not sure where
> Tony is and we hope he's okay and that he'll be back soon. But if you
> meant that it is not maintained as in bugs aren't being fixed, then
> I'd have to slightly disagree. Maybe not as fast as commercial
> organizations seem to think should come for free, but still they are
> being worked on.
> 
> > evolves, more complex displays and buses appear and have to be supported,
> > the subsystem shows its aging. For example, there is currently no way to
> > write reusable across multiple platforms display drivers.
> 
> At first I misread your point as talking about multi-headed displays
> which you're correct is not so great in fbdev. But "write reusable
> across multi-platform display driver", I did not understand fully. I
> maintain a fbdev driver, broadsheetfb, that we're using on arm and x86
> without problems and my presumption is other fbdev drivers are also
> capable of this unless the author made it explicitly platform
> specific.

Ok, let me explain what exactly I meant. Above I referred to "display 
drivers," which is not the same as a "framebuffer controller driver" or 
whatever you would call it. By framebuffer controller driver I mean a 
driver for the actual graphics engine on a certain graphics card or an 
SoC. This is the part, that reads data from the actual framebuffer and 
outputs it to some hardware interface to a display device. Now this 
interface can be a VGA or a DVI connector, it can be one of several bus 
types, used with various LCD displays. In many cases this is all you have 
to do to get the output to your display. But in some cases the actual 
display on the other side of this bus also requires a driver. That can be 
some kind of a smart display, it can be a panel with an attached display 
controller, that must be at least configured, say, over SPI, it can be a 
display, attached to the host over the MIPI DSI bus, and implementing some 
proprietary commands. In each of these cases you will have to write a 
display driver for this specific display or controller type, and your 
framebuffer driver will have to interface with that display driver. Now, 
obviously, those displays can be connected to a variety of host systems, 
in which case you will want to reuse that display driver. This means, 
there has to be a standard fb-driver - display-driver API. AFAICS, this is 
currently not implemented in fbdev, please, correct me if I am wrong.

> In my experience with adding defio to the fbdev infra, the
> fbdev community seemed quite good and I did not notice any aging
> problems. I realize there's probably issues that you're encountering
> where fbdev might be weak, this is good, and if you raise them
> specifically, I think the community can work together to address the
> issues.
> 
> >
> > OTOH V4L2 has a standard video output driver support, it is not very
> > widely used, in the userspace I know only of gstreamer, that somehow
> > supports video-output v4l2 devices in latest versions. But, being a part
> > of the v4l2 subsystem, these drivers already now can take a full advantage
> > of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
> >
> > So, how can we help graphics driver developers on the one hand by
> > providing them with a capable driver framework (v4l2) and on the other
> > hand by simplifying the task of interfacing to the user-space?
> 
> I think it would help if there were more specific elaborations on the
> functionality that you'd want the fbdev community to improve and how
> it could reuse v4l2 for this.

Since some time the V4L2 kernel driver API includes a v4l2-subdev API, 
which is used to interface between drivers for various single components. 
Typical examples are USB camera bridges and camera sensors in webcams, 
TV-decoders and PCI framegrabber controllers, any of the above video 
signal sources (sensors or decoders) and a camera interface controller on 
a SoC. In all of the above cases the v4l2-subdev API is used, which allows 
you to use the same video sensor driver in a webcam configuration or with 
an SoC.

> > How about a v4l2-output - fbdev translation layer? You write a v4l2-output
> > driver and get a framebuffer device free of charge... TBH, I haven't given
> > this too much of a thought, but so far I don't see anything that would
> > make this impossible in principle. The video buffer management is quite
> > different between the two systems, but maybe we can teach video-output
> > drivers to work with just one buffer too? Anyway, feel free to tell me why
> > this is an absolutely impossible / impractical idea;)
> 
> Sounds interesting. I think that idea seems viable but I'm not sure
> we'd want every usb webcam to register an fbdev interface if that is
> part of the thoughts you're having. If you could elaborate on the
> benefits, that'd be great.

No, sorry, webcam is a video-input (capture) device, whereas I am talking 
about video output devices like TV encoders. My example is the sh-mobile 
Video Output Unit (VOU) driver, that I've written recently. For it first 
of all we had to decide which subsystem to select - V4L2 output or fbdev. 
On the one hand, if we used fbdev, using the driver would be dead simple - 
just put a console on it, or the X-server or any other user-space 
software, capable of talking to an fbdev. OTOH we would have to come up 
with some (proprietary) API to interface to tv-encoders. So, we chose to 
use v4l2, which gave us a standard API, but the only user-space solution 
we found to interface to v4l2 output drivers was gstreamer, which also 
just recently acquired this capability and is not yet very well developed 
in that respect. That's why we thought about a v4l2-output - fbdev 
interface driver.

> Thanks,
> jaya

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

RE: Idea of a v4l -> fb interface driver

[Hiremath, Vaibhav]

> -----Original Message-----
> From: linux-fbdev-owner@vger.kernel.org [mailto:linux-fbdev-
> owner@vger.kernel.org] On Behalf Of Guennadi Liakhovetski
> Sent: Wednesday, May 26, 2010 7:40 PM
> To: linux-fbdev@vger.kernel.org
> Subject: Idea of a v4l -> fb interface driver
> 
> This message has been earlier sent to the V4L mailing list
> 
> Linux Media Mailing List <linux-media@vger.kernel.org>
I replied to Linux-Media list, but saw this thread again. So pasting my response here and adding linux-media to CC.


> 
> When replying, please add it to CC. This is a discussion proposal for the
> V4L mini summit, that is going to take place in June in Helsinki, Finland.
> Any opinions very welcome.
> 
> 		V4L(2) video output vs. framebuffer.
> 
> Problem: Currently the standard way to provide graphical output on various
> (embedded) displays like LCDs is to use a framebuffer driver. The
> interface is well supported and widely adopted in the user-space, many
> applications, including the X-server, various libraries like directfb,
> gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
> number of problems. It is unmaintained. The infrastructure is not being
> further developed, every specific hardware driver is being supported by
> the respective architecture community. But as video output hardware
> evolves, more complex displays and buses appear and have to be supported,
> the subsystem shows its aging. For example, there is currently no way to
> write reusable across multiple platforms display drivers.
> 
[Hiremath, Vaibhav] Up to certain extent yes you are correct.

> OTOH V4L2 has a standard video output driver support, it is not very
> widely used, in the userspace I know only of gstreamer, that somehow
> supports video-output v4l2 devices in latest versions. But, being a part
> of the v4l2 subsystem, these drivers already now can take a full advantage
> of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
> 
> So, how can we help graphics driver developers on the one hand by
> providing them with a capable driver framework (v4l2) and on the other
> hand by simplifying the task of interfacing to the user-space?
> 
[Hiremath, Vaibhav] I think this is really complex question which requires healthy discussion over list.

> How about a v4l2-output - fbdev translation layer? You write a v4l2-output
> driver and get a framebuffer device free of charge... TBH, I haven't given
> this too much of a thought, but so far I don't see anything that would
> make this impossible in principle. The video buffer management is quite
> different between the two systems, but maybe we can teach video-output
> drivers to work with just one buffer too? 
[Hiremath, Vaibhav] I believe V4L2 buf won't limit you to do this. Atleast in case of OMAP v4L2 display driver we are sticking to last buffer if application fails to queue one. So for me this is single buffer keeps on displaying unless application queue next buffer.

> Anyway, feel free to tell me why
> this is an absolutely impossible / impractical idea;)
> 
[Hiremath, Vaibhav] If I understanding correctly you are trying to propose something like,

Without changing Fbdev interface to user space application, create translation layers which will allow driver developer to write driver under V4L2 framework providing /dev/fbx but using V4L2 API/framework.

Also as mentioned by Jaya, it would be great if you put benefits we are targeting would be helpful.

Thanks,
Vaibhav

> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Thu, 27 May 2010, Hiremath, Vaibhav wrote:

> > OTOH V4L2 has a standard video output driver support, it is not very
> > widely used, in the userspace I know only of gstreamer, that somehow
> > supports video-output v4l2 devices in latest versions. But, being a part
> > of the v4l2 subsystem, these drivers already now can take a full advantage
> > of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
> > 
> > So, how can we help graphics driver developers on the one hand by
> > providing them with a capable driver framework (v4l2) and on the other
> > hand by simplifying the task of interfacing to the user-space?
> > 
> [Hiremath, Vaibhav] I think this is really complex question which 
> requires healthy discussion over list.

So, let's do that;)

> > How about a v4l2-output - fbdev translation layer? You write a v4l2-output
> > driver and get a framebuffer device free of charge... TBH, I haven't given
> > this too much of a thought, but so far I don't see anything that would
> > make this impossible in principle. The video buffer management is quite
> > different between the two systems, but maybe we can teach video-output
> > drivers to work with just one buffer too? 
>
> [Hiremath, Vaibhav] I believe V4L2 buf won't limit you to do this. 
> Atleast in case of OMAP v4L2 display driver we are sticking to last 
> buffer if application fails to queue one. So for me this is single 
> buffer keeps on displaying unless application queue next buffer.

Good, so, that's not a problem.

> > Anyway, feel free to tell me why
> > this is an absolutely impossible / impractical idea;)
> > 
> [Hiremath, Vaibhav] If I understanding correctly you are trying to 
> propose something like,
> 
> Without changing Fbdev interface to user space application, create 
> translation layers which will allow driver developer to write driver 
> under V4L2 framework providing /dev/fbx but using V4L2 API/framework.

Exactly.

> Also as mentioned by Jaya, it would be great if you put benefits we are 
> targeting would be helpful.

One of the benefits is the availability the subdevice API, and the 
forthcoming media controller API. I think, on some SoCs graphics 
processing units (scalers, format converters, compressors / decompressors) 
can be configured to either video input or output paths, so, it would make 
sense to manage them from one (v4l) driver framework. And I don't see a 
reason why you cannot have a /dev/fbX interface to the user-space at the 
same time. Yes, you can code it into your v4l driver, and some drivers do 
that already, but why not have it once for all? Last but not lease, having 
multiple incompatible subsystems in the kernel for pretty much the same 
task seems somewhat redundant to me. Yes, I know, this is not a complete 
redundancy, both v4l and fbdev have features, unsupported by the other, 
but IMHO some redundancy is definitely there.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

RE: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Thu, 27 May 2010, Hiremath, Vaibhav wrote:

> > OTOH V4L2 has a standard video output driver support, it is not very
> > widely used, in the userspace I know only of gstreamer, that somehow
> > supports video-output v4l2 devices in latest versions. But, being a part
> > of the v4l2 subsystem, these drivers already now can take a full advantage
> > of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
> > 
> > So, how can we help graphics driver developers on the one hand by
> > providing them with a capable driver framework (v4l2) and on the other
> > hand by simplifying the task of interfacing to the user-space?
> > 
> [Hiremath, Vaibhav] I think this is really complex question which 
> requires healthy discussion over list.

So, let's do that;)

> > How about a v4l2-output - fbdev translation layer? You write a v4l2-output
> > driver and get a framebuffer device free of charge... TBH, I haven't given
> > this too much of a thought, but so far I don't see anything that would
> > make this impossible in principle. The video buffer management is quite
> > different between the two systems, but maybe we can teach video-output
> > drivers to work with just one buffer too? 
>
> [Hiremath, Vaibhav] I believe V4L2 buf won't limit you to do this. 
> Atleast in case of OMAP v4L2 display driver we are sticking to last 
> buffer if application fails to queue one. So for me this is single 
> buffer keeps on displaying unless application queue next buffer.

Good, so, that's not a problem.

> > Anyway, feel free to tell me why
> > this is an absolutely impossible / impractical idea;)
> > 
> [Hiremath, Vaibhav] If I understanding correctly you are trying to 
> propose something like,
> 
> Without changing Fbdev interface to user space application, create 
> translation layers which will allow driver developer to write driver 
> under V4L2 framework providing /dev/fbx but using V4L2 API/framework.

Exactly.

> Also as mentioned by Jaya, it would be great if you put benefits we are 
> targeting would be helpful.

One of the benefits is the availability the subdevice API, and the 
forthcoming media controller API. I think, on some SoCs graphics 
processing units (scalers, format converters, compressors / decompressors) 
can be configured to either video input or output paths, so, it would make 
sense to manage them from one (v4l) driver framework. And I don't see a 
reason why you cannot have a /dev/fbX interface to the user-space at the 
same time. Yes, you can code it into your v4l driver, and some drivers do 
that already, but why not have it once for all? Last but not lease, having 
multiple incompatible subsystems in the kernel for pretty much the same 
task seems somewhat redundant to me. Yes, I know, this is not a complete 
redundancy, both v4l and fbdev have features, unsupported by the other, 
but IMHO some redundancy is definitely there.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Jaya Kumar]

On Thu, May 27, 2010 at 2:56 PM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> (adding V4L ML to CC and preserving the complete reply for V4L readers)
>
> On Thu, 27 May 2010, Jaya Kumar wrote:
>
>> On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski
>> <g.liakhovetski@gmx.de> wrote:
> Ok, let me explain what exactly I meant. Above I referred to "display
> drivers," which is not the same as a "framebuffer controller driver" or
> whatever you would call it. By framebuffer controller driver I mean a
> driver for the actual graphics engine on a certain graphics card or an
> SoC. This is the part, that reads data from the actual framebuffer and
> outputs it to some hardware interface to a display device. Now this
> interface can be a VGA or a DVI connector, it can be one of several bus
> types, used with various LCD displays. In many cases this is all you have
> to do to get the output to your display. But in some cases the actual
> display on the other side of this bus also requires a driver. That can be
> some kind of a smart display, it can be a panel with an attached display
> controller, that must be at least configured, say, over SPI, it can be a
> display, attached to the host over the MIPI DSI bus, and implementing some
> proprietary commands. In each of these cases you will have to write a
> display driver for this specific display or controller type, and your
> framebuffer driver will have to interface with that display driver. Now,
> obviously, those displays can be connected to a variety of host systems,
> in which case you will want to reuse that display driver. This means,
> there has to be a standard fb-driver - display-driver API. AFAICS, this is
> currently not implemented in fbdev, please, correct me if I am wrong.

Thanks Guennadi, your clarification is useful. Yes, you are correct.
There is no general fbdev API provided so that a host controller
driver and a independent display panel driver can interface in a clean
abstracted way.

You've raised the MIPI-DSI issue. It is a good area to focus the
discussion on for fbdev minded people and one that needs to be
resolved soon so that we don't get dozens of host controller specific
mipi display panel drivers. I had seen that omap2 fbdev has a portion
of the MIPI-DSI command set exposed to their various display panel
drivers which then hands off these commands to the omap specific
lcd_mipid.c which uses spi. I see you've also implemented a similar
concept in sh-mobile. When I saw the multiple display panel drivers
showing up in omap, I raised a concern with Tomi and I think there was
an intent to try to improve the abstraction. I'm not sure how far that
has progressed. Are you saying v4l would help us in that area? I'm not
yet able to follow the details of how using v4l would help address the
need for mipi-dsi abstraction. Could you elaborate on that?

Thanks,
jaya

Re: Idea of a v4l -> fb interface driver

[Jaya Kumar]

On Thu, May 27, 2010 at 2:56 PM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> (adding V4L ML to CC and preserving the complete reply for V4L readers)
>
> On Thu, 27 May 2010, Jaya Kumar wrote:
>
>> On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski
>> <g.liakhovetski@gmx.de> wrote:
> Ok, let me explain what exactly I meant. Above I referred to "display
> drivers," which is not the same as a "framebuffer controller driver" or
> whatever you would call it. By framebuffer controller driver I mean a
> driver for the actual graphics engine on a certain graphics card or an
> SoC. This is the part, that reads data from the actual framebuffer and
> outputs it to some hardware interface to a display device. Now this
> interface can be a VGA or a DVI connector, it can be one of several bus
> types, used with various LCD displays. In many cases this is all you have
> to do to get the output to your display. But in some cases the actual
> display on the other side of this bus also requires a driver. That can be
> some kind of a smart display, it can be a panel with an attached display
> controller, that must be at least configured, say, over SPI, it can be a
> display, attached to the host over the MIPI DSI bus, and implementing some
> proprietary commands. In each of these cases you will have to write a
> display driver for this specific display or controller type, and your
> framebuffer driver will have to interface with that display driver. Now,
> obviously, those displays can be connected to a variety of host systems,
> in which case you will want to reuse that display driver. This means,
> there has to be a standard fb-driver - display-driver API. AFAICS, this is
> currently not implemented in fbdev, please, correct me if I am wrong.

Thanks Guennadi, your clarification is useful. Yes, you are correct.
There is no general fbdev API provided so that a host controller
driver and a independent display panel driver can interface in a clean
abstracted way.

You've raised the MIPI-DSI issue. It is a good area to focus the
discussion on for fbdev minded people and one that needs to be
resolved soon so that we don't get dozens of host controller specific
mipi display panel drivers. I had seen that omap2 fbdev has a portion
of the MIPI-DSI command set exposed to their various display panel
drivers which then hands off these commands to the omap specific
lcd_mipid.c which uses spi. I see you've also implemented a similar
concept in sh-mobile. When I saw the multiple display panel drivers
showing up in omap, I raised a concern with Tomi and I think there was
an intent to try to improve the abstraction. I'm not sure how far that
has progressed. Are you saying v4l would help us in that area? I'm not
yet able to follow the details of how using v4l would help address the
need for mipi-dsi abstraction. Could you elaborate on that?

Thanks,
jaya

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Thu, 27 May 2010, Jaya Kumar wrote:

> You've raised the MIPI-DSI issue. It is a good area to focus the
> discussion on for fbdev minded people and one that needs to be
> resolved soon so that we don't get dozens of host controller specific
> mipi display panel drivers. I had seen that omap2 fbdev has a portion
> of the MIPI-DSI command set exposed to their various display panel
> drivers which then hands off these commands to the omap specific
> lcd_mipid.c which uses spi. I see you've also implemented a similar
> concept in sh-mobile. When I saw the multiple display panel drivers
> showing up in omap, I raised a concern with Tomi and I think there was
> an intent to try to improve the abstraction. I'm not sure how far that
> has progressed. Are you saying v4l would help us in that area? I'm not
> yet able to follow the details of how using v4l would help address the
> need for mipi-dsi abstraction. Could you elaborate on that?

Well, I thought about an abstract driver for MIPI DSI... But, there is not 
really much there, that you can abstract. I've created a generic 
mipi_display.h header, that contains defines for display related (DSI, 
DCS) commands and transaction types. Once this header is in the mainline, 
we plan to convert OMAP drivers to it too. To talk to MIPI displays you 
need a capability to send and receive generic short and long telegrams, 
so, providing higher level functions like get_display_id() or 
soft_reset(), probably, wouldn't make sense. What you do need a proper API 
for is, when you start supporting proprietary display-specific commands 
and want to reuse those display drivers with different MIPI DSI hosts. For 
that we will want a generic API like .send_short_command(), 
.send_short_command_param(), etc.

As for using v4l2 for MIPI displays - well, I am not sure it makes sense 
at all. This could make sense if, e.g., you were writing a driver for a 
graphics controller, capable to talk to various PHYs over a fixed bus 
(which is actually also the case with the sh-mobile LCDC), then you could 
design it, using V4L2, in the following way:

/dev/videoX                /dev/fbX
   |                           |
   |   /- - fbdev translate - -/
   v   v
v4l2 output device driver
       |
       v
   v4l2-subdev API
    |    ...    |
    v           v
MIPI PHY ...  parallel PHY
 driver  ...    driver
    |
    v
 MIPI bus
abstraction
    |
    v
MIPI display
  driver

So, you would use the v4l2-subdev API to abstract various PHY drivers. The 
/dev/fbX link above would, certainly, only exist if we implement the 
v4l2-output - fbdev translation driver.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Thu, 27 May 2010, Jaya Kumar wrote:

> You've raised the MIPI-DSI issue. It is a good area to focus the
> discussion on for fbdev minded people and one that needs to be
> resolved soon so that we don't get dozens of host controller specific
> mipi display panel drivers. I had seen that omap2 fbdev has a portion
> of the MIPI-DSI command set exposed to their various display panel
> drivers which then hands off these commands to the omap specific
> lcd_mipid.c which uses spi. I see you've also implemented a similar
> concept in sh-mobile. When I saw the multiple display panel drivers
> showing up in omap, I raised a concern with Tomi and I think there was
> an intent to try to improve the abstraction. I'm not sure how far that
> has progressed. Are you saying v4l would help us in that area? I'm not
> yet able to follow the details of how using v4l would help address the
> need for mipi-dsi abstraction. Could you elaborate on that?

Well, I thought about an abstract driver for MIPI DSI... But, there is not 
really much there, that you can abstract. I've created a generic 
mipi_display.h header, that contains defines for display related (DSI, 
DCS) commands and transaction types. Once this header is in the mainline, 
we plan to convert OMAP drivers to it too. To talk to MIPI displays you 
need a capability to send and receive generic short and long telegrams, 
so, providing higher level functions like get_display_id() or 
soft_reset(), probably, wouldn't make sense. What you do need a proper API 
for is, when you start supporting proprietary display-specific commands 
and want to reuse those display drivers with different MIPI DSI hosts. For 
that we will want a generic API like .send_short_command(), 
.send_short_command_param(), etc.

As for using v4l2 for MIPI displays - well, I am not sure it makes sense 
at all. This could make sense if, e.g., you were writing a driver for a 
graphics controller, capable to talk to various PHYs over a fixed bus 
(which is actually also the case with the sh-mobile LCDC), then you could 
design it, using V4L2, in the following way:

/dev/videoX                /dev/fbX
   |                           |
   |   /- - fbdev translate - -/
   v   v
v4l2 output device driver
       |
       v
   v4l2-subdev API
    |    ...    |
    v           v
MIPI PHY ...  parallel PHY
 driver  ...    driver
    |
    v
 MIPI bus
abstraction
    |
    v
MIPI display
  driver

So, you would use the v4l2-subdev API to abstract various PHY drivers. The 
/dev/fbX link above would, certainly, only exist if we implement the 
v4l2-output - fbdev translation driver.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Udo Richter]

Am 27.05.2010 08:44, schrieb Hiremath, Vaibhav:
>> 		V4L(2) video output vs. framebuffer.
>>
>> Problem: Currently the standard way to provide graphical output on various
>> (embedded) displays like LCDs is to use a framebuffer driver. The
>> interface is well supported and widely adopted in the user-space, many
>> applications, including the X-server, various libraries like directfb,
>> gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
>> number of problems. It is unmaintained. The infrastructure is not being
>> further developed, every specific hardware driver is being supported by
>> the respective architecture community. But as video output hardware
>> evolves, more complex displays and buses appear and have to be supported,
>> the subsystem shows its aging. For example, there is currently no way to
>> write reusable across multiple platforms display drivers.


To add another point of view: I'm not completely sure how much these
topics overlap, but another area where there's display output available,
but not using some generic interface like fbdev, are DVB adapters with
video output capabilities, e.g. /dev/dvb/adapterX/osdY devices and similar.

The 'old' style Technotrend Full Featured DVB cards can only display
either mpeg streams or very basic and restricted OSD overlays, but the
newer generation of HD capable video decoders are usually capable of
displaying an RGB32 video overlay in HD resolution. If these decoder
cards would provide a framebuffer device, these devices could instantly
be used for various media applications on the TV, like xbmc. Actually,
the missing ability to run generic apps on DVB output devices is one of
their biggest disadvantage over regular graphics cards with video
acceleration.

Maybe such a v4l-fbdev interface could handle such devices too?


Cheers,

Udo

Re: Idea of a v4l -> fb interface driver

[Udo Richter]

Am 27.05.2010 08:44, schrieb Hiremath, Vaibhav:
>> 		V4L(2) video output vs. framebuffer.
>>
>> Problem: Currently the standard way to provide graphical output on various
>> (embedded) displays like LCDs is to use a framebuffer driver. The
>> interface is well supported and widely adopted in the user-space, many
>> applications, including the X-server, various libraries like directfb,
>> gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
>> number of problems. It is unmaintained. The infrastructure is not being
>> further developed, every specific hardware driver is being supported by
>> the respective architecture community. But as video output hardware
>> evolves, more complex displays and buses appear and have to be supported,
>> the subsystem shows its aging. For example, there is currently no way to
>> write reusable across multiple platforms display drivers.


To add another point of view: I'm not completely sure how much these
topics overlap, but another area where there's display output available,
but not using some generic interface like fbdev, are DVB adapters with
video output capabilities, e.g. /dev/dvb/adapterX/osdY devices and similar.

The 'old' style Technotrend Full Featured DVB cards can only display
either mpeg streams or very basic and restricted OSD overlays, but the
newer generation of HD capable video decoders are usually capable of
displaying an RGB32 video overlay in HD resolution. If these decoder
cards would provide a framebuffer device, these devices could instantly
be used for various media applications on the TV, like xbmc. Actually,
the missing ability to run generic apps on DVB output devices is one of
their biggest disadvantage over regular graphics cards with video
acceleration.

Maybe such a v4l-fbdev interface could handle such devices too?


Cheers,

Udo

Re: Idea of a v4l -> fb interface driver

[Alex Deucher]

On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> (adding V4L ML to CC and preserving the complete reply for V4L readers)
>
> On Thu, 27 May 2010, Jaya Kumar wrote:
>
>> On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski
>> <g.liakhovetski@gmx.de> wrote:
>> > Problem: Currently the standard way to provide graphical output on various
>> > (embedded) displays like LCDs is to use a framebuffer driver. The
>> > interface is well supported and widely adopted in the user-space, many
>> > applications, including the X-server, various libraries like directfb,
>> > gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
>> > number of problems. It is unmaintained. The infrastructure is not being
>> > further developed, every specific hardware driver is being supported by
>> > the respective architecture community. But as video output hardware
>>
>> I understand the issue you are raising, but to be clear there are
>> several developers, Geert, Krzysztof, and others who are helping with
>> the role of fbdev maintainer while Tony is away. If you meant that it
>> has no specific currently active maintainer person, when you wrote
>> "unmaintained", then I agree that is correct.
>
> Exactly, I just interpreted this excerpt from MAINTAINERS:
>
> FRAMEBUFFER LAYER
> L:      linux-fbdev@vger.kernel.org
> W:      http://linux-fbdev.sourceforge.net/
> S:      Orphan
>
>> We're not sure where
>> Tony is and we hope he's okay and that he'll be back soon. But if you
>> meant that it is not maintained as in bugs aren't being fixed, then
>> I'd have to slightly disagree. Maybe not as fast as commercial
>> organizations seem to think should come for free, but still they are
>> being worked on.
>>
>> > evolves, more complex displays and buses appear and have to be supported,
>> > the subsystem shows its aging. For example, there is currently no way to
>> > write reusable across multiple platforms display drivers.
>>
>> At first I misread your point as talking about multi-headed displays
>> which you're correct is not so great in fbdev. But "write reusable
>> across multi-platform display driver", I did not understand fully. I
>> maintain a fbdev driver, broadsheetfb, that we're using on arm and x86
>> without problems and my presumption is other fbdev drivers are also
>> capable of this unless the author made it explicitly platform
>> specific.
>
> Ok, let me explain what exactly I meant. Above I referred to "display
> drivers," which is not the same as a "framebuffer controller driver" or
> whatever you would call it. By framebuffer controller driver I mean a
> driver for the actual graphics engine on a certain graphics card or an
> SoC. This is the part, that reads data from the actual framebuffer and
> outputs it to some hardware interface to a display device. Now this
> interface can be a VGA or a DVI connector, it can be one of several bus
> types, used with various LCD displays. In many cases this is all you have
> to do to get the output to your display. But in some cases the actual
> display on the other side of this bus also requires a driver. That can be
> some kind of a smart display, it can be a panel with an attached display
> controller, that must be at least configured, say, over SPI, it can be a
> display, attached to the host over the MIPI DSI bus, and implementing some
> proprietary commands. In each of these cases you will have to write a
> display driver for this specific display or controller type, and your
> framebuffer driver will have to interface with that display driver. Now,
> obviously, those displays can be connected to a variety of host systems,
> in which case you will want to reuse that display driver. This means,
> there has to be a standard fb-driver - display-driver API. AFAICS, this is
> currently not implemented in fbdev, please, correct me if I am wrong.


Another API to consider in the drm kms (kernel modesetting) interface.
 The kms API deals properly with advanced display hardware and
properly handles crtcs, encoders, and connectors.  It also provides
fbdev api emulation.

Alex


>
>> In my experience with adding defio to the fbdev infra, the
>> fbdev community seemed quite good and I did not notice any aging
>> problems. I realize there's probably issues that you're encountering
>> where fbdev might be weak, this is good, and if you raise them
>> specifically, I think the community can work together to address the
>> issues.
>>
>> >
>> > OTOH V4L2 has a standard video output driver support, it is not very
>> > widely used, in the userspace I know only of gstreamer, that somehow
>> > supports video-output v4l2 devices in latest versions. But, being a part
>> > of the v4l2 subsystem, these drivers already now can take a full advantage
>> > of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
>> >
>> > So, how can we help graphics driver developers on the one hand by
>> > providing them with a capable driver framework (v4l2) and on the other
>> > hand by simplifying the task of interfacing to the user-space?
>>
>> I think it would help if there were more specific elaborations on the
>> functionality that you'd want the fbdev community to improve and how
>> it could reuse v4l2 for this.
>
> Since some time the V4L2 kernel driver API includes a v4l2-subdev API,
> which is used to interface between drivers for various single components.
> Typical examples are USB camera bridges and camera sensors in webcams,
> TV-decoders and PCI framegrabber controllers, any of the above video
> signal sources (sensors or decoders) and a camera interface controller on
> a SoC. In all of the above cases the v4l2-subdev API is used, which allows
> you to use the same video sensor driver in a webcam configuration or with
> an SoC.
>
>> > How about a v4l2-output - fbdev translation layer? You write a v4l2-output
>> > driver and get a framebuffer device free of charge... TBH, I haven't given
>> > this too much of a thought, but so far I don't see anything that would
>> > make this impossible in principle. The video buffer management is quite
>> > different between the two systems, but maybe we can teach video-output
>> > drivers to work with just one buffer too? Anyway, feel free to tell me why
>> > this is an absolutely impossible / impractical idea;)
>>
>> Sounds interesting. I think that idea seems viable but I'm not sure
>> we'd want every usb webcam to register an fbdev interface if that is
>> part of the thoughts you're having. If you could elaborate on the
>> benefits, that'd be great.
>
> No, sorry, webcam is a video-input (capture) device, whereas I am talking
> about video output devices like TV encoders. My example is the sh-mobile
> Video Output Unit (VOU) driver, that I've written recently. For it first
> of all we had to decide which subsystem to select - V4L2 output or fbdev.
> On the one hand, if we used fbdev, using the driver would be dead simple -
> just put a console on it, or the X-server or any other user-space
> software, capable of talking to an fbdev. OTOH we would have to come up
> with some (proprietary) API to interface to tv-encoders. So, we chose to
> use v4l2, which gave us a standard API, but the only user-space solution
> we found to interface to v4l2 output drivers was gstreamer, which also
> just recently acquired this capability and is not yet very well developed
> in that respect. That's why we thought about a v4l2-output - fbdev
> interface driver.
>
>> Thanks,
>> jaya
>
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Idea of a v4l -> fb interface driver

[Alex Deucher]

On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> (adding V4L ML to CC and preserving the complete reply for V4L readers)
>
> On Thu, 27 May 2010, Jaya Kumar wrote:
>
>> On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski
>> <g.liakhovetski@gmx.de> wrote:
>> > Problem: Currently the standard way to provide graphical output on various
>> > (embedded) displays like LCDs is to use a framebuffer driver. The
>> > interface is well supported and widely adopted in the user-space, many
>> > applications, including the X-server, various libraries like directfb,
>> > gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a
>> > number of problems. It is unmaintained. The infrastructure is not being
>> > further developed, every specific hardware driver is being supported by
>> > the respective architecture community. But as video output hardware
>>
>> I understand the issue you are raising, but to be clear there are
>> several developers, Geert, Krzysztof, and others who are helping with
>> the role of fbdev maintainer while Tony is away. If you meant that it
>> has no specific currently active maintainer person, when you wrote
>> "unmaintained", then I agree that is correct.
>
> Exactly, I just interpreted this excerpt from MAINTAINERS:
>
> FRAMEBUFFER LAYER
> L:      linux-fbdev@vger.kernel.org
> W:      http://linux-fbdev.sourceforge.net/
> S:      Orphan
>
>> We're not sure where
>> Tony is and we hope he's okay and that he'll be back soon. But if you
>> meant that it is not maintained as in bugs aren't being fixed, then
>> I'd have to slightly disagree. Maybe not as fast as commercial
>> organizations seem to think should come for free, but still they are
>> being worked on.
>>
>> > evolves, more complex displays and buses appear and have to be supported,
>> > the subsystem shows its aging. For example, there is currently no way to
>> > write reusable across multiple platforms display drivers.
>>
>> At first I misread your point as talking about multi-headed displays
>> which you're correct is not so great in fbdev. But "write reusable
>> across multi-platform display driver", I did not understand fully. I
>> maintain a fbdev driver, broadsheetfb, that we're using on arm and x86
>> without problems and my presumption is other fbdev drivers are also
>> capable of this unless the author made it explicitly platform
>> specific.
>
> Ok, let me explain what exactly I meant. Above I referred to "display
> drivers," which is not the same as a "framebuffer controller driver" or
> whatever you would call it. By framebuffer controller driver I mean a
> driver for the actual graphics engine on a certain graphics card or an
> SoC. This is the part, that reads data from the actual framebuffer and
> outputs it to some hardware interface to a display device. Now this
> interface can be a VGA or a DVI connector, it can be one of several bus
> types, used with various LCD displays. In many cases this is all you have
> to do to get the output to your display. But in some cases the actual
> display on the other side of this bus also requires a driver. That can be
> some kind of a smart display, it can be a panel with an attached display
> controller, that must be at least configured, say, over SPI, it can be a
> display, attached to the host over the MIPI DSI bus, and implementing some
> proprietary commands. In each of these cases you will have to write a
> display driver for this specific display or controller type, and your
> framebuffer driver will have to interface with that display driver. Now,
> obviously, those displays can be connected to a variety of host systems,
> in which case you will want to reuse that display driver. This means,
> there has to be a standard fb-driver - display-driver API. AFAICS, this is
> currently not implemented in fbdev, please, correct me if I am wrong.


Another API to consider in the drm kms (kernel modesetting) interface.
 The kms API deals properly with advanced display hardware and
properly handles crtcs, encoders, and connectors.  It also provides
fbdev api emulation.

Alex


>
>> In my experience with adding defio to the fbdev infra, the
>> fbdev community seemed quite good and I did not notice any aging
>> problems. I realize there's probably issues that you're encountering
>> where fbdev might be weak, this is good, and if you raise them
>> specifically, I think the community can work together to address the
>> issues.
>>
>> >
>> > OTOH V4L2 has a standard video output driver support, it is not very
>> > widely used, in the userspace I know only of gstreamer, that somehow
>> > supports video-output v4l2 devices in latest versions. But, being a part
>> > of the v4l2 subsystem, these drivers already now can take a full advantage
>> > of all v4l2 APIs, including the v4l2-subdev API for the driver reuse.
>> >
>> > So, how can we help graphics driver developers on the one hand by
>> > providing them with a capable driver framework (v4l2) and on the other
>> > hand by simplifying the task of interfacing to the user-space?
>>
>> I think it would help if there were more specific elaborations on the
>> functionality that you'd want the fbdev community to improve and how
>> it could reuse v4l2 for this.
>
> Since some time the V4L2 kernel driver API includes a v4l2-subdev API,
> which is used to interface between drivers for various single components.
> Typical examples are USB camera bridges and camera sensors in webcams,
> TV-decoders and PCI framegrabber controllers, any of the above video
> signal sources (sensors or decoders) and a camera interface controller on
> a SoC. In all of the above cases the v4l2-subdev API is used, which allows
> you to use the same video sensor driver in a webcam configuration or with
> an SoC.
>
>> > How about a v4l2-output - fbdev translation layer? You write a v4l2-output
>> > driver and get a framebuffer device free of charge... TBH, I haven't given
>> > this too much of a thought, but so far I don't see anything that would
>> > make this impossible in principle. The video buffer management is quite
>> > different between the two systems, but maybe we can teach video-output
>> > drivers to work with just one buffer too? Anyway, feel free to tell me why
>> > this is an absolutely impossible / impractical idea;)
>>
>> Sounds interesting. I think that idea seems viable but I'm not sure
>> we'd want every usb webcam to register an fbdev interface if that is
>> part of the thoughts you're having. If you could elaborate on the
>> benefits, that'd be great.
>
> No, sorry, webcam is a video-input (capture) device, whereas I am talking
> about video output devices like TV encoders. My example is the sh-mobile
> Video Output Unit (VOU) driver, that I've written recently. For it first
> of all we had to decide which subsystem to select - V4L2 output or fbdev.
> On the one hand, if we used fbdev, using the driver would be dead simple -
> just put a console on it, or the X-server or any other user-space
> software, capable of talking to an fbdev. OTOH we would have to come up
> with some (proprietary) API to interface to tv-encoders. So, we chose to
> use v4l2, which gave us a standard API, but the only user-space solution
> we found to interface to v4l2 output drivers was gstreamer, which also
> just recently acquired this capability and is not yet very well developed
> in that respect. That's why we thought about a v4l2-output - fbdev
> interface driver.
>
>> Thanks,
>> jaya
>
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Thu, 27 May 2010, Alex Deucher wrote:

> On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski <g.liakhovetski@gmx.de> wrote:

...

> > Ok, let me explain what exactly I meant. Above I referred to "display
> > drivers," which is not the same as a "framebuffer controller driver" or
> > whatever you would call it. By framebuffer controller driver I mean a
> > driver for the actual graphics engine on a certain graphics card or an
> > SoC. This is the part, that reads data from the actual framebuffer and
> > outputs it to some hardware interface to a display device. Now this
> > interface can be a VGA or a DVI connector, it can be one of several bus
> > types, used with various LCD displays. In many cases this is all you have
> > to do to get the output to your display. But in some cases the actual
> > display on the other side of this bus also requires a driver. That can be
> > some kind of a smart display, it can be a panel with an attached display
> > controller, that must be at least configured, say, over SPI, it can be a
> > display, attached to the host over the MIPI DSI bus, and implementing some
> > proprietary commands. In each of these cases you will have to write a
> > display driver for this specific display or controller type, and your
> > framebuffer driver will have to interface with that display driver. Now,
> > obviously, those displays can be connected to a variety of host systems,
> > in which case you will want to reuse that display driver. This means,
> > there has to be a standard fb-driver - display-driver API. AFAICS, this is
> > currently not implemented in fbdev, please, correct me if I am wrong.
> 
> 
> Another API to consider in the drm kms (kernel modesetting) interface.
>  The kms API deals properly with advanced display hardware and
> properly handles crtcs, encoders, and connectors.  It also provides
> fbdev api emulation.

Well, is KMS planned as a replacement for both fbdev and user-space 
graphics drivers? I mean, if you'd be writing a new fb driver for a 
relatively simple embedded SoC, would KMS apriori be a preferred API?

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Thu, 27 May 2010, Alex Deucher wrote:

> On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski <g.liakhovetski@gmx.de> wrote:

...

> > Ok, let me explain what exactly I meant. Above I referred to "display
> > drivers," which is not the same as a "framebuffer controller driver" or
> > whatever you would call it. By framebuffer controller driver I mean a
> > driver for the actual graphics engine on a certain graphics card or an
> > SoC. This is the part, that reads data from the actual framebuffer and
> > outputs it to some hardware interface to a display device. Now this
> > interface can be a VGA or a DVI connector, it can be one of several bus
> > types, used with various LCD displays. In many cases this is all you have
> > to do to get the output to your display. But in some cases the actual
> > display on the other side of this bus also requires a driver. That can be
> > some kind of a smart display, it can be a panel with an attached display
> > controller, that must be at least configured, say, over SPI, it can be a
> > display, attached to the host over the MIPI DSI bus, and implementing some
> > proprietary commands. In each of these cases you will have to write a
> > display driver for this specific display or controller type, and your
> > framebuffer driver will have to interface with that display driver. Now,
> > obviously, those displays can be connected to a variety of host systems,
> > in which case you will want to reuse that display driver. This means,
> > there has to be a standard fb-driver - display-driver API. AFAICS, this is
> > currently not implemented in fbdev, please, correct me if I am wrong.
> 
> 
> Another API to consider in the drm kms (kernel modesetting) interface.
>  The kms API deals properly with advanced display hardware and
> properly handles crtcs, encoders, and connectors.  It also provides
> fbdev api emulation.

Well, is KMS planned as a replacement for both fbdev and user-space 
graphics drivers? I mean, if you'd be writing a new fb driver for a 
relatively simple embedded SoC, would KMS apriori be a preferred API?

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

(re-adding lists to CC)

On Thu, 27 May 2010, Rob Clark wrote:

> Hi Guennadi,
> 
> Sounds like an interesting idea... but how about the inverse?  A v4l2
> interface on top of fbdev.  If v4l2 was more widely available as an output
> device, perhaps more userspace software would utilize it.

Don't see any advantage in doing this apart from "attracting user-space 
developers to develop for v4l2 output interface," which doesn't seem like 
a worthy goal in itself. Whereas with my translation you get access to 
existing user-space applications and to a powerful in-kernel API, and 
achieve a better interoperability with video-input streams.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

(re-adding lists to CC)

On Thu, 27 May 2010, Rob Clark wrote:

> Hi Guennadi,
> 
> Sounds like an interesting idea... but how about the inverse?  A v4l2
> interface on top of fbdev.  If v4l2 was more widely available as an output
> device, perhaps more userspace software would utilize it.

Don't see any advantage in doing this apart from "attracting user-space 
developers to develop for v4l2 output interface," which doesn't seem like 
a worthy goal in itself. Whereas with my translation you get access to 
existing user-space applications and to a powerful in-kernel API, and 
achieve a better interoperability with video-input streams.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Alex Deucher]

On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> On Thu, 27 May 2010, Alex Deucher wrote:
>
>> On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski <g.liakhovetski@gmx.de> wrote:
>
> ...
>
>> > Ok, let me explain what exactly I meant. Above I referred to "display
>> > drivers," which is not the same as a "framebuffer controller driver" or
>> > whatever you would call it. By framebuffer controller driver I mean a
>> > driver for the actual graphics engine on a certain graphics card or an
>> > SoC. This is the part, that reads data from the actual framebuffer and
>> > outputs it to some hardware interface to a display device. Now this
>> > interface can be a VGA or a DVI connector, it can be one of several bus
>> > types, used with various LCD displays. In many cases this is all you have
>> > to do to get the output to your display. But in some cases the actual
>> > display on the other side of this bus also requires a driver. That can be
>> > some kind of a smart display, it can be a panel with an attached display
>> > controller, that must be at least configured, say, over SPI, it can be a
>> > display, attached to the host over the MIPI DSI bus, and implementing some
>> > proprietary commands. In each of these cases you will have to write a
>> > display driver for this specific display or controller type, and your
>> > framebuffer driver will have to interface with that display driver. Now,
>> > obviously, those displays can be connected to a variety of host systems,
>> > in which case you will want to reuse that display driver. This means,
>> > there has to be a standard fb-driver - display-driver API. AFAICS, this is
>> > currently not implemented in fbdev, please, correct me if I am wrong.
>>
>>
>> Another API to consider in the drm kms (kernel modesetting) interface.
>>  The kms API deals properly with advanced display hardware and
>> properly handles crtcs, encoders, and connectors.  It also provides
>> fbdev api emulation.
>
> Well, is KMS planned as a replacement for both fbdev and user-space
> graphics drivers? I mean, if you'd be writing a new fb driver for a
> relatively simple embedded SoC, would KMS apriori be a preferred API?

It's become the defacto standard for X and things like EGL are being
built onto of the API.  As for the kms vs fbdev, kms provides a nice
API for complex display setups with multiple display controllers and
connectors while fbdev assumes one monitor/connector/encoder per
device.  The fbdev and console stuff has yet to take advantage of this
flexibility, I'm not sure what will happen there.  fbdev emulation is
provided by kms, but it has to hide the complexity of the attached
displays.  For an soc with a single encoder and display, there's
probably not much advantage over fbdev, but if you have an soc that
can do TMDS and LVDS and possibly analog tv out, it gets more
interesting.

drm has historically been tied to pci, but Jordan Crouse recently
posted changes to support platform devices:
http://lists.freedesktop.org/archives/dri-devel/2010-May/000887.html

Alex

Re: Idea of a v4l -> fb interface driver

[Alex Deucher]

On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> On Thu, 27 May 2010, Alex Deucher wrote:
>
>> On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski <g.liakhovetski@gmx.de> wrote:
>
> ...
>
>> > Ok, let me explain what exactly I meant. Above I referred to "display
>> > drivers," which is not the same as a "framebuffer controller driver" or
>> > whatever you would call it. By framebuffer controller driver I mean a
>> > driver for the actual graphics engine on a certain graphics card or an
>> > SoC. This is the part, that reads data from the actual framebuffer and
>> > outputs it to some hardware interface to a display device. Now this
>> > interface can be a VGA or a DVI connector, it can be one of several bus
>> > types, used with various LCD displays. In many cases this is all you have
>> > to do to get the output to your display. But in some cases the actual
>> > display on the other side of this bus also requires a driver. That can be
>> > some kind of a smart display, it can be a panel with an attached display
>> > controller, that must be at least configured, say, over SPI, it can be a
>> > display, attached to the host over the MIPI DSI bus, and implementing some
>> > proprietary commands. In each of these cases you will have to write a
>> > display driver for this specific display or controller type, and your
>> > framebuffer driver will have to interface with that display driver. Now,
>> > obviously, those displays can be connected to a variety of host systems,
>> > in which case you will want to reuse that display driver. This means,
>> > there has to be a standard fb-driver - display-driver API. AFAICS, this is
>> > currently not implemented in fbdev, please, correct me if I am wrong.
>>
>>
>> Another API to consider in the drm kms (kernel modesetting) interface.
>>  The kms API deals properly with advanced display hardware and
>> properly handles crtcs, encoders, and connectors.  It also provides
>> fbdev api emulation.
>
> Well, is KMS planned as a replacement for both fbdev and user-space
> graphics drivers? I mean, if you'd be writing a new fb driver for a
> relatively simple embedded SoC, would KMS apriori be a preferred API?

It's become the defacto standard for X and things like EGL are being
built onto of the API.  As for the kms vs fbdev, kms provides a nice
API for complex display setups with multiple display controllers and
connectors while fbdev assumes one monitor/connector/encoder per
device.  The fbdev and console stuff has yet to take advantage of this
flexibility, I'm not sure what will happen there.  fbdev emulation is
provided by kms, but it has to hide the complexity of the attached
displays.  For an soc with a single encoder and display, there's
probably not much advantage over fbdev, but if you have an soc that
can do TMDS and LVDS and possibly analog tv out, it gets more
interesting.

drm has historically been tied to pci, but Jordan Crouse recently
posted changes to support platform devices:
http://lists.freedesktop.org/archives/dri-devel/2010-May/000887.html

Alex

Re: Idea of a v4l -> fb interface driver

[Florian Tobias Schandinat]

Alex Deucher schrieb:
> On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski
> <g.liakhovetski@gmx.de> wrote:
>> On Thu, 27 May 2010, Alex Deucher wrote:
>>
>>>
>>> Another API to consider in the drm kms (kernel modesetting) interface.
>>>  The kms API deals properly with advanced display hardware and
>>> properly handles crtcs, encoders, and connectors.  It also provides
>>> fbdev api emulation.
>> Well, is KMS planned as a replacement for both fbdev and user-space
>> graphics drivers? I mean, if you'd be writing a new fb driver for a
>> relatively simple embedded SoC, would KMS apriori be a preferred API?
> 
> It's become the defacto standard for X and things like EGL are being
> built onto of the API.  As for the kms vs fbdev, kms provides a nice
> API for complex display setups with multiple display controllers and
> connectors while fbdev assumes one monitor/connector/encoder per
> device.  The fbdev and console stuff has yet to take advantage of this
> flexibility, I'm not sure what will happen there.  fbdev emulation is
> provided by kms, but it has to hide the complexity of the attached
> displays.  For an soc with a single encoder and display, there's
> probably not much advantage over fbdev, but if you have an soc that
> can do TMDS and LVDS and possibly analog tv out, it gets more
> interesting.

Well hiding complexity is actually the job of an API. I don't see any 
need for major changes in fbdev for complex display setups. In most 
cases as a userspace application you really don't want to be bothered 
how many different output devices you have and control each 
individually, you just want an area to draw and to know/control what 
area the user is expected to see and that's already provided in fbdev.
If the user wants the same content on multiple outputs just configure 
the driver to do so.
If he wants different (independent) content on each output, just provide 
multiple /dev/fbX devices. I admit that we could use a controlling 
interface here that decides which user (application) might draw at a 
time to the interface which they currently only do if they are the 
active VT.
If you want 2 or more outputs to be merged as one just configure this in 
the driver.
The only thing that is impossible to do in fbdev is controlling 2 or 
more independent display outputs that access the same buffer. But that's 
not an issue I think.
The things above only could use a unification of how to set them up on 
module load time (as only limited runtime changes are permited given 
that we must always be able to support a mode that we once entered 
during runtime).

The thing that's really missing in fbdev is a way to allow hardware 
acceleration for userspace.


Regards,

Florian Tobias Schandinat

Re: Idea of a v4l -> fb interface driver

[Florian Tobias Schandinat]

Alex Deucher schrieb:
> On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski
> <g.liakhovetski@gmx.de> wrote:
>> On Thu, 27 May 2010, Alex Deucher wrote:
>>
>>>
>>> Another API to consider in the drm kms (kernel modesetting) interface.
>>>  The kms API deals properly with advanced display hardware and
>>> properly handles crtcs, encoders, and connectors.  It also provides
>>> fbdev api emulation.
>> Well, is KMS planned as a replacement for both fbdev and user-space
>> graphics drivers? I mean, if you'd be writing a new fb driver for a
>> relatively simple embedded SoC, would KMS apriori be a preferred API?
> 
> It's become the defacto standard for X and things like EGL are being
> built onto of the API.  As for the kms vs fbdev, kms provides a nice
> API for complex display setups with multiple display controllers and
> connectors while fbdev assumes one monitor/connector/encoder per
> device.  The fbdev and console stuff has yet to take advantage of this
> flexibility, I'm not sure what will happen there.  fbdev emulation is
> provided by kms, but it has to hide the complexity of the attached
> displays.  For an soc with a single encoder and display, there's
> probably not much advantage over fbdev, but if you have an soc that
> can do TMDS and LVDS and possibly analog tv out, it gets more
> interesting.

Well hiding complexity is actually the job of an API. I don't see any 
need for major changes in fbdev for complex display setups. In most 
cases as a userspace application you really don't want to be bothered 
how many different output devices you have and control each 
individually, you just want an area to draw and to know/control what 
area the user is expected to see and that's already provided in fbdev.
If the user wants the same content on multiple outputs just configure 
the driver to do so.
If he wants different (independent) content on each output, just provide 
multiple /dev/fbX devices. I admit that we could use a controlling 
interface here that decides which user (application) might draw at a 
time to the interface which they currently only do if they are the 
active VT.
If you want 2 or more outputs to be merged as one just configure this in 
the driver.
The only thing that is impossible to do in fbdev is controlling 2 or 
more independent display outputs that access the same buffer. But that's 
not an issue I think.
The things above only could use a unification of how to set them up on 
module load time (as only limited runtime changes are permited given 
that we must always be able to support a mode that we once entered 
during runtime).

The thing that's really missing in fbdev is a way to allow hardware 
acceleration for userspace.


Regards,

Florian Tobias Schandinat

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Fri, 28 May 2010, Florian Tobias Schandinat wrote:

> Well hiding complexity is actually the job of an API. I don't see any need for
> major changes in fbdev for complex display setups. In most cases as a
> userspace application you really don't want to be bothered how many different
> output devices you have and control each individually, you just want an area
> to draw and to know/control what area the user is expected to see and that's
> already provided in fbdev.
> If the user wants the same content on multiple outputs just configure the
> driver to do so.
> If he wants different (independent) content on each output, just provide
> multiple /dev/fbX devices. I admit that we could use a controlling interface
> here that decides which user (application) might draw at a time to the
> interface which they currently only do if they are the active VT.
> If you want 2 or more outputs to be merged as one just configure this in the
> driver.
> The only thing that is impossible to do in fbdev is controlling 2 or more
> independent display outputs that access the same buffer. But that's not an
> issue I think.
> The things above only could use a unification of how to set them up on module
> load time (as only limited runtime changes are permited given that we must
> always be able to support a mode that we once entered during runtime).
> 
> The thing that's really missing in fbdev is a way to allow hardware
> acceleration for userspace.

How about a "simple" use-case, that I asked about in another my mail: how 
do you inform fbdev users, if a (DVI) display has been disconnected and 
another one with a different resolution has been connected?

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Guennadi Liakhovetski]

On Fri, 28 May 2010, Florian Tobias Schandinat wrote:

> Well hiding complexity is actually the job of an API. I don't see any need for
> major changes in fbdev for complex display setups. In most cases as a
> userspace application you really don't want to be bothered how many different
> output devices you have and control each individually, you just want an area
> to draw and to know/control what area the user is expected to see and that's
> already provided in fbdev.
> If the user wants the same content on multiple outputs just configure the
> driver to do so.
> If he wants different (independent) content on each output, just provide
> multiple /dev/fbX devices. I admit that we could use a controlling interface
> here that decides which user (application) might draw at a time to the
> interface which they currently only do if they are the active VT.
> If you want 2 or more outputs to be merged as one just configure this in the
> driver.
> The only thing that is impossible to do in fbdev is controlling 2 or more
> independent display outputs that access the same buffer. But that's not an
> issue I think.
> The things above only could use a unification of how to set them up on module
> load time (as only limited runtime changes are permited given that we must
> always be able to support a mode that we once entered during runtime).
> 
> The thing that's really missing in fbdev is a way to allow hardware
> acceleration for userspace.

How about a "simple" use-case, that I asked about in another my mail: how 
do you inform fbdev users, if a (DVI) display has been disconnected and 
another one with a different resolution has been connected?

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: Idea of a v4l -> fb interface driver

[Alex Deucher]

On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat
<FlorianSchandinat@gmx.de> wrote:
> Alex Deucher schrieb:
>>
>> On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski
>> <g.liakhovetski@gmx.de> wrote:
>>>
>>> On Thu, 27 May 2010, Alex Deucher wrote:
>>>
>>>>
>>>> Another API to consider in the drm kms (kernel modesetting) interface.
>>>>  The kms API deals properly with advanced display hardware and
>>>> properly handles crtcs, encoders, and connectors.  It also provides
>>>> fbdev api emulation.
>>>
>>> Well, is KMS planned as a replacement for both fbdev and user-space
>>> graphics drivers? I mean, if you'd be writing a new fb driver for a
>>> relatively simple embedded SoC, would KMS apriori be a preferred API?
>>
>> It's become the defacto standard for X and things like EGL are being
>> built onto of the API.  As for the kms vs fbdev, kms provides a nice
>> API for complex display setups with multiple display controllers and
>> connectors while fbdev assumes one monitor/connector/encoder per
>> device.  The fbdev and console stuff has yet to take advantage of this
>> flexibility, I'm not sure what will happen there.  fbdev emulation is
>> provided by kms, but it has to hide the complexity of the attached
>> displays.  For an soc with a single encoder and display, there's
>> probably not much advantage over fbdev, but if you have an soc that
>> can do TMDS and LVDS and possibly analog tv out, it gets more
>> interesting.
>
> Well hiding complexity is actually the job of an API. I don't see any need
> for major changes in fbdev for complex display setups. In most cases as a
> userspace application you really don't want to be bothered how many
> different output devices you have and control each individually, you just
> want an area to draw and to know/control what area the user is expected to
> see and that's already provided in fbdev.

Users want to be able to dynamically change their display config on the fly.

> If the user wants the same content on multiple outputs just configure the
> driver to do so.

KMS provide an API to do that and a nice internal abstraction for handling it.

> If he wants different (independent) content on each output, just provide
> multiple /dev/fbX devices. I admit that we could use a controlling interface
> here that decides which user (application) might draw at a time to the
> interface which they currently only do if they are the active VT.
> If you want 2 or more outputs to be merged as one just configure this in the
> driver.
> The only thing that is impossible to do in fbdev is controlling 2 or more
> independent display outputs that access the same buffer. But that's not an
> issue I think.
> The things above only could use a unification of how to set them up on
> module load time (as only limited runtime changes are permited given that we
> must always be able to support a mode that we once entered during runtime).
>

What about changing outputs on the fly (turn off VGA, turn on DVI,
switch between multi-head and single-head, etc) or encoders shared
between multiple connectors (think a single dac shared between a VGA
and a TV port); how do you expose them easily as separate fbdevs?
Lots of stuff is doable with fbdev, but it's nicer with kms.

Alex

> The thing that's really missing in fbdev is a way to allow hardware
> acceleration for userspace.
>
>
> Regards,
>
> Florian Tobias Schandinat
>
>

Re: Idea of a v4l -> fb interface driver

[Alex Deucher]

On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat
<FlorianSchandinat@gmx.de> wrote:
> Alex Deucher schrieb:
>>
>> On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski
>> <g.liakhovetski@gmx.de> wrote:
>>>
>>> On Thu, 27 May 2010, Alex Deucher wrote:
>>>
>>>>
>>>> Another API to consider in the drm kms (kernel modesetting) interface.
>>>>  The kms API deals properly with advanced display hardware and
>>>> properly handles crtcs, encoders, and connectors.  It also provides
>>>> fbdev api emulation.
>>>
>>> Well, is KMS planned as a replacement for both fbdev and user-space
>>> graphics drivers? I mean, if you'd be writing a new fb driver for a
>>> relatively simple embedded SoC, would KMS apriori be a preferred API?
>>
>> It's become the defacto standard for X and things like EGL are being
>> built onto of the API.  As for the kms vs fbdev, kms provides a nice
>> API for complex display setups with multiple display controllers and
>> connectors while fbdev assumes one monitor/connector/encoder per
>> device.  The fbdev and console stuff has yet to take advantage of this
>> flexibility, I'm not sure what will happen there.  fbdev emulation is
>> provided by kms, but it has to hide the complexity of the attached
>> displays.  For an soc with a single encoder and display, there's
>> probably not much advantage over fbdev, but if you have an soc that
>> can do TMDS and LVDS and possibly analog tv out, it gets more
>> interesting.
>
> Well hiding complexity is actually the job of an API. I don't see any need
> for major changes in fbdev for complex display setups. In most cases as a
> userspace application you really don't want to be bothered how many
> different output devices you have and control each individually, you just
> want an area to draw and to know/control what area the user is expected to
> see and that's already provided in fbdev.

Users want to be able to dynamically change their display config on the fly.

> If the user wants the same content on multiple outputs just configure the
> driver to do so.

KMS provide an API to do that and a nice internal abstraction for handling it.

> If he wants different (independent) content on each output, just provide
> multiple /dev/fbX devices. I admit that we could use a controlling interface
> here that decides which user (application) might draw at a time to the
> interface which they currently only do if they are the active VT.
> If you want 2 or more outputs to be merged as one just configure this in the
> driver.
> The only thing that is impossible to do in fbdev is controlling 2 or more
> independent display outputs that access the same buffer. But that's not an
> issue I think.
> The things above only could use a unification of how to set them up on
> module load time (as only limited runtime changes are permited given that we
> must always be able to support a mode that we once entered during runtime).
>

What about changing outputs on the fly (turn off VGA, turn on DVI,
switch between multi-head and single-head, etc) or encoders shared
between multiple connectors (think a single dac shared between a VGA
and a TV port); how do you expose them easily as separate fbdevs?
Lots of stuff is doable with fbdev, but it's nicer with kms.

Alex

> The thing that's really missing in fbdev is a way to allow hardware
> acceleration for userspace.
>
>
> Regards,
>
> Florian Tobias Schandinat
>
>

Re: Idea of a v4l -> fb interface driver

[Florian Tobias Schandinat]

Guennadi Liakhovetski schrieb:
> On Fri, 28 May 2010, Florian Tobias Schandinat wrote:
> 
>> Well hiding complexity is actually the job of an API. I don't see any need for
>> major changes in fbdev for complex display setups. In most cases as a
>> userspace application you really don't want to be bothered how many different
>> output devices you have and control each individually, you just want an area
>> to draw and to know/control what area the user is expected to see and that's
>> already provided in fbdev.
>> If the user wants the same content on multiple outputs just configure the
>> driver to do so.
>> If he wants different (independent) content on each output, just provide
>> multiple /dev/fbX devices. I admit that we could use a controlling interface
>> here that decides which user (application) might draw at a time to the
>> interface which they currently only do if they are the active VT.
>> If you want 2 or more outputs to be merged as one just configure this in the
>> driver.
>> The only thing that is impossible to do in fbdev is controlling 2 or more
>> independent display outputs that access the same buffer. But that's not an
>> issue I think.
>> The things above only could use a unification of how to set them up on module
>> load time (as only limited runtime changes are permited given that we must
>> always be able to support a mode that we once entered during runtime).
>>
>> The thing that's really missing in fbdev is a way to allow hardware
>> acceleration for userspace.
> 
> How about a "simple" use-case, that I asked about in another my mail: how 
> do you inform fbdev users, if a (DVI) display has been disconnected and 
> another one with a different resolution has been connected?

Yes that's a problem. The thing is that the virtual terminal requires us 
to always be able to switch to a resolution we once supported. Probably 
what I would do in such a case is switching the screen off and let the 
user figure out that he has done something "wrong". As we don't really 
know our users (applications) there is not much we can do but wait for 
the next check_var to solve this. So yes things that force us to be 
incompatible with our previous behaviour can do some harm if the user is 
not aware of it.

Note 1: Interesting that you mentioned viafb, the driver that is 
currently nearly completely incapable to determine any output device 
limitations.

Note 2: set_par returns a value but that's rather a mistake and we can't 
rely anyone to react on a sane way on error. check_var is the only place 
where framebuffers can say that they don't support something after this 
they have to support it regardless of any external events.


Thanks,

Florian Tobias Schandinat

Re: Idea of a v4l -> fb interface driver

[Florian Tobias Schandinat]

Guennadi Liakhovetski schrieb:
> On Fri, 28 May 2010, Florian Tobias Schandinat wrote:
> 
>> Well hiding complexity is actually the job of an API. I don't see any need for
>> major changes in fbdev for complex display setups. In most cases as a
>> userspace application you really don't want to be bothered how many different
>> output devices you have and control each individually, you just want an area
>> to draw and to know/control what area the user is expected to see and that's
>> already provided in fbdev.
>> If the user wants the same content on multiple outputs just configure the
>> driver to do so.
>> If he wants different (independent) content on each output, just provide
>> multiple /dev/fbX devices. I admit that we could use a controlling interface
>> here that decides which user (application) might draw at a time to the
>> interface which they currently only do if they are the active VT.
>> If you want 2 or more outputs to be merged as one just configure this in the
>> driver.
>> The only thing that is impossible to do in fbdev is controlling 2 or more
>> independent display outputs that access the same buffer. But that's not an
>> issue I think.
>> The things above only could use a unification of how to set them up on module
>> load time (as only limited runtime changes are permited given that we must
>> always be able to support a mode that we once entered during runtime).
>>
>> The thing that's really missing in fbdev is a way to allow hardware
>> acceleration for userspace.
> 
> How about a "simple" use-case, that I asked about in another my mail: how 
> do you inform fbdev users, if a (DVI) display has been disconnected and 
> another one with a different resolution has been connected?

Yes that's a problem. The thing is that the virtual terminal requires us 
to always be able to switch to a resolution we once supported. Probably 
what I would do in such a case is switching the screen off and let the 
user figure out that he has done something "wrong". As we don't really 
know our users (applications) there is not much we can do but wait for 
the next check_var to solve this. So yes things that force us to be 
incompatible with our previous behaviour can do some harm if the user is 
not aware of it.

Note 1: Interesting that you mentioned viafb, the driver that is 
currently nearly completely incapable to determine any output device 
limitations.

Note 2: set_par returns a value but that's rather a mistake and we can't 
rely anyone to react on a sane way on error. check_var is the only place 
where framebuffers can say that they don't support something after this 
they have to support it regardless of any external events.


Thanks,

Florian Tobias Schandinat

Re: Idea of a v4l -> fb interface driver

[Ville Syrjälä]

On Fri, May 28, 2010 at 03:41:46PM -0400, Alex Deucher wrote:
> On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat
> > If he wants different (independent) content on each output, just provide
> > multiple /dev/fbX devices. I admit that we could use a controlling interface
> > here that decides which user (application) might draw at a time to the
> > interface which they currently only do if they are the active VT.
> > If you want 2 or more outputs to be merged as one just configure this in the
> > driver.
> > The only thing that is impossible to do in fbdev is controlling 2 or more
> > independent display outputs that access the same buffer. But that's not an
> > issue I think.
> > The things above only could use a unification of how to set them up on
> > module load time (as only limited runtime changes are permited given that we
> > must always be able to support a mode that we once entered during runtime).
> >
> 
> What about changing outputs on the fly (turn off VGA, turn on DVI,
> switch between multi-head and single-head, etc) or encoders shared
> between multiple connectors (think a single dac shared between a VGA
> and a TV port); how do you expose them easily as separate fbdevs?
> Lots of stuff is doable with fbdev, but it's nicer with kms.

But actually getting your data onto the screen is a lot easier with
fbdev. There's no standard API in drm to actually allocate the
framebuffer and manipulate it. You always need a user space driver
to go along with the kernel bits.

I'm not saying fbdev is better than drm/kms but at least it can be
used to write simple applications that work across different
hardware. Perhaps that's something that should be addressed in the
drm API.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: Idea of a v4l -> fb interface driver

[Ville Syrjälä]

On Fri, May 28, 2010 at 03:41:46PM -0400, Alex Deucher wrote:
> On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat
> > If he wants different (independent) content on each output, just provide
> > multiple /dev/fbX devices. I admit that we could use a controlling interface
> > here that decides which user (application) might draw at a time to the
> > interface which they currently only do if they are the active VT.
> > If you want 2 or more outputs to be merged as one just configure this in the
> > driver.
> > The only thing that is impossible to do in fbdev is controlling 2 or more
> > independent display outputs that access the same buffer. But that's not an
> > issue I think.
> > The things above only could use a unification of how to set them up on
> > module load time (as only limited runtime changes are permited given that we
> > must always be able to support a mode that we once entered during runtime).
> >
> 
> What about changing outputs on the fly (turn off VGA, turn on DVI,
> switch between multi-head and single-head, etc) or encoders shared
> between multiple connectors (think a single dac shared between a VGA
> and a TV port); how do you expose them easily as separate fbdevs?
> Lots of stuff is doable with fbdev, but it's nicer with kms.

But actually getting your data onto the screen is a lot easier with
fbdev. There's no standard API in drm to actually allocate the
framebuffer and manipulate it. You always need a user space driver
to go along with the kernel bits.

I'm not saying fbdev is better than drm/kms but at least it can be
used to write simple applications that work across different
hardware. Perhaps that's something that should be addressed in the
drm API.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: Idea of a v4l -> fb interface driver

[Dave Airlie]

On Sat, May 29, 2010 at 6:06 AM, Ville Syrjälä <syrjala@sci.fi> wrote:
> On Fri, May 28, 2010 at 03:41:46PM -0400, Alex Deucher wrote:
>> On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat
>> > If he wants different (independent) content on each output, just provide
>> > multiple /dev/fbX devices. I admit that we could use a controlling interface
>> > here that decides which user (application) might draw at a time to the
>> > interface which they currently only do if they are the active VT.
>> > If you want 2 or more outputs to be merged as one just configure this in the
>> > driver.
>> > The only thing that is impossible to do in fbdev is controlling 2 or more
>> > independent display outputs that access the same buffer. But that's not an
>> > issue I think.
>> > The things above only could use a unification of how to set them up on
>> > module load time (as only limited runtime changes are permited given that we
>> > must always be able to support a mode that we once entered during runtime).
>> >
>>
>> What about changing outputs on the fly (turn off VGA, turn on DVI,
>> switch between multi-head and single-head, etc) or encoders shared
>> between multiple connectors (think a single dac shared between a VGA
>> and a TV port); how do you expose them easily as separate fbdevs?
>> Lots of stuff is doable with fbdev, but it's nicer with kms.
>
> But actually getting your data onto the screen is a lot easier with
> fbdev. There's no standard API in drm to actually allocate the
> framebuffer and manipulate it. You always need a user space driver
> to go along with the kernel bits.
>
> I'm not saying fbdev is better than drm/kms but at least it can be
> used to write simple applications that work across different
> hardware. Perhaps that's something that should be addressed in the
> drm API.
>

http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg48461.html

I started writing a "dumb" API for KMS, it got stuck on whether to
expose cursors or not, I must dig out the branch.

It basically was a create + map API. I'll see if I can get it finished off.

The main reason we avoided a fully generic interface is there isn't
really a generic way to abstract acceleration on a modern GPU, and
buffer allocation on most modern GPUs doesn't want a linear simple
buffer. We felt doing a compromised generic interface would lead
people down the wrong path into believing they could easily move from
the "dumb" interface to a real accelerated one.

There is a userspace library called libkms that abstracts this stuff,
but I'd like to just have the kernel do it.

Dave.

Re: Idea of a v4l -> fb interface driver

[Dave Airlie]

On Sat, May 29, 2010 at 6:06 AM, Ville Syrjälä <syrjala@sci.fi> wrote:
> On Fri, May 28, 2010 at 03:41:46PM -0400, Alex Deucher wrote:
>> On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat
>> > If he wants different (independent) content on each output, just provide
>> > multiple /dev/fbX devices. I admit that we could use a controlling interface
>> > here that decides which user (application) might draw at a time to the
>> > interface which they currently only do if they are the active VT.
>> > If you want 2 or more outputs to be merged as one just configure this in the
>> > driver.
>> > The only thing that is impossible to do in fbdev is controlling 2 or more
>> > independent display outputs that access the same buffer. But that's not an
>> > issue I think.
>> > The things above only could use a unification of how to set them up on
>> > module load time (as only limited runtime changes are permited given that we
>> > must always be able to support a mode that we once entered during runtime).
>> >
>>
>> What about changing outputs on the fly (turn off VGA, turn on DVI,
>> switch between multi-head and single-head, etc) or encoders shared
>> between multiple connectors (think a single dac shared between a VGA
>> and a TV port); how do you expose them easily as separate fbdevs?
>> Lots of stuff is doable with fbdev, but it's nicer with kms.
>
> But actually getting your data onto the screen is a lot easier with
> fbdev. There's no standard API in drm to actually allocate the
> framebuffer and manipulate it. You always need a user space driver
> to go along with the kernel bits.
>
> I'm not saying fbdev is better than drm/kms but at least it can be
> used to write simple applications that work across different
> hardware. Perhaps that's something that should be addressed in the
> drm API.
>

http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg48461.html

I started writing a "dumb" API for KMS, it got stuck on whether to
expose cursors or not, I must dig out the branch.

It basically was a create + map API. I'll see if I can get it finished off.

The main reason we avoided a fully generic interface is there isn't
really a generic way to abstract acceleration on a modern GPU, and
buffer allocation on most modern GPUs doesn't want a linear simple
buffer. We felt doing a compromised generic interface would lead
people down the wrong path into believing they could easily move from
the "dumb" interface to a real accelerated one.

There is a userspace library called libkms that abstracts this stuff,
but I'd like to just have the kernel do it.

Dave.