All of lore.kernel.org
 help / color / mirror / Atom feed
* Audio Video synchronization for data received from a HDMI receiver chip
@ 2011-05-12  5:48 Bhupesh SHARMA
  2011-05-12 16:59 ` Charlie X. Liu
  0 siblings, 1 reply; 10+ messages in thread
From: Bhupesh SHARMA @ 2011-05-12  5:48 UTC (permalink / raw)
  To: linux-media; +Cc: Laurent Pinchart, Guennadi Liakhovetski, Hans Verkuil

Hi Linux media folks,

We are considering putting an advanced HDMI receiver chip on our SoC,
to allow reception of HDMI audio and video. The chip receives HDMI data
from a host like a set-up box or DVD player. It provides a video data interface
and SPDIF/I2S audio data interface.

We plan to support the HDMI video using the V4L2 framework and the HDMI
audio using ALSA framework.

Now, what seems to be intriguing us is how the audio-video synchronization
will be maintained? Will a separate bridging entity required to ensure the same
or whether this can be left upon a user space application like mplayer or gstreamer.

Also is there a existing interface between the V4L2 and ALSA frameworks and the same
can be used in our design?

Regards,
Bhupesh
ST Microelectronics

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: Audio Video synchronization for data received from a HDMI receiver chip
  2011-05-12  5:48 Audio Video synchronization for data received from a HDMI receiver chip Bhupesh SHARMA
@ 2011-05-12 16:59 ` Charlie X. Liu
  2011-05-12 20:29   ` Hans Verkuil
  0 siblings, 1 reply; 10+ messages in thread
From: Charlie X. Liu @ 2011-05-12 16:59 UTC (permalink / raw)
  To: 'Bhupesh SHARMA', linux-media
  Cc: 'Laurent Pinchart', 'Guennadi Liakhovetski',
	'Hans Verkuil'

Which HDMI receiver chip?

-----Original Message-----
From: linux-media-owner@vger.kernel.org
[mailto:linux-media-owner@vger.kernel.org] On Behalf Of Bhupesh SHARMA
Sent: Wednesday, May 11, 2011 10:49 PM
To: linux-media@vger.kernel.org
Cc: Laurent Pinchart; Guennadi Liakhovetski; Hans Verkuil
Subject: Audio Video synchronization for data received from a HDMI receiver
chip

Hi Linux media folks,

We are considering putting an advanced HDMI receiver chip on our SoC,
to allow reception of HDMI audio and video. The chip receives HDMI data
from a host like a set-up box or DVD player. It provides a video data
interface
and SPDIF/I2S audio data interface.

We plan to support the HDMI video using the V4L2 framework and the HDMI
audio using ALSA framework.

Now, what seems to be intriguing us is how the audio-video synchronization
will be maintained? Will a separate bridging entity required to ensure the
same
or whether this can be left upon a user space application like mplayer or
gstreamer.

Also is there a existing interface between the V4L2 and ALSA frameworks and
the same
can be used in our design?

Regards,
Bhupesh
ST Microelectronics
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Audio Video synchronization for data received from a HDMI receiver chip
  2011-05-12 16:59 ` Charlie X. Liu
@ 2011-05-12 20:29   ` Hans Verkuil
       [not found]     ` <BANLkTi=rpQEkroia3kUqp6zUHTQk3k220Q@mail.gmail.com>
  0 siblings, 1 reply; 10+ messages in thread
From: Hans Verkuil @ 2011-05-12 20:29 UTC (permalink / raw)
  To: Charlie X. Liu
  Cc: 'Bhupesh SHARMA', linux-media, 'Laurent Pinchart',
	'Guennadi Liakhovetski'

On Thursday, May 12, 2011 18:59:33 Charlie X. Liu wrote:
> Which HDMI receiver chip?

Indeed, that's my question as well :-)

Anyway, this question comes up regularly. V4L2 provides timestamps for each
frame, so that's no problem. But my understanding is that ALSA does not give
you timestamps, so if there are processing delays between audio and video, then
you have no way of knowing. The obvious solution is to talk to the ALSA people
to see if some sort of timestamping is possible, but nobody has done that.

This is either because everyone that needs it hacks around it instead of trying
to really solve it, or because it is never a problem in practice.

Hope this helps!

Regards,

	Hans

> 
> -----Original Message-----
> From: linux-media-owner@vger.kernel.org
> [mailto:linux-media-owner@vger.kernel.org] On Behalf Of Bhupesh SHARMA
> Sent: Wednesday, May 11, 2011 10:49 PM
> To: linux-media@vger.kernel.org
> Cc: Laurent Pinchart; Guennadi Liakhovetski; Hans Verkuil
> Subject: Audio Video synchronization for data received from a HDMI receiver
> chip
> 
> Hi Linux media folks,
> 
> We are considering putting an advanced HDMI receiver chip on our SoC,
> to allow reception of HDMI audio and video. The chip receives HDMI data
> from a host like a set-up box or DVD player. It provides a video data
> interface
> and SPDIF/I2S audio data interface.
> 
> We plan to support the HDMI video using the V4L2 framework and the HDMI
> audio using ALSA framework.
> 
> Now, what seems to be intriguing us is how the audio-video synchronization
> will be maintained? Will a separate bridging entity required to ensure the
> same
> or whether this can be left upon a user space application like mplayer or
> gstreamer.
> 
> Also is there a existing interface between the V4L2 and ALSA frameworks and
> the same
> can be used in our design?
> 
> Regards,
> Bhupesh
> ST Microelectronics
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: Audio Video synchronization for data received from a HDMI receiver chip
       [not found]     ` <BANLkTi=rpQEkroia3kUqp6zUHTQk3k220Q@mail.gmail.com>
@ 2011-05-18  4:10       ` Bhupesh SHARMA
  2011-05-18  6:32         ` Hans Verkuil
  0 siblings, 1 reply; 10+ messages in thread
From: Bhupesh SHARMA @ 2011-05-18  4:10 UTC (permalink / raw)
  To: Charlie X. Liu, Hans Verkuil
  Cc: laurent.pinchart, g.liakhovetski, linux-media, alsa-devel

Hi,

(adding alsa mailing list in cc)

> On Thursday, May 12, 2011 18:59:33 Charlie X. Liu wrote:
> > Which HDMI receiver chip?
> 
> Indeed, that's my question as well :-)

We use Sil 9135 receiver chip which is provided by Silicon Image.
Please see details here: http://www.siliconimage.com/products/product.aspx?pid=109
 
> Anyway, this question comes up regularly. V4L2 provides timestamps for
> each
> frame, so that's no problem. But my understanding is that ALSA does not
> give
> you timestamps, so if there are processing delays between audio and
> video, then
> you have no way of knowing. The obvious solution is to talk to the ALSA
> people
> to see if some sort of timestamping is possible, but nobody has done
> that.

I am aware of the time stamping feature provided by V4L2, but I am also
not sure whether the same feature is supported by ALSA. I have included
alsa-mailing list also in copy of this mail. Let's see if we can get
some sort of confirmation on this from them.
 
> This is either because everyone that needs it hacks around it instead
> of trying
> to really solve it, or because it is never a problem in practice.

What should be the proper solution according to you to solve this issue.
Do we require a Audio-Video Bridge kind of utility/mechanism?

Regards,
Bhupesh

> 
> >
> > -----Original Message-----
> > From: linux-media-owner@vger.kernel.org
> > [mailto:linux-media-owner@vger.kernel.org] On Behalf Of Bhupesh
> SHARMA
> > Sent: Wednesday, May 11, 2011 10:49 PM
> > To: linux-media@vger.kernel.org
> > Cc: Laurent Pinchart; Guennadi Liakhovetski; Hans Verkuil
> > Subject: Audio Video synchronization for data received from a HDMI
> receiver
> > chip
> >
> > Hi Linux media folks,
> >
> > We are considering putting an advanced HDMI receiver chip on our SoC,
> > to allow reception of HDMI audio and video. The chip receives HDMI
> data
> > from a host like a set-up box or DVD player. It provides a video data
> > interface
> > and SPDIF/I2S audio data interface.
> >
> > We plan to support the HDMI video using the V4L2 framework and the
> HDMI
> > audio using ALSA framework.
> >
> > Now, what seems to be intriguing us is how the audio-video
> synchronization
> > will be maintained? Will a separate bridging entity required to
> ensure the
> > same
> > or whether this can be left upon a user space application like
> mplayer or
> > gstreamer.
> >
> > Also is there a existing interface between the V4L2 and ALSA
> frameworks and
> > the same
> > can be used in our design?
> >
> > Regards,
> > Bhupesh
> > ST Microelectronics
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-
> media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> --
> regards
> Shiraz Hashim

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Audio Video synchronization for data received from a HDMI receiver chip
  2011-05-18  4:10       ` Bhupesh SHARMA
@ 2011-05-18  6:32         ` Hans Verkuil
  2011-05-19  5:39           ` Bhupesh SHARMA
  0 siblings, 1 reply; 10+ messages in thread
From: Hans Verkuil @ 2011-05-18  6:32 UTC (permalink / raw)
  To: Bhupesh SHARMA
  Cc: Charlie X. Liu, laurent.pinchart, g.liakhovetski, linux-media,
	alsa-devel

On Wednesday, May 18, 2011 06:10:43 Bhupesh SHARMA wrote:
> Hi,
> 
> (adding alsa mailing list in cc)
> 
> > On Thursday, May 12, 2011 18:59:33 Charlie X. Liu wrote:
> > > Which HDMI receiver chip?
> > 
> > Indeed, that's my question as well :-)
> 
> We use Sil 9135 receiver chip which is provided by Silicon Image.
> Please see details here: http://www.siliconimage.com/products/product.aspx?pid=109
>  
> > Anyway, this question comes up regularly. V4L2 provides timestamps for
> > each
> > frame, so that's no problem. But my understanding is that ALSA does not
> > give
> > you timestamps, so if there are processing delays between audio and
> > video, then
> > you have no way of knowing. The obvious solution is to talk to the ALSA
> > people
> > to see if some sort of timestamping is possible, but nobody has done
> > that.
> 
> I am aware of the time stamping feature provided by V4L2, but I am also
> not sure whether the same feature is supported by ALSA. I have included
> alsa-mailing list also in copy of this mail. Let's see if we can get
> some sort of confirmation on this from them.
>  
> > This is either because everyone that needs it hacks around it instead
> > of trying
> > to really solve it, or because it is never a problem in practice.
> 
> What should be the proper solution according to you to solve this issue.
> Do we require a Audio-Video Bridge kind of utility/mechanism?

I don't believe so. All you need is reliable time stamping for your audio
and video streams. That's enough for userspace to detect AV sync issues.

Regards,

	Hans

> 
> Regards,
> Bhupesh
> 
> > 
> > >
> > > -----Original Message-----
> > > From: linux-media-owner@vger.kernel.org
> > > [mailto:linux-media-owner@vger.kernel.org] On Behalf Of Bhupesh
> > SHARMA
> > > Sent: Wednesday, May 11, 2011 10:49 PM
> > > To: linux-media@vger.kernel.org
> > > Cc: Laurent Pinchart; Guennadi Liakhovetski; Hans Verkuil
> > > Subject: Audio Video synchronization for data received from a HDMI
> > receiver
> > > chip
> > >
> > > Hi Linux media folks,
> > >
> > > We are considering putting an advanced HDMI receiver chip on our SoC,
> > > to allow reception of HDMI audio and video. The chip receives HDMI
> > data
> > > from a host like a set-up box or DVD player. It provides a video data
> > > interface
> > > and SPDIF/I2S audio data interface.
> > >
> > > We plan to support the HDMI video using the V4L2 framework and the
> > HDMI
> > > audio using ALSA framework.
> > >
> > > Now, what seems to be intriguing us is how the audio-video
> > synchronization
> > > will be maintained? Will a separate bridging entity required to
> > ensure the
> > > same
> > > or whether this can be left upon a user space application like
> > mplayer or
> > > gstreamer.
> > >
> > > Also is there a existing interface between the V4L2 and ALSA
> > frameworks and
> > > the same
> > > can be used in our design?
> > >
> > > Regards,
> > > Bhupesh
> > > ST Microelectronics
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-
> > media" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > >
> > >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media"
> > in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> > 
> > 
> > --
> > regards
> > Shiraz Hashim
> 
> 

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: Audio Video synchronization for data received from a HDMI receiver chip
  2011-05-18  6:32         ` Hans Verkuil
@ 2011-05-19  5:39           ` Bhupesh SHARMA
  2011-05-19  6:55               ` Hans Verkuil
  0 siblings, 1 reply; 10+ messages in thread
From: Bhupesh SHARMA @ 2011-05-19  5:39 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Charlie X. Liu, laurent.pinchart, g.liakhovetski, linux-media,
	alsa-devel

> On Wednesday, May 18, 2011 06:10:43 Bhupesh SHARMA wrote:
> > Hi,
> >
> > (adding alsa mailing list in cc)
> >
> > > On Thursday, May 12, 2011 18:59:33 Charlie X. Liu wrote:
> > > > Which HDMI receiver chip?
> > >
> > > Indeed, that's my question as well :-)
> >
> > We use Sil 9135 receiver chip which is provided by Silicon Image.
> > Please see details here:
> http://www.siliconimage.com/products/product.aspx?pid=109
> >
> > > Anyway, this question comes up regularly. V4L2 provides timestamps
> for
> > > each
> > > frame, so that's no problem. But my understanding is that ALSA does
> not
> > > give
> > > you timestamps, so if there are processing delays between audio and
> > > video, then
> > > you have no way of knowing. The obvious solution is to talk to the
> ALSA
> > > people
> > > to see if some sort of timestamping is possible, but nobody has
> done
> > > that.
> >
> > I am aware of the time stamping feature provided by V4L2, but I am
> also
> > not sure whether the same feature is supported by ALSA. I have
> included
> > alsa-mailing list also in copy of this mail. Let's see if we can get
> > some sort of confirmation on this from them.
> >
> > > This is either because everyone that needs it hacks around it
> instead
> > > of trying
> > > to really solve it, or because it is never a problem in practice.
> >
> > What should be the proper solution according to you to solve this
> issue.
> > Do we require a Audio-Video Bridge kind of utility/mechanism?
> 
> I don't believe so. All you need is reliable time stamping for your
> audio
> and video streams. That's enough for userspace to detect AV sync
> issues.
> 

Hi Hans,

I have another doubt regarding the framework choice for the entire
system that I have, especially the video part of the system. The overall
system is similar to the one depicted below:

HDMI data --> HDMI receiver chip --> Video Port IP on SoC --> System DDR

HDMI data is received from external world (from say a set-up box or dvd player),
which is fed to the HDMI receiver chip on-board and then parallel data lines feed
this data to a Video Port IP on the SoC which has a DMA master interface and
hence can push the data thus received directly on system DDR.

Now, I can figure out that there will be two drivers required here:
# HDMI receiver chip driver (which is essentially a v4l2 subdev being controller via I2C)
# Video Port driver (which is a v4l2 bridge driver)

Is my understanding correct?
Are there any HDMI receiver subdev driver and video bridge driver already available which I can 
use for reference?

Also will the audio ALSA driver fit as a subdev driver in the entire system?

P.S. I use the terminology 'bridge' and 'subdev' as mentioned in 'v4l2-subdev.h' header file.
Regards,
Bhupesh

> Regards,
> 
> 	Hans
> 
> >
> > Regards,
> > Bhupesh
> >
> > >
> > > >
> > > > -----Original Message-----
> > > > From: linux-media-owner@vger.kernel.org
> > > > [mailto:linux-media-owner@vger.kernel.org] On Behalf Of Bhupesh
> > > SHARMA
> > > > Sent: Wednesday, May 11, 2011 10:49 PM
> > > > To: linux-media@vger.kernel.org
> > > > Cc: Laurent Pinchart; Guennadi Liakhovetski; Hans Verkuil
> > > > Subject: Audio Video synchronization for data received from a
> HDMI
> > > receiver
> > > > chip
> > > >
> > > > Hi Linux media folks,
> > > >
> > > > We are considering putting an advanced HDMI receiver chip on our
> SoC,
> > > > to allow reception of HDMI audio and video. The chip receives
> HDMI
> > > data
> > > > from a host like a set-up box or DVD player. It provides a video
> data
> > > > interface
> > > > and SPDIF/I2S audio data interface.
> > > >
> > > > We plan to support the HDMI video using the V4L2 framework and
> the
> > > HDMI
> > > > audio using ALSA framework.
> > > >
> > > > Now, what seems to be intriguing us is how the audio-video
> > > synchronization
> > > > will be maintained? Will a separate bridging entity required to
> > > ensure the
> > > > same
> > > > or whether this can be left upon a user space application like
> > > mplayer or
> > > > gstreamer.
> > > >
> > > > Also is there a existing interface between the V4L2 and ALSA
> > > frameworks and
> > > > the same
> > > > can be used in our design?
> > > >
> > > > Regards,
> > > > Bhupesh
> > > > ST Microelectronics
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe linux-
> > > media" in
> > > > the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at  http://vger.kernel.org/majordomo-
> info.html
> > > >
> > > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-
> media"
> > > in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > >
> > >
> > >
> > > --
> > > regards
> > > Shiraz Hashim
> >
> >

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Audio Video synchronization for data received from a HDMI receiver chip
  2011-05-19  5:39           ` Bhupesh SHARMA
@ 2011-05-19  6:55               ` Hans Verkuil
  0 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2011-05-19  6:55 UTC (permalink / raw)
  To: Bhupesh SHARMA
  Cc: Charlie X. Liu, laurent.pinchart, g.liakhovetski, linux-media,
	alsa-devel

On Thursday, May 19, 2011 07:39:32 Bhupesh SHARMA wrote:
> > On Wednesday, May 18, 2011 06:10:43 Bhupesh SHARMA wrote:
> > > Hi,
> > >
> > > (adding alsa mailing list in cc)
> > >
> > > > On Thursday, May 12, 2011 18:59:33 Charlie X. Liu wrote:
> > > > > Which HDMI receiver chip?
> > > >
> > > > Indeed, that's my question as well :-)
> > >
> > > We use Sil 9135 receiver chip which is provided by Silicon Image.
> > > Please see details here:
> > http://www.siliconimage.com/products/product.aspx?pid=109
> > >
> > > > Anyway, this question comes up regularly. V4L2 provides timestamps
> > for
> > > > each
> > > > frame, so that's no problem. But my understanding is that ALSA does
> > not
> > > > give
> > > > you timestamps, so if there are processing delays between audio and
> > > > video, then
> > > > you have no way of knowing. The obvious solution is to talk to the
> > ALSA
> > > > people
> > > > to see if some sort of timestamping is possible, but nobody has
> > done
> > > > that.
> > >
> > > I am aware of the time stamping feature provided by V4L2, but I am
> > also
> > > not sure whether the same feature is supported by ALSA. I have
> > included
> > > alsa-mailing list also in copy of this mail. Let's see if we can get
> > > some sort of confirmation on this from them.
> > >
> > > > This is either because everyone that needs it hacks around it
> > instead
> > > > of trying
> > > > to really solve it, or because it is never a problem in practice.
> > >
> > > What should be the proper solution according to you to solve this
> > issue.
> > > Do we require a Audio-Video Bridge kind of utility/mechanism?
> > 
> > I don't believe so. All you need is reliable time stamping for your
> > audio
> > and video streams. That's enough for userspace to detect AV sync
> > issues.
> > 
> 
> Hi Hans,
> 
> I have another doubt regarding the framework choice for the entire
> system that I have, especially the video part of the system. The overall
> system is similar to the one depicted below:
> 
> HDMI data --> HDMI receiver chip --> Video Port IP on SoC --> System DDR
> 
> HDMI data is received from external world (from say a set-up box or dvd player),
> which is fed to the HDMI receiver chip on-board and then parallel data lines feed
> this data to a Video Port IP on the SoC which has a DMA master interface and
> hence can push the data thus received directly on system DDR.
> 
> Now, I can figure out that there will be two drivers required here:
> # HDMI receiver chip driver (which is essentially a v4l2 subdev being controller via I2C)
> # Video Port driver (which is a v4l2 bridge driver)
> 
> Is my understanding correct?

Yes.

> Are there any HDMI receiver subdev driver and video bridge driver already available which I can 
> use for reference?

Video bridge drivers are easier: examples are in drivers/media/video/s5p-fimc
or in drivers/media/video/davinci. Note that you should use the new videobuf2
framework instead of the older videobuf framework. s5p-fimc is using vb2 already.
but the vpif capture and display drivers in the davinci directory do not.

With regards to HDMI receivers: these are still under development. One example
is here: http://git.linuxtv.org/hverkuil/cisco.git?a=shortlog;h=refs/heads/cobalt

This tree contains a driver for the adv7604 HDMI/Graphics receiver. It is fairly
simplistic at the moment, our internal driver is developed a lot further but I
haven't had the chance yet to update the git tree with our latest code (Cisco
is developing this driver). In addition the HDMI API for V4L2 is still under
development. It requires some V4L2 core support to be merged first (control events)
before we can continue with that.

> 
> Also will the audio ALSA driver fit as a subdev driver in the entire system?

No, although I have heard that the ALSA developers are looking at a subdev-like
approach. There are several V4L drivers that support ALSA as a separate driver
(cx18 for example). This is usually not a problem.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Audio Video synchronization for data received from a HDMI receiver chip
@ 2011-05-19  6:55               ` Hans Verkuil
  0 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2011-05-19  6:55 UTC (permalink / raw)
  To: Bhupesh SHARMA
  Cc: alsa-devel, Charlie X. Liu, linux-media, laurent.pinchart,
	g.liakhovetski

On Thursday, May 19, 2011 07:39:32 Bhupesh SHARMA wrote:
> > On Wednesday, May 18, 2011 06:10:43 Bhupesh SHARMA wrote:
> > > Hi,
> > >
> > > (adding alsa mailing list in cc)
> > >
> > > > On Thursday, May 12, 2011 18:59:33 Charlie X. Liu wrote:
> > > > > Which HDMI receiver chip?
> > > >
> > > > Indeed, that's my question as well :-)
> > >
> > > We use Sil 9135 receiver chip which is provided by Silicon Image.
> > > Please see details here:
> > http://www.siliconimage.com/products/product.aspx?pid=109
> > >
> > > > Anyway, this question comes up regularly. V4L2 provides timestamps
> > for
> > > > each
> > > > frame, so that's no problem. But my understanding is that ALSA does
> > not
> > > > give
> > > > you timestamps, so if there are processing delays between audio and
> > > > video, then
> > > > you have no way of knowing. The obvious solution is to talk to the
> > ALSA
> > > > people
> > > > to see if some sort of timestamping is possible, but nobody has
> > done
> > > > that.
> > >
> > > I am aware of the time stamping feature provided by V4L2, but I am
> > also
> > > not sure whether the same feature is supported by ALSA. I have
> > included
> > > alsa-mailing list also in copy of this mail. Let's see if we can get
> > > some sort of confirmation on this from them.
> > >
> > > > This is either because everyone that needs it hacks around it
> > instead
> > > > of trying
> > > > to really solve it, or because it is never a problem in practice.
> > >
> > > What should be the proper solution according to you to solve this
> > issue.
> > > Do we require a Audio-Video Bridge kind of utility/mechanism?
> > 
> > I don't believe so. All you need is reliable time stamping for your
> > audio
> > and video streams. That's enough for userspace to detect AV sync
> > issues.
> > 
> 
> Hi Hans,
> 
> I have another doubt regarding the framework choice for the entire
> system that I have, especially the video part of the system. The overall
> system is similar to the one depicted below:
> 
> HDMI data --> HDMI receiver chip --> Video Port IP on SoC --> System DDR
> 
> HDMI data is received from external world (from say a set-up box or dvd player),
> which is fed to the HDMI receiver chip on-board and then parallel data lines feed
> this data to a Video Port IP on the SoC which has a DMA master interface and
> hence can push the data thus received directly on system DDR.
> 
> Now, I can figure out that there will be two drivers required here:
> # HDMI receiver chip driver (which is essentially a v4l2 subdev being controller via I2C)
> # Video Port driver (which is a v4l2 bridge driver)
> 
> Is my understanding correct?

Yes.

> Are there any HDMI receiver subdev driver and video bridge driver already available which I can 
> use for reference?

Video bridge drivers are easier: examples are in drivers/media/video/s5p-fimc
or in drivers/media/video/davinci. Note that you should use the new videobuf2
framework instead of the older videobuf framework. s5p-fimc is using vb2 already.
but the vpif capture and display drivers in the davinci directory do not.

With regards to HDMI receivers: these are still under development. One example
is here: http://git.linuxtv.org/hverkuil/cisco.git?a=shortlog;h=refs/heads/cobalt

This tree contains a driver for the adv7604 HDMI/Graphics receiver. It is fairly
simplistic at the moment, our internal driver is developed a lot further but I
haven't had the chance yet to update the git tree with our latest code (Cisco
is developing this driver). In addition the HDMI API for V4L2 is still under
development. It requires some V4L2 core support to be merged first (control events)
before we can continue with that.

> 
> Also will the audio ALSA driver fit as a subdev driver in the entire system?

No, although I have heard that the ALSA developers are looking at a subdev-like
approach. There are several V4L drivers that support ALSA as a separate driver
(cx18 for example). This is usually not a problem.

Regards,

	Hans

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: Audio Video synchronization for data received from a HDMI receiver chip
  2011-05-19  6:55               ` Hans Verkuil
  (?)
@ 2011-05-19 11:10               ` Bhupesh SHARMA
  2011-05-19 11:38                 ` Hans Verkuil
  -1 siblings, 1 reply; 10+ messages in thread
From: Bhupesh SHARMA @ 2011-05-19 11:10 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Charlie X. Liu, laurent.pinchart, g.liakhovetski, linux-media,
	alsa-devel

> > Hi Hans,
> >
> > I have another doubt regarding the framework choice for the entire
> > system that I have, especially the video part of the system. The
> overall
> > system is similar to the one depicted below:
> >
> > HDMI data --> HDMI receiver chip --> Video Port IP on SoC --> System
> DDR
> >
> > HDMI data is received from external world (from say a set-up box or
> dvd player),
> > which is fed to the HDMI receiver chip on-board and then parallel
> data lines feed
> > this data to a Video Port IP on the SoC which has a DMA master
> interface and
> > hence can push the data thus received directly on system DDR.
> >
> > Now, I can figure out that there will be two drivers required here:
> > # HDMI receiver chip driver (which is essentially a v4l2 subdev being
> controller via I2C)
> > # Video Port driver (which is a v4l2 bridge driver)
> >
> > Is my understanding correct?
> 
> Yes.

Thanks for clarifying this.

> > Are there any HDMI receiver subdev driver and video bridge driver
> already available which I can
> > use for reference?
> 
> Video bridge drivers are easier: examples are in
> drivers/media/video/s5p-fimc
> or in drivers/media/video/davinci. Note that you should use the new
> videobuf2
> framework instead of the older videobuf framework. s5p-fimc is using
> vb2 already.
> but the vpif capture and display drivers in the davinci directory do
> not.

I quickly had a look at the s5p-fimc and davinci approaches, but I found that
the video bridge drivers supported in both the Samsung and TI SoCs,
support video post-processing operations whereas in our case the Video Port
IP performs almost no additional processing and only passes the unpacked RBG
raw data received from HDMI bus to system DDR via a DMA master interface.

So as such these are no format conversion operations(rgb-to-yuv or vice-versa),
image resizing operations (cropping, scaling..) and image quality operations
(filtering, distortion removal) available in the H/W block.

What should be the correct choice in such a case?
 
> With regards to HDMI receivers: these are still under development. One
> example
> is here:
> http://git.linuxtv.org/hverkuil/cisco.git?a=shortlog;h=refs/heads/cobal
> t
> 
> This tree contains a driver for the adv7604 HDMI/Graphics receiver. It
> is fairly
> simplistic at the moment, our internal driver is developed a lot
> further but I
> haven't had the chance yet to update the git tree with our latest code
> (Cisco
> is developing this driver). In addition the HDMI API for V4L2 is still
> under
> development. It requires some V4L2 core support to be merged first
> (control events)
> before we can continue with that.

Ok, I will have a look at the git tree. Thanks for sharing the same.

> >
> > Also will the audio ALSA driver fit as a subdev driver in the entire
> system?
> 
> No, although I have heard that the ALSA developers are looking at a
> subdev-like
> approach. There are several V4L drivers that support ALSA as a separate
> driver
> (cx18 for example). This is usually not a problem.

Ok, I will have a look at the cx18 driver.

Regards,
Bhupesh

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: Audio Video synchronization for data received from a HDMI receiver chip
  2011-05-19 11:10               ` Bhupesh SHARMA
@ 2011-05-19 11:38                 ` Hans Verkuil
  0 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2011-05-19 11:38 UTC (permalink / raw)
  To: Bhupesh SHARMA
  Cc: Charlie X. Liu, laurent.pinchart, g.liakhovetski, linux-media,
	alsa-devel

>> > Hi Hans,
>> >
>> > I have another doubt regarding the framework choice for the entire
>> > system that I have, especially the video part of the system. The
>> overall
>> > system is similar to the one depicted below:
>> >
>> > HDMI data --> HDMI receiver chip --> Video Port IP on SoC --> System
>> DDR
>> >
>> > HDMI data is received from external world (from say a set-up box or
>> dvd player),
>> > which is fed to the HDMI receiver chip on-board and then parallel
>> data lines feed
>> > this data to a Video Port IP on the SoC which has a DMA master
>> interface and
>> > hence can push the data thus received directly on system DDR.
>> >
>> > Now, I can figure out that there will be two drivers required here:
>> > # HDMI receiver chip driver (which is essentially a v4l2 subdev being
>> controller via I2C)
>> > # Video Port driver (which is a v4l2 bridge driver)
>> >
>> > Is my understanding correct?
>>
>> Yes.
>
> Thanks for clarifying this.
>
>> > Are there any HDMI receiver subdev driver and video bridge driver
>> already available which I can
>> > use for reference?
>>
>> Video bridge drivers are easier: examples are in
>> drivers/media/video/s5p-fimc
>> or in drivers/media/video/davinci. Note that you should use the new
>> videobuf2
>> framework instead of the older videobuf framework. s5p-fimc is using
>> vb2 already.
>> but the vpif capture and display drivers in the davinci directory do
>> not.
>
> I quickly had a look at the s5p-fimc and davinci approaches, but I found
> that
> the video bridge drivers supported in both the Samsung and TI SoCs,
> support video post-processing operations whereas in our case the Video
> Port
> IP performs almost no additional processing and only passes the unpacked
> RBG
> raw data received from HDMI bus to system DDR via a DMA master interface.

This is similar to the davinci vpif-capture/vpif-display drivers for the
TI dm646x SoCs. Those videoports are also just a DMA engine without video
processing.

> So as such these are no format conversion operations(rgb-to-yuv or
> vice-versa),
> image resizing operations (cropping, scaling..) and image quality
> operations
> (filtering, distortion removal) available in the H/W block.
>
> What should be the correct choice in such a case?

vpif-capture/display drivers are identical in that respect.

Regards,

       Hans


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2011-05-19 11:38 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-05-12  5:48 Audio Video synchronization for data received from a HDMI receiver chip Bhupesh SHARMA
2011-05-12 16:59 ` Charlie X. Liu
2011-05-12 20:29   ` Hans Verkuil
     [not found]     ` <BANLkTi=rpQEkroia3kUqp6zUHTQk3k220Q@mail.gmail.com>
2011-05-18  4:10       ` Bhupesh SHARMA
2011-05-18  6:32         ` Hans Verkuil
2011-05-19  5:39           ` Bhupesh SHARMA
2011-05-19  6:55             ` Hans Verkuil
2011-05-19  6:55               ` Hans Verkuil
2011-05-19 11:10               ` Bhupesh SHARMA
2011-05-19 11:38                 ` Hans Verkuil

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.