* [Regression 5.14] media: dvb userspace api @ 2021-08-11 12:15 Soeren Moch 2021-08-19 11:31 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 14+ messages in thread From: Soeren Moch @ 2021-08-11 12:15 UTC (permalink / raw) To: Linus Torvalds, Mauro Carvalho Chehab Cc: Linux Media Mailing List, linux-kernel Commit 819fbd3d8ef36c09576c2a0ffea503f5c46e9177 ("media: dvb header files: move some headers to staging") moved audio, video, and osd parts of the media DVB API to staging and out of kernel headers. But this is part of the media userspace API, removing this causes regressions. There already is a RedHat bug filed against this [1], and cannot be resolved there, of course. Please revert the above mentioned commit. Linus, Please help to keep the media DVB API intact. From all my previous experience with Mauro, he would otherwise just ignore this request and later claim: it was removed and cannot be brought back. The userspace behind this API is a program suite called VDR ("video disk recorder"), which was part of the linux media ecosystem from the beginning, is still part of linux distributions like RedHat/Fedora, Debian, SuSE, Ubuntu, easyVDR, yaVDR, is actively developed further, and runs with a bigger community behind it. Mauro, From many previous discussions you know that the av7110 driver, the DVB API, and especially also the output part of it, is in active use. I also asked several times to pull the saa716x driver [2], which also implements the full DVB API, among others for the successor cards of saa7146/av7110-based so called full-featured DVB cards. I also offered several times to maintain both drivers, and the related API. From your side there was no support whatsoever for this DVB API, you are fighting to remove it, against better knowledge that this is used. You gave no technical reason for this. Just, "it is an old API, I don't like it". And you wrote, that you do not understand it. This probably is the main reason for all the related problems. Your request to convert everything from the DVB API to the video4linux2 API is like "I don't like serial drivers, it is an old API, convert everything from serial to drm-framebuffer drivers, if you want to get boot messages out of the kernel." DVB and v4l2 are totally different APIs with different purpose, different supported hardware, different supported userspace applications. Just because there are much more drivers implementing v4l2 than DVB output is no good reason to kill this API, associated drivers and the community that keeps all this running. If you don't want to maintain the full DVB API and av7110/saa716x drivers any longer, which is obvious, there is a better solution than just ripping all this out. I just renew my offer to take over this job. If there really is a broken frontend support for av7110, I will have a look at this. Can you provide more detailed information about this? There are many flavours of this card with different frontends, so maybe I missed this. Your commit message "the decoder supports only MPEG2, with is not compatible with several modern DVB streams" is at least misleading. The most popular satellite TV provider in Germany (Astra) still transmits most of the interesting programs MPEG-2 encoded, so also this is actively used and no reason to retire this card. In all my previous discussions with maintainers from arm/mvebu, arm/imx, arm/rockchip, arm/soc, net, net/wireless, staging, usb, I always experienced technical support and fruitful discussions, which lead to fixed bugs, working hardware, and happy users. Only media is special. Here it can take up to five months, 3 full linux release cycles, to get a patch merged that was marked as fix and marked for stable, with only a single review comment in all that time: add more comments. In media the supporter recommends to maintain drivers outside the kernel, because people are happy with that. Of course nobody in the related community is happy with that, and maintaining the driver itself is the easiest part for me. Other people maintain scripts and how-tos for integrating the driver into different distributions with different update policies and scripts, that is even more work. All that would not be necessary, if the driver would just be pulled. So I really hope we also for media can come to a point, where supporters support the community, where maintainers maintain drivers they understand, patches are reviewed benevolently within reasonable time, and existing drivers with working hardware and happy users, only implementing long existing APIs (in mainline), are kept working, especially when someone exits who volunteers to maintain this. So please - revert this userspace API breakage still for 5.14 (commit 819fbd3d8ef36c09576c2a0ffea503f5c46e9177) - move the related documentation back from staging (revert commit 793e52d4e77d49737ad83cb11925c98f4907fcb1) - move the long existing and working av7110 driver back from staging (revert commit 989cf18ed08f8b6efd1d1592d1d0108fa09b98f5) - consider pulling the saa716x driver (based on the current 5.13 branch, I'm happy to provide a new pull request) - transfer maintainer-ship for this to me Linus, if you would accept a direct pull request from me, I would be happy to provide one, for the requested reverts (for 5.14) of in-tree DVB API and av7110 driver, but also for the long existing but still out-of-tree saa716x driver (for 5.15). Thanks, Soeren [1] https://bugzilla.redhat.com/show_bug.cgi?id=1989125 [2] https://github.com/s-moch/linux-saa716x ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-11 12:15 [Regression 5.14] media: dvb userspace api Soeren Moch @ 2021-08-19 11:31 ` Mauro Carvalho Chehab 2021-08-21 13:58 ` Manu Abraham 2021-08-22 15:21 ` Soeren Moch 0 siblings, 2 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2021-08-19 11:31 UTC (permalink / raw) To: Soeren Moch; +Cc: Linus Torvalds, Linux Media Mailing List, linux-kernel Em Wed, 11 Aug 2021 14:15:02 +0200 Soeren Moch <smoch@web.de> escreveu: > Commit 819fbd3d8ef36c09576c2a0ffea503f5c46e9177 ("media: dvb header > files: move some headers to staging") moved audio, video, and osd parts > of the media DVB API to staging and out of kernel headers. But this is > part of the media userspace API, removing this causes regressions. There's no regression: a legacy driver (av7110) for a device that stopped being manufactured 15 years ago and that doesn't work anymore with current Digital TV transmissions was removed, together with the API that it was implemented inside such driver's code. More details below. > There > already is a RedHat bug filed against this [1], and cannot be resolved > there, of course. Please revert the above mentioned commit. > > > Linus, > > Please help to keep the media DVB API intact. From all my previous > experience with Mauro, he would otherwise just ignore this request and > later claim: it was removed and cannot be brought back. The userspace > behind this API is a program suite called VDR ("video disk recorder"), > which was part of the linux media ecosystem from the beginning, is still > part of linux distributions like RedHat/Fedora, Debian, SuSE, Ubuntu, > easyVDR, yaVDR, is actively developed further, and runs with a bigger > community behind it. > > > Mauro, > > From many previous discussions you know that the av7110 driver, the DVB > API, and especially also the output part of it, is in active use. The av7110 hardware was developed up to 1999. Its Linux API was implemented by a company called Convergence which has long gone (they stopped working on it back in 2004, afaikt). The av7110 production stopped ~15 years ago. This is a legacy hardware, which supports only the first generation of DVB standards, and had an integrated MPEG-2 decoder. As most DVB transmissions use MPEG4 or newer encoding schemas that didn't exist back in 1999, it doesn't make any sense keeping such driver upstream nowadays. The API that got removed was written to control the av7110 MPEG-2 decoder, and was never integrated at the DVB core: the av7110 had a driver-specific implementation inside its code. Besides that, the API was never fully documented: there are several ioctls from the now removed API that never had any in-kernel implementation, nor had and descriptions at the specs. None of the current upstream maintainers have any glue about what such ioctls are supposed to do, nor do we have any av7110 hardware to test it. > I also > asked several times to pull the saa716x driver [2], which also > implements the full DVB API, among others for the successor cards of > saa7146/av7110-based so called full-featured DVB cards. I also offered > several times to maintain both drivers, and the related API. The saa716x driver you're mentioned is an out of tree driver. We don't keep APIs at the upstream Kernel due to OOT drivers. Btw, there's no need for that: if you have an OOT tree, you can simply place the API headers for whatever API your device requires. - Now, if you want to upstream your driver, I gave you already a way to do it in the past: we need to develop an interface that it is not dependent on any hardware-specific functionality, but can be evolved with time and can support different families of codec protocols. It should also be properly documented. Those are the goals already achieved by the V4L2 codec API: it already supports MPEG2, MPEG4, HEVC and other types of codec, and can easily be integrated with a DVB device via the media controller API. Thanks, Mauro ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-19 11:31 ` Mauro Carvalho Chehab @ 2021-08-21 13:58 ` Manu Abraham 2021-08-22 15:21 ` Soeren Moch 1 sibling, 0 replies; 14+ messages in thread From: Manu Abraham @ 2021-08-21 13:58 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Soeren Moch, Linus Torvalds, Linux Media Mailing List, linux-kernel Dearest Mauro, I am not trying to annoy you or anyone else with my response here, but: On Thu, Aug 19, 2021 at 5:01 PM Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote: > > Em Wed, 11 Aug 2021 14:15:02 +0200 > Soeren Moch <smoch@web.de> escreveu: > > > Commit 819fbd3d8ef36c09576c2a0ffea503f5c46e9177 ("media: dvb header > > files: move some headers to staging") moved audio, video, and osd parts > > of the media DVB API to staging and out of kernel headers. But this is > > part of the media userspace API, removing this causes regressions. > > There's no regression: a legacy driver (av7110) for a device that stopped > being manufactured 15 years ago and that doesn't work anymore with current > Digital TV transmissions was removed, together with the API that it was > implemented inside such driver's code. Please do not exaggerate.. (I can write more precise technical details in here, but that will not solve the real issue at hand.) You have only your own viewpoint, refuse to listen to anyone else. Wonder why all the DVB developers left development ? It's all about you, yourself and you. Linus doesn't care about anything else, you have been very lucky! You need serious introspection about yourself. Take a deep breath, think for yourself, why I stopped submitting code. Forget myself, think about the list of developers who were around, but not now. People need to have fun with what they are doing, but you make it, everything about yourself. It's all about maintaining connections, rather than destroying them. At least during these uncertain times, please stop the narrow thinking. If you find my view offending, just ignore it, no need to give another thousand mile long essay; I am on my way .. Friendly Regards, MA ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-19 11:31 ` Mauro Carvalho Chehab 2021-08-21 13:58 ` Manu Abraham @ 2021-08-22 15:21 ` Soeren Moch 2021-08-22 17:47 ` Mauro Carvalho Chehab 1 sibling, 1 reply; 14+ messages in thread From: Soeren Moch @ 2021-08-22 15:21 UTC (permalink / raw) To: Mauro Carvalho Chehab, Linus Torvalds Cc: Linux Media Mailing List, linux-kernel On 19.08.21 13:31, Mauro Carvalho Chehab wrote: > Em Wed, 11 Aug 2021 14:15:02 +0200 > Soeren Moch <smoch@web.de> escreveu: > >> Commit 819fbd3d8ef36c09576c2a0ffea503f5c46e9177 ("media: dvb header >> files: move some headers to staging") moved audio, video, and osd parts >> of the media DVB API to staging and out of kernel headers. But this is >> part of the media userspace API, removing this causes regressions. > There's no regression: a legacy driver (av7110) for a device that stopped > being manufactured 15 years ago and that doesn't work anymore with current > Digital TV transmissions was removed, together with the API that it was > implemented inside such driver's code. What you write here is simply not true. The "device" (saa7146/av7110-based full-featured DVB card) is working well. Also with current Digital TV transmissions (e.g. Astra Satellite TV in Europe). The DVB API never was implemented in driver's code, it is linux userspace API (include/uapi/linux/dvb/). You moved files out of the uapi part of kernel headers, parts of e.g. RedHat userspace stops to build due to this. So it is a userspace regression. > More details below. > >> There >> already is a RedHat bug filed against this [1], and cannot be resolved >> there, of course. Please revert the above mentioned commit. >> >> >> Linus, >> >> Please help to keep the media DVB API intact. From all my previous >> experience with Mauro, he would otherwise just ignore this request and >> later claim: it was removed and cannot be brought back. The userspace >> behind this API is a program suite called VDR ("video disk recorder"), >> which was part of the linux media ecosystem from the beginning, is still >> part of linux distributions like RedHat/Fedora, Debian, SuSE, Ubuntu, >> easyVDR, yaVDR, is actively developed further, and runs with a bigger >> community behind it. >> >> >> Mauro, >> >> From many previous discussions you know that the av7110 driver, the DVB >> API, and especially also the output part of it, is in active use. > The av7110 hardware was developed up to 1999. Its Linux API was implemented > by a company called Convergence which has long gone (they stopped working > on it back in 2004, afaikt). The av7110 production stopped ~15 years ago. But the cards work perfectly well. I own two such cards, there is no problem at all with them. Other members of the community even test with -rc3 kernels and file RedHat bugs. So there clearly is interest in them. The DVB API was developed by Convergence, maintained by folks on linuxtv.org later, adopted by other companies (Fujitsu-Siemens, Technotrend) which developed boards and maintained drivers. There still is a community behind this, e.g. on vdr-portal.de . > This is a legacy hardware, which supports only the first generation of DVB > standards, and had an integrated MPEG-2 decoder. As most DVB transmissions > use MPEG4 or newer encoding schemas that didn't exist back in 1999, it > doesn't make any sense keeping such driver upstream nowadays. As I wrote in my previous email: most private TV stations in Germany provide their free-to-air satellite programs MPEG-2 encoded only. Therefore this is very popular and it absolutely makes sense to keep this driver upstream. > The API that got removed was written to control the av7110 MPEG-2 decoder, > and was never integrated at the DVB core: the av7110 had a driver-specific > implementation inside its code. This is simply not true. The DVB API is linux userspace API, absolutely nothing driver specific inside driver's code. Also this API is not specific to any audio or video encoding standard as you imply here. > Besides that, the API was never fully documented: there are several ioctls > from the now removed API that never had any in-kernel implementation, nor > had and descriptions at the specs. None of the current upstream maintainers > have any glue about what such ioctls are supposed to do, nor do we have any > av7110 hardware to test it. It's not the fault of the community that you have no clue about this API. So let someone maintain this API and driver who has clue about API, driver, hardware and the userspace behind all this. >> I also >> asked several times to pull the saa716x driver [2], which also >> implements the full DVB API, among others for the successor cards of >> saa7146/av7110-based so called full-featured DVB cards. I also offered >> several times to maintain both drivers, and the related API. > The saa716x driver you're mentioned is an out of tree driver. > We don't keep APIs at the upstream Kernel due to OOT drivers. Strong words to distract from the main point: This regression report is about upstream DVB userspace API and the saa7146/av7110 driver, both part of mainline linux for a long time. > Btw, there's no need for that: if you have an OOT tree, you can simply > place the API headers for whatever API your device requires. Thanks for answering questions that nobody asked. > - > > Now, if you want to upstream your driver, I gave you already a > way to do it in the past: we need to develop an interface that it > is not dependent on any hardware-specific functionality, but can > be evolved with time and can support different families of codec > protocols. It should also be properly documented. The DVB API already is an interface that is not dependent on hardware-specific functionality. This can be seen easily as the exact same userspace API is used by the saa7146/av7110 and by the saa716x driver for totally different hardware. The DVB API is about sending and receiving DVB streams and controlling hardware that is related to that (e.g. PID filters, conditional access devices, OSDs). There is nothing about controlling codecs in this API, as I already told you. You will not find any documentation about how this API controls codecs, because it simply is not used for this purpose. The DVB API does not care about the type of audio and video streams that are encapsulated in the DVB stream. The DVB output API is designed to not even know if a stream is sent to a codec device, it could e.g. be a dvb-t modulator or any other type of DVB transmission device. > Those are the goals already achieved by the V4L2 codec API: > it already supports MPEG2, MPEG4, HEVC and other types of codec, > and can easily be integrated with a DVB device via the media > controller API. V4L2 is great for controlling codecs. But there is no such functionality required in the saa716x driver, so why should I use this API there? The TT S2-6400 card can of course handle MPEG-4 AVC, I also told you that. Why do you repeatedly refuse to accept this fact? The DVB API simply does not care about codecs, so the saa716x driver does not need to know at which point in time it sends a DVB stream with MPEG-2 encoded video, or when with MPEG-4 AVC encoded video, or HEVC, or VVC. There is nothing to control for v4l2. Mauro, since you wrote again you have no clue about the DVB output API, please stop spreading wrong claims about this API. Please stop fighting against the community that uses this API, long existing drivers implementing it, and existing and working related hardware. If you don't can or want to maintain this API, let others do the job. Also please explain what exact problem you encountered with full-featured DVB cards, that you think is so severe to justify the removal of the driver. Especially as you wrote above that you do not know the driver (and probably the related userspace application) and have no hardware to test. > Thanks, > Mauro Mauro, you stripped almost everything from my previous email, you did not answer any questions or gave any explanation for your behavior. You refused the fact, that you caused userspace regressions, this is what this email thread is about. What is that supposed to mean? You really do not know the difference between linux userspace API and private driver headers? Your answer was not helpful at all in resolving the reported regression. So I repeat my request from the previous email: Please - revert this userspace API breakage still for 5.14 (commit 819fbd3d8ef36c09576c2a0ffea503f5c46e9177) - move the related documentation back from staging (revert commit 793e52d4e77d49737ad83cb11925c98f4907fcb1) - move the long existing and working av7110 driver back from staging (revert commit 989cf18ed08f8b6efd1d1592d1d0108fa09b98f5) - consider pulling the saa716x driver (based on the current 5.13 branch, I'm happy to provide a new pull request) - transfer maintainer-ship for this to me Regards, Soeren ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-22 15:21 ` Soeren Moch @ 2021-08-22 17:47 ` Mauro Carvalho Chehab 2021-08-23 14:59 ` Soeren Moch 0 siblings, 1 reply; 14+ messages in thread From: Mauro Carvalho Chehab @ 2021-08-22 17:47 UTC (permalink / raw) To: Soeren Moch; +Cc: Linus Torvalds, Linux Media Mailing List, linux-kernel Em Sun, 22 Aug 2021 17:21:41 +0200 Soeren Moch <smoch@web.de> escreveu: > > There's no regression: a legacy driver (av7110) for a device that stopped > > being manufactured 15 years ago and that doesn't work anymore with current > > Digital TV transmissions was removed, together with the API that it was > > implemented inside such driver's code. > What you write here is simply not true. > > The "device" (saa7146/av7110-based full-featured DVB card) is working > well. Probably not true - at least for some av7110-based boards - as there was a regression that no user ever noticed: https://lore.kernel.org/lkml/20210503115736.2104747-59-gregkh@linuxfoundation.org/ This was noticed only too late, due to a review of the offended patch caused by "hypocrite commits"[1]. [1] https://lwn.net/Articles/854645/. > Also with current Digital TV transmissions (e.g. Astra Satellite > TV in Europe). The DVB API never was implemented in driver's code, it is > linux userspace API (include/uapi/linux/dvb/). The DVB API used by all upstream drivers is implemented inside the DVB core (at drivers/media/dvb-core/). The "full-featured" API consists on the DVB API + some extra ioctls defined at the uAPI headers, plus an av7110 implementation, which covered only part of the ioctls that were defined at the headers. > You moved files out of the uapi part of kernel headers, parts of e.g. > RedHat userspace stops to build due to this. So it is a userspace > regression. Again, this is not a Kernel regression. There isn't a single bit of code inside the Kernel that it would require the "full-feat" uapi. If an app implements support to some OOT driver(s), it has to carry on the OOT userspace code for such driver(s), together with any needed headers for such support. So, you should submit a patch to such app maintainers directly - and/or to the distro packages that would have packages providing support for such OOT driver(s). Btw, as far as I'm aware, Red Hat's Kernels don't come with the saa716x OOT driver. > > The av7110 production stopped ~15 years ago. > But the cards work perfectly well. I own two such cards, there is no > problem at all with them. Other members of the community even test with > -rc3 kernels and file RedHat bugs. So there clearly is interest in them. Nobody is telling otherwise. If people want to use OOT drivers, that's OK. Nobody is preventing people to use it, nor to use the apps developed for such devices. Keeping av7110 in-kernel has been a waste of limited upstream development resources. A couple of years ago, we needed to fix the av7110 API due to year-2038 issues. From time to time, we get bugs affecting it (even security ones), as the code has been bit-rotting for a long time. The most recent one probably broke the driver without nobody noticing it for a couple of Kernel reviews, as mentioned above. > > This is a legacy hardware, which supports only the first generation of DVB > > standards, and had an integrated MPEG-2 decoder. As most DVB transmissions > > use MPEG4 or newer encoding schemas that didn't exist back in 1999, it > > doesn't make any sense keeping such driver upstream nowadays. > As I wrote in my previous email: most private TV stations in Germany > provide their free-to-air satellite programs MPEG-2 encoded only. > Therefore this is very popular and it absolutely makes sense to keep > this driver upstream. It is probably a lot easier to get a modern DVB "budget" card with supports not only MPEG-2 but all encoding standards than to find an old "full-feat" DVB card with av7110 those days. Who still provides saa71xx chips those days? As far as I'm aware, Philips semiconductors (who used to produce such chipsets) was split into NXP in 2006, and the entire digital TV chipset line moved altogether. I can't find any references those days to any saa71xx at either NXP or Philips websites. > > The API that got removed was written to control the av7110 MPEG-2 decoder, > > and was never integrated at the DVB core: the av7110 had a driver-specific > > implementation inside its code. > This is simply not true. > The DVB API is linux userspace API, absolutely nothing driver specific > inside driver's code. From upstream PoV, it is an av7110-specific API, as all in-kernel support was inside av7110 driver's code. > > The saa716x driver you're mentioned is an out of tree driver. > > We don't keep APIs at the upstream Kernel due to OOT drivers. > Strong words to distract from the main point: > This regression report is about upstream DVB userspace API and the > saa7146/av7110 driver, both part of mainline linux for a long time. Removing a legacy driver is not a regression. See, you're the one trying to distract from the main point: The saa716x driver is OOT. There was never any upstream support for it. None of the patches you're mentioning prevents anyone to keep building it as an OOT driver. What it was removed was the av7110, together with the API header files that were used only by this single driver. > you stripped almost everything from my previous email, you did not > answer any questions or gave any explanation for your behavior. I striped the points already discussed with you when I gave you feedback about the saa716x patchset [2], as this is a completely separate discussion than the removal of av7110 support. In summary, it makes no sense to claim that saa716x OOT driver broke because a different driver was removed upstream. Thanks, Mauro [2] https://lore.kernel.org/linux-media/20180307121438.389f411c@vento.lan/ ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-22 17:47 ` Mauro Carvalho Chehab @ 2021-08-23 14:59 ` Soeren Moch 2021-08-23 16:58 ` Linus Torvalds 0 siblings, 1 reply; 14+ messages in thread From: Soeren Moch @ 2021-08-23 14:59 UTC (permalink / raw) To: Mauro Carvalho Chehab, Linus Torvalds Cc: Linux Media Mailing List, linux-kernel On 22.08.21 19:47, Mauro Carvalho Chehab wrote: > Em Sun, 22 Aug 2021 17:21:41 +0200 > Soeren Moch <smoch@web.de> escreveu: > >>> There's no regression: a legacy driver (av7110) for a device that stopped >>> being manufactured 15 years ago and that doesn't work anymore with current >>> Digital TV transmissions was removed, together with the API that it was >>> implemented inside such driver's code. >> What you write here is simply not true. >> >> The "device" (saa7146/av7110-based full-featured DVB card) is working >> well. > Probably not true - at least for some av7110-based boards - as there was a > regression that no user ever noticed: > > https://lore.kernel.org/lkml/20210503115736.2104747-59-gregkh@linuxfoundation.org/ Did you read the patch? Ignoring errors may be wrong, but this causes no regression for any user because i2c transfers to the frontend simply just succeed always in normal operation. BTW, for me this is another mess you created here. Why did you move the frontend driver out of the dvb-frontends directory into the av7110 driver, that as such is totally independent from the sp8870 frontend? > > This was noticed only too late, due to a review of the offended patch > caused by "hypocrite commits"[1]. > > [1] https://lwn.net/Articles/854645/. All this is totally unrelated to this regression report on the DVB userspace API. > >> Also with current Digital TV transmissions (e.g. Astra Satellite >> TV in Europe). The DVB API never was implemented in driver's code, it is >> linux userspace API (include/uapi/linux/dvb/). > The DVB API used by all upstream drivers is implemented inside the DVB > core (at drivers/media/dvb-core/). Simply not true. The headers in include/uapi/linux/dvb/ define the DVB userspace API. Parts of this API have only one in-tree user: the saa7146/av7110 driver. Other parts are used from many drivers for tens, probably hundreds, of different hardware devices, so common helper functions in dvb-core make sense. > > The "full-featured" API consists on the DVB API + some extra ioctls > defined at the uAPI headers, plus an av7110 implementation, which > covered only part of the ioctls that were defined at the headers. The DVB API is a well-designed API, in contrast to what you repeatedly claim designed independently from special hardware requirements. There are several parts of the API (frontend, dmx, ca, net, audio, video, osd). Hardware devices implementing all related functionality are called "full-featured cards" (in contrast to budget cards or e.g. all types of DVB sticks that usually implement input functionality related to the frontend and dmx part of the API, there is no "full-featured API"). All defined API calls are integral part of the API, there are no "extra ioctls" just because you personally like some API calls more than others. Yes, not all defined API calls had been used by in-tree drivers. You already removed these API call definitions some time ago, what causes real pain for the users of the out-of-tree saa716x driver with no advantage for in-tree linux-media. But at least this did not cause regressions for in-tree drivers and the related userspace applications. This time you removed API call definitions that are used by the in-tree saa7146/av7110 driver. This causes regressions for the users of this in-tree driver, because the related userspace-application stops to build. This exactly is what this regression report is about. > >> You moved files out of the uapi part of kernel headers, parts of e.g. >> RedHat userspace stops to build due to this. So it is a userspace >> regression. > Again, this is not a Kernel regression. Yes, it is a userspace regression caused by your changes of the linux DVB userspace API. > There isn't a single bit of > code inside the Kernel that it would require the "full-feat" uapi. This is obviously wrong, since the in-tree saa7146/av7110 driver implements the removed API call definitions. > > If an app implements support to some OOT driver(s), it has to carry on the > OOT userspace code for such driver(s), together with any needed headers for > such support. So, you should submit a patch to such app maintainers > directly - and/or to the distro packages that would have packages > providing support for such OOT driver(s). > > Btw, as far as I'm aware, Red Hat's Kernels don't come with the > saa716x OOT driver. Mauro, please stop spreading FUD. This regression report is about the part of the DVB API that is implemented by the in-tree saa7146/av7110 driver and it's related userspace. Both are part of the RedHat linux distribution. > >>> The av7110 production stopped ~15 years ago. >> But the cards work perfectly well. I own two such cards, there is no >> problem at all with them. Other members of the community even test with >> -rc3 kernels and file RedHat bugs. So there clearly is interest in them. > Nobody is telling otherwise. If people want to use OOT drivers, that's > OK. Nobody is preventing people to use it, nor to use the apps developed > for such devices. This discussion here is about av7110, see the 3rd-level citation "The av7110 production stopped ~15 years ago." above. And the saa7146/av7110 driver is in-tree, and the related userspace experences regressions that I report here. > > Keeping av7110 in-kernel has been a waste of limited upstream development > resources. Simply not true. The driver is used, therefore it is not wasted time to maintain it. And for years I repeatedly offered to do this maintenance, transfer maintainer-ship and you immediately git rid of this "overwhelming burden" to maintain this driver. As additional advantage I understand the driver and related API, do the required maintenance for the similar saa716x driver (out-of-tree) anyway, and have hardware to test (for both drivers). > A couple of years ago, we needed to fix the av7110 API due to > year-2038 issues. Simply not true. Nothing in this driver or the DVB API is related to wall-clock time. Only DVB timestamps are used. So there is no year-2038 issue. > From time to time, we get bugs affecting it (even security > ones), as the code has been bit-rotting for a long time. Simply not true. Oliver Endriss did a great job in maintaining this driver, and it still works perfectly fine. > The most recent one > probably broke the driver without nobody noticing it for a couple of Kernel > reviews, as mentioned above. No, see above. The linked patch caused a bug, but no user-visible regression. And this DVB-T frontend is optional anyway and not very popular. > >>> This is a legacy hardware, which supports only the first generation of DVB >>> standards, and had an integrated MPEG-2 decoder. As most DVB transmissions >>> use MPEG4 or newer encoding schemas that didn't exist back in 1999, it >>> doesn't make any sense keeping such driver upstream nowadays. >> As I wrote in my previous email: most private TV stations in Germany >> provide their free-to-air satellite programs MPEG-2 encoded only. >> Therefore this is very popular and it absolutely makes sense to keep >> this driver upstream. > It is probably a lot easier to get a modern DVB "budget" card with > supports not only MPEG-2 but all encoding standards than to find an > old "full-feat" DVB card with av7110 those days. Maybe. But when the most popular satellite TV provider broadcasts MPEG-2 encoded video and people are happy with their working cards, why throw away a running system? Besides that, people usually use full-featured cards together with additional modern budget cards/sticks to be able to record different DVB programs while seeing a different program as liveview. > > Who still provides saa71xx chips those days? As far as I'm aware, > Philips semiconductors (who used to produce such chipsets) was split > into NXP in 2006, and the entire digital TV chipset line moved > altogether. I can't find any references those days to any saa71xx > at either NXP or Philips websites. This documentation is only provided with non-disclosure agreement. Luckily I received such documentation under NDA together with firmware code and schematics. So I am in a very good position to maintain this. >>> The API that got removed was written to control the av7110 MPEG-2 decoder, >>> and was never integrated at the DVB core: the av7110 had a driver-specific >>> implementation inside its code. >> This is simply not true. >> The DVB API is linux userspace API, absolutely nothing driver specific >> inside driver's code. > From upstream PoV, it is an av7110-specific API, as all in-kernel support > was inside av7110 driver's code. And from userspace point-of-view ist is linux userspace API. > >>> The saa716x driver you're mentioned is an out of tree driver. >>> We don't keep APIs at the upstream Kernel due to OOT drivers. >> Strong words to distract from the main point: >> This regression report is about upstream DVB userspace API and the >> saa7146/av7110 driver, both part of mainline linux for a long time. > Removing a legacy driver is not a regression. This in-tree driver is actively used by a whole community and works perfectly fine. The corresponding userspace applications are in many major linux distributions. Why are you fighting so hard to remove this driver and kill the community and the userspace behind it? If linux would only contain drivers for hardware that is personally used by subsystem maintainers, then I think this would be a very poor user experience. > See, you're the one > trying to distract from the main point: > > The saa716x driver is OOT. There was never any upstream support > for it. None of the patches you're mentioning prevents anyone > to keep building it as an OOT driver. > > What it was removed was the av7110, together with the API header > files that were used only by this single driver. And the userspace. And this causes regressions, e.g. in RedHat. And this is what this regression report is about. > >> you stripped almost everything from my previous email, you did not >> answer any questions or gave any explanation for your behavior. > I striped the points already discussed with you when I gave you > feedback about the saa716x patchset [2], as this is a completely > separate discussion than the removal of av7110 support. It's indeed a separate discussion. But you brought up this topic here. If someone wants to learn how something works and why it is implemented the way it is, I'm happy to explain. Unfortunately you always immediately shoot down this discussion with: implement something else, no explanation why, no technical discussion about pros and cons of alternative implementations. > > In summary, it makes no sense to claim that saa716x OOT driver > broke because a different driver was removed upstream. I nowhere claimed that. It's not me who is bending truth and twisting words all the time. > > Thanks, > Mauro > > [2] https://lore.kernel.org/linux-media/20180307121438.389f411c@vento.lan/ Mauro, please explain why you act as you do, especially as this is so totally different from what I know from other linux subsystems. There is a great DVB API and lots of drivers implementing it upstream for decades. There is a community behind it and related userspace in many major linux distributions. Working hardware also in newer generations. You do not understand this API (as you wrote yourself), but you kill all technical discussions immediately when I explain details. You write maintaining this DVB API and drivers is "waste of limited upstream development resources", but you completely ignore my offer to take over maintainer-ship and kill every reference to this offer immediately in your answers. On the other side there is apparently plenty of time for whitespace cleanup, comment cleanup, character encoding cleanup, documentation reference fixes. There is nothing wrong with this, but I think it's a bad excuse for not having time to maintain DVB drivers (for what you are paid according to the MAINTAINERS entry). You recommend to maintain drivers out-of-tree, just because you don't like the API the driver implements, although this API is part of linux media for decades. No technical discussions, no idea how to support the community. Now you try to kill working in-tree drivers, you cause regressions for existing userspace applications. No explanation of any reasons, no idea how to solve this issue. No effort to work with the community. No progress here in all the emails in this thread. For me every answer sounds like: I do what I want anyway, shut up! Or which point I missed here? Linus, Is what I described directly above the new linux maintenance policy? Or is linux media a private kingdom where the community should keep away? Is this a place where the subsystem maintainer is on a mission to destroy everything instead of maintaining and improving it? Please tell me what I understood wrong here. Thanks, Soeren ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-23 14:59 ` Soeren Moch @ 2021-08-23 16:58 ` Linus Torvalds 2021-08-23 20:16 ` Soeren Moch ` (2 more replies) 0 siblings, 3 replies; 14+ messages in thread From: Linus Torvalds @ 2021-08-23 16:58 UTC (permalink / raw) To: Soeren Moch; +Cc: Mauro Carvalho Chehab, Linux Media Mailing List, linux-kernel On Mon, Aug 23, 2021 at 7:59 AM Soeren Moch <smoch@web.de> wrote: > > Linus, > > Is what I described directly above the new linux maintenance policy? Or > is linux media a private kingdom where the community should keep away? > Is this a place where the subsystem maintainer is on a mission to > destroy everything instead of maintaining and improving it? Please tell > me what I understood wrong here. So technically, the regression policy for the kernel is purely about the ABI - the _binary_ interface. That seems to not have broken - old programs continue to work. We very much try to discourage user space applications from using the kernel header files directly - even projects like glibc etc are supposed to _copy_ them, not include the kernel headers. Exactly because re-organization and changes to the kernel tree shouldn't be something that then causes random problems elsewhere that are so hard to test - and synchronize - from the kernel standpoint (or from the standpoint of the other end). That clearly doesn't seem to be the case in this situation. Which is annoying as heck. Mauro: there clearly _are_ users of those header files, and even apparently that one old driver out there. And those headers were in the 'uapi' directory, so while it is annoying how user space programs used them this way, I think it's also not entirely unreasonable. I have reverted the header file move. But I would also heartily recommend that whatever user program includes those headers (VDR - anything else?) should take snapshots of these specific kernel headers. I'm not convinced that it makes sense to move the av7110 driver back from staging - it may continue to work, but it _is_ old and there is no maintenance - and I would certainly suggest that any other out-of-tree driver that uses these old interfaces that nothing else implements shouldn't do so, considering that nothing else implements them. So the only thing I did was move the header files back, and mark that move to be backported to 5.13 stable. Linus ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-23 16:58 ` Linus Torvalds @ 2021-08-23 20:16 ` Soeren Moch 2021-08-24 7:47 ` Mauro Carvalho Chehab 2021-08-25 2:55 ` Manu Abraham 2 siblings, 0 replies; 14+ messages in thread From: Soeren Moch @ 2021-08-23 20:16 UTC (permalink / raw) To: Linus Torvalds Cc: Mauro Carvalho Chehab, Linux Media Mailing List, linux-kernel On 23.08.21 18:58, Linus Torvalds wrote: > On Mon, Aug 23, 2021 at 7:59 AM Soeren Moch <smoch@web.de> wrote: >> Linus, >> >> Is what I described directly above the new linux maintenance policy? Or >> is linux media a private kingdom where the community should keep away? >> Is this a place where the subsystem maintainer is on a mission to >> destroy everything instead of maintaining and improving it? Please tell >> me what I understood wrong here. Thanks for your quick answer. > So technically, the regression policy for the kernel is purely about > the ABI - the _binary_ interface. That seems to not have broken - old > programs continue to work. OK, as long as the related driver lives at least in staging. Without the driver of course the hardware and userspace will not work any longer with new kernel versions. > We very much try to discourage user space applications from using the > kernel header files directly - even projects like glibc etc are > supposed to _copy_ them, not include the kernel headers. OK, thanks for pointing to this policy. > Exactly because re-organization and changes to the kernel tree > shouldn't be something that then causes random problems elsewhere that > are so hard to test - and synchronize - from the kernel standpoint (or > from the standpoint of the other end). > > That clearly doesn't seem to be the case in this situation. Which is > annoying as heck. > > Mauro: there clearly _are_ users of those header files, and even > apparently that one old driver out there. And those headers were in > the 'uapi' directory, so while it is annoying how user space programs > used them this way, I think it's also not entirely unreasonable. > > I have reverted the header file move. But I would also heartily > recommend that whatever user program includes those headers (VDR - > anything else?) should take snapshots of these specific kernel > headers. I will try to modify the related userspace accordingly, but it will take time until this ripples through to distributions. I'm not aware of other applications than VDR (especially 2 Plugins) using these 3 header files. > I'm not convinced that it makes sense to move the av7110 driver back > from staging - it may continue to work, but it _is_ old and there is > no maintenance - It is old, but there are users out there - including me - that still use such card and driver. I would be happy to maintain this driver, if I'm allowed to do so. I already offered this for several years. How long can this driver stay in staging? Would you move the driver back from staging when I do proper maintenance for it? Is it normal linux policy to remove drivers after a certain period of time, even if a driver still has users and someone that volunteers to maintain it? > and I would certainly suggest that any other > out-of-tree driver that uses these old interfaces that nothing else > implements shouldn't do so, considering that nothing else implements > them. The hardware of these so called full-featured DVB cards is very special. It is a Satellite-/Cable-TV Set-Top-Box on a PCI/PCIe card. Since only two generations of these cards exist (the first generation in many variants), only two drivers implement the full DVB API. There are no other drivers for similar hardware, nothing uses other APIs for this type of hardware. Given that all drivers for this type of hardware use the API in question, would you accept the second driver for the second generation of full-featured DVB cards (saa716x) [1] to be pulled into mainline linux? > So the only thing I did was move the header files back, and mark that > move to be backported to 5.13 stable. Thanks for moving the header files back. In 5.13.12 the API files are still there at the old position. Best regards, Soeren [1] https://github.com/s-moch/linux-saa716x ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-23 16:58 ` Linus Torvalds 2021-08-23 20:16 ` Soeren Moch @ 2021-08-24 7:47 ` Mauro Carvalho Chehab 2021-08-24 20:01 ` Honza P 2021-08-25 2:55 ` Manu Abraham 2 siblings, 1 reply; 14+ messages in thread From: Mauro Carvalho Chehab @ 2021-08-24 7:47 UTC (permalink / raw) To: Linus Torvalds; +Cc: Soeren Moch, Linux Media Mailing List, linux-kernel Em Mon, 23 Aug 2021 09:58:00 -0700 Linus Torvalds <torvalds@linux-foundation.org> escreveu: > On Mon, Aug 23, 2021 at 7:59 AM Soeren Moch <smoch@web.de> wrote: > > > > Linus, > > > > Is what I described directly above the new linux maintenance policy? Or > > is linux media a private kingdom where the community should keep away? > > Is this a place where the subsystem maintainer is on a mission to > > destroy everything instead of maintaining and improving it? Please tell > > me what I understood wrong here. > > So technically, the regression policy for the kernel is purely about > the ABI - the _binary_ interface. That seems to not have broken - old > programs continue to work. > > We very much try to discourage user space applications from using the > kernel header files directly - even projects like glibc etc are > supposed to _copy_ them, not include the kernel headers. Unfortunately, media APIs aren't part of projects like glibc. Almost all open source media apps keep their own copies of the uAPI header files. As far as I'm aware, the "full-feat" API is implemented only by some modules of VDR. I don't know any other open source application using such headers. > Exactly because re-organization and changes to the kernel tree > shouldn't be something that then causes random problems elsewhere that > are so hard to test - and synchronize - from the kernel standpoint (or > from the standpoint of the other end). > > That clearly doesn't seem to be the case in this situation. Which is > annoying as heck. > > Mauro: there clearly _are_ users of those header files, and even > apparently that one old driver out there. And those headers were in > the 'uapi' directory, so while it is annoying how user space programs > used them this way, I think it's also not entirely unreasonable. > > I have reverted the header file move. But I would also heartily > recommend that whatever user program includes those headers (VDR - > anything else?) should take snapshots of these specific kernel > headers. > > I'm not convinced that it makes sense to move the av7110 driver back > from staging - it may continue to work, but it _is_ old and there is > no maintenance - and I would certainly suggest that any other > out-of-tree driver that uses these old interfaces that nothing else > implements shouldn't do so, considering that nothing else implements > them. > > So the only thing I did was move the header files back, and mark that > move to be backported to 5.13 stable. Ok. Thanks, Mauro ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-24 7:47 ` Mauro Carvalho Chehab @ 2021-08-24 20:01 ` Honza P 0 siblings, 0 replies; 14+ messages in thread From: Honza P @ 2021-08-24 20:01 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Linus Torvalds, Soeren Moch, Linux Media Mailing List, linux-kernel út 24. 8. 2021 v 9:50 odesílatel Mauro Carvalho Chehab <mchehab+huawei@kernel.org> napsal: > > Em Mon, 23 Aug 2021 09:58:00 -0700 > Linus Torvalds <torvalds@linux-foundation.org> escreveu: > > > On Mon, Aug 23, 2021 at 7:59 AM Soeren Moch <smoch@web.de> wrote: > > > > > > Linus, > > > > > > Is what I described directly above the new linux maintenance policy? Or > > > is linux media a private kingdom where the community should keep away? > > > Is this a place where the subsystem maintainer is on a mission to > > > destroy everything instead of maintaining and improving it? Please tell > > > me what I understood wrong here. > > > > So technically, the regression policy for the kernel is purely about > > the ABI - the _binary_ interface. That seems to not have broken - old > > programs continue to work. > > > > We very much try to discourage user space applications from using the > > kernel header files directly - even projects like glibc etc are > > supposed to _copy_ them, not include the kernel headers. > > Unfortunately, media APIs aren't part of projects like glibc. Almost all > open source media apps keep their own copies of the uAPI header files. > > As far as I'm aware, the "full-feat" API is implemented only by some > modules of VDR. I don't know any other open source application using > such headers. > You definitely missed tons of users of linux based set-top-boxes, powered by open-source DVB frondend Enigma2 (and also still big enough older devices based on Enigma 1 project). For ex here: https://github.com/OpenPLi/enigma2 /Honza (also retired dvb developer, disgusted the way how media subsystem was driven) ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-23 16:58 ` Linus Torvalds 2021-08-23 20:16 ` Soeren Moch 2021-08-24 7:47 ` Mauro Carvalho Chehab @ 2021-08-25 2:55 ` Manu Abraham 2021-08-25 6:33 ` Mauro Carvalho Chehab 2 siblings, 1 reply; 14+ messages in thread From: Manu Abraham @ 2021-08-25 2:55 UTC (permalink / raw) To: Linus Torvalds Cc: Soeren Moch, Mauro Carvalho Chehab, Linux Media Mailing List, linux-kernel On Mon, Aug 23, 2021 at 10:30 PM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > I have reverted the header file move. But I would also heartily > recommend that whatever user program includes those headers (VDR - > anything else?) should take snapshots of these specific kernel > headers. > > I'm not convinced that it makes sense to move the av7110 driver back > from staging - it may continue to work, but it _is_ old and there is > no maintenance - and I would certainly suggest that any other > out-of-tree driver that uses these old interfaces that nothing else > implements shouldn't do so, considering that nothing else implements > them. Sorry for barging in between your discussion, but it seemed like you really missed some perspective and hence. My 2 cents worth: * Revert the header changes. * Let someone else with knowledge of it take over the maintenance of the av7110 driver. - This would allow other hardware also to be easily accommodated in a similar manner. * Pull the out of tree drivers and allocate maintenance of those, to someone who understands them. Don't you want more hardware to be supported out of the box ? Should there be no driver for those DVB output hardware, but only for V4L2 output devices ? There exists other hardware which As a person who worked on another av7110 like driver (saa716x based s2 6400), which I can confirm. The API is supposed to simplify life, not make it even more complex. These devices would need lot of workarounds to work with the API that which Mauro advocates, which might even break those drivers given their complexity and size. It would make life a lot easier for the users, Mauro himself and this long never ending discussion can be put to rest. Thanks, MA ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-25 2:55 ` Manu Abraham @ 2021-08-25 6:33 ` Mauro Carvalho Chehab 2021-08-25 16:16 ` Manu Abraham 0 siblings, 1 reply; 14+ messages in thread From: Mauro Carvalho Chehab @ 2021-08-25 6:33 UTC (permalink / raw) To: Manu Abraham Cc: Linus Torvalds, Soeren Moch, Linux Media Mailing List, linux-kernel Em Wed, 25 Aug 2021 08:25:57 +0530 Manu Abraham <abraham.manu@gmail.com> escreveu: > On Mon, Aug 23, 2021 at 10:30 PM Linus Torvalds > <torvalds@linux-foundation.org> wrote: > > > > I have reverted the header file move. But I would also heartily > > recommend that whatever user program includes those headers (VDR - > > anything else?) should take snapshots of these specific kernel > > headers. > > > > I'm not convinced that it makes sense to move the av7110 driver back > > from staging - it may continue to work, but it _is_ old and there is > > no maintenance - and I would certainly suggest that any other > > out-of-tree driver that uses these old interfaces that nothing else > > implements shouldn't do so, considering that nothing else implements > > them. > > Sorry for barging in between your discussion, but it seemed like you really > missed some perspective and hence. > > My 2 cents worth: > * Revert the header changes. > > * Let someone else with knowledge of it take over the maintenance > of the av7110 driver. > > - This would allow other hardware also to be easily accommodated > in a similar manner. > > * Pull the out of tree drivers and allocate maintenance of those, to > someone who understands them. Don't you want more hardware to be > supported out of the box ? > > Should there be no driver for those DVB output hardware, but only for > V4L2 output devices ? > > There exists other hardware which As a person who worked on another > av7110 like driver (saa716x based s2 6400), which I can confirm. > The API is supposed to simplify life, not make it even more complex. > These devices would need lot of workarounds to work with the API that > which Mauro advocates, which might even break those drivers given > their complexity and size. > > It would make life a lot easier for the users, Mauro himself and > this long never ending discussion can be put to rest. The "full-featured" API that it is implemented on av7110 always had troubles. This is not only my view, but also the view of the original API authors,as can be seen at the DVBv4 WIP documentation: https://www.linuxtv.org/downloads/legacy/linux-dvb-api-v4/linux-dvb-api-v4-0-3.pdf It clearly says that, on chapter 2.2: "2.2 Linux DVB API Version 3 problems ... The Linux DVB API Version 3 has very limited support for modern hardware." The "modern" there refers to hardware back in 2005! I worked on a project back 8 years ago that tried to use it for TV sets. It didn't work, because the API assumed a 1:1 mapping between tuners and A/V codecs, which works for simpler embedded hardware, but didn't cover smart TV hardware, where the number of frontends, demods and A/V codecs were different. You may even have multiple channels being displayed at the same time (Picture in Picture). On today's embedded hardware, you need something like the media controller, in order to dynamically re-configure the hardware pipelines between: - multiple tuners (DVB-C, DVB-T/T2, DVB-S/S2); - multiple demods[1]; - multiple A/V decoders; - display compositor; - audio I/O; - CA modules; - encrypt/decrypt hardware (required on some Countries in order to allow recording programs on storage); - storage. [1] There are even some demods that can dynamically change the maximum number of PIDs it can filter. Modern hardware can have, let's say, a max of 4 input MPEG-TS, and a max of 512 filters, which works like 4 independent demods, where the number of filters per demod is adjusted dynamically. This is currently problem for DVB subsystem, as it allocates statically the PID filters for the max number of PIDs, meaning that a large amount of RAM would be wasted if one would reserve space for the maximum possible capacity per demod (it would require space for 4x512 buffers on such hardware, meaning that 3/4 of buffer memory would be wasted). As I always said, I'm open to discuss an API that would properly address what's needed, but such API should support modern embedded hardware, and should be designed to allow it to be extended to support to future needs. That's not the case of the DVBv3 "full-feat" API, which was developed to support a hardware component developed more than 23 years ago[2]. [2] The Rev 3.1 datasheet of av7110 was written in June, 1998: https://pdf1.alldatasheet.com/datasheet-pdf/view/130554/TI/TMS320AV7110.html - From driver's perspective, it makes no sense to keep support for av7110, as TI stopped production of TMS320AV7110 a very long time ago. They don't even mention this product number anymore on their website. Thanks, Mauro ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-25 6:33 ` Mauro Carvalho Chehab @ 2021-08-25 16:16 ` Manu Abraham 2021-08-26 12:26 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 14+ messages in thread From: Manu Abraham @ 2021-08-25 16:16 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Linus Torvalds, Soeren Moch, Linux Media Mailing List, linux-kernel On Wed, Aug 25, 2021 at 12:03 PM Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote: > > Em Wed, 25 Aug 2021 08:25:57 +0530 > Manu Abraham <abraham.manu@gmail.com> escreveu: > > > On Mon, Aug 23, 2021 at 10:30 PM Linus Torvalds > > <torvalds@linux-foundation.org> wrote: > > > > > > I have reverted the header file move. But I would also heartily > > > recommend that whatever user program includes those headers (VDR - > > > anything else?) should take snapshots of these specific kernel > > > headers. > > > > > > I'm not convinced that it makes sense to move the av7110 driver back > > > from staging - it may continue to work, but it _is_ old and there is > > > no maintenance - and I would certainly suggest that any other > > > out-of-tree driver that uses these old interfaces that nothing else > > > implements shouldn't do so, considering that nothing else implements > > > them. > > > > Sorry for barging in between your discussion, but it seemed like you really > > missed some perspective and hence. > > > > My 2 cents worth: > > * Revert the header changes. > > > > * Let someone else with knowledge of it take over the maintenance > > of the av7110 driver. > > > > - This would allow other hardware also to be easily accommodated > > in a similar manner. > > > > * Pull the out of tree drivers and allocate maintenance of those, to > > someone who understands them. Don't you want more hardware to be > > supported out of the box ? > > > > Should there be no driver for those DVB output hardware, but only for > > V4L2 output devices ? > > > > There exists other hardware which As a person who worked on another > > av7110 like driver (saa716x based s2 6400), which I can confirm. > > The API is supposed to simplify life, not make it even more complex. > > These devices would need lot of workarounds to work with the API that > > which Mauro advocates, which might even break those drivers given > > their complexity and size. > > > > It would make life a lot easier for the users, Mauro himself and > > this long never ending discussion can be put to rest. > > The "full-featured" API that it is implemented on av7110 always had > troubles. This is not only my view, but also the view of the > original API authors,as can be seen at the DVBv4 WIP documentation: > > https://www.linuxtv.org/downloads/legacy/linux-dvb-api-v4/linux-dvb-api-v4-0-3.pdf > > It clearly says that, on chapter 2.2: > > "2.2 Linux DVB API Version 3 problems That's very misleading ! In fact, the legacy V3 API was upgraded to 3.1 and 3.2 and those issues were ironed out. You are talking about V3 while V3.2 fixed those issues. The V4 API documentation is legacy and was there even before the V3.2 API existed. It was even upgraded to V5, skipping V4 ! > > The Linux DVB API Version 3 has very limited support for > modern hardware." > > The "modern" there refers to hardware back in 2005! This is exactly what I wrote just above. > I worked on a project back 8 years ago that tried to use it for TV > sets. It didn't work, because the API assumed a 1:1 mapping between > tuners and A/V codecs, which works for simpler embedded hardware, > but didn't cover smart TV hardware, where the number of frontends, > demods and A/V codecs were different. You may even have multiple > channels being displayed at the same time (Picture in Picture). > > On today's embedded hardware, you need something like the media > controller, in order to dynamically re-configure the hardware > pipelines between: > > - multiple tuners (DVB-C, DVB-T/T2, DVB-S/S2); > - multiple demods[1]; > - multiple A/V decoders; > - display compositor; > - audio I/O; > - CA modules; > - encrypt/decrypt hardware (required on some Countries in order > to allow recording programs on storage); > - storage. Multiple frontends, tuners/demods, CAM's were already supported There is no encrypt/decrypt hardware, either you have hardware CAM's or SoftCAM's, which do the decryption for DVB streams. These already exist with the old API itself. The S2 6400, KNC S2 Twin and most others do have multiple first and second generation frontends. The DVB demux provides multiple PID's, you can have multiple PiP's whatever you want. For some SoC's with A/V codecs what you are saying is true. It does not make sense for PCTV hardware to use the pipelines you apparently describe. Such SoC's make use the extended API that you advocate, but the standard PCTV, or standard STB hardware we all are used to need not use the convoluted API being advocated. For those SoC's one may use the V4L2 output. But for DVB output devices, it makes no sense but going many steps backwards to use the V4L2 output. > From driver's perspective, it makes no sense to keep support for av7110, > as TI stopped production of TMS320AV7110 a very long time ago. They > don't even mention this product number anymore on their website. > What I meant: If there are some users for some hardware, it is impolite to rip them out, especially when someone is volunteering to maintain them. Rather than impolite, that's quite rude and arrogant. I believe that is the de facto Linux kernel principle still, unless there is some real reason to rip it out. FWIW, my 2c worth: a) leave the headers where they belong, already done by Linus. b) av7110: hand over the maintenance to someone who is happy and has time to fiddle around with c) Pull in the saa716x driver. I wrote the saa716x driver with NXP support and with additional help from the community. It would be good to maintain the credits to the original developers though. You can pull the saa716x driver from Soeren, if he needs some help, I can chip in whatever possible way. Let him have some fun with the driver. Mauro: Look, it's not good to convolute and pollute the discussion with stuff that are not relevant. Please! Have a heart; Don't do this drama.. People will just hate you for eons, for no good reason ! Friendly Regards, MA ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Regression 5.14] media: dvb userspace api 2021-08-25 16:16 ` Manu Abraham @ 2021-08-26 12:26 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2021-08-26 12:26 UTC (permalink / raw) To: Manu Abraham Cc: Linus Torvalds, Soeren Moch, Linux Media Mailing List, linux-kernel Em Wed, 25 Aug 2021 21:46:23 +0530 Manu Abraham <abraham.manu@gmail.com> escreveu: > > The "full-featured" API that it is implemented on av7110 always had > > troubles. This is not only my view, but also the view of the > > original API authors,as can be seen at the DVBv4 WIP documentation: > > > > https://www.linuxtv.org/downloads/legacy/linux-dvb-api-v4/linux-dvb-api-v4-0-3.pdf > > > > It clearly says that, on chapter 2.2: > > > > "2.2 Linux DVB API Version 3 problems > > > That's very misleading ! In fact, the legacy V3 API was upgraded to 3.1 and 3.2 > and those issues were ironed out. You are talking about V3 while V3.2 > fixed those > issues. No. When Linux version 2.6.12-rc2 started using git, the DVB API version was already 3.1. This was in April, 2005, which is the the same date that rev 0.3 of the DVBv4 spec was released[1]. [1] https://www.linuxtv.org/downloads/legacy/linux-dvb-api-v4/linux-dvb-api-v4-0-3.pdf DVB API version 3.2 was merged in 2007 on this commit: commit 2435be11ae1afb64ac7dfb25e10b6e3037ab0522 Author: Hans Verkuil <hverkuil@xs4all.nl> Date: Fri Apr 27 12:31:09 2007 -0300 V4L/DVB (5307): Add support for the cx23415 MPEG decoding features. The cx23415 adds some extra features that this DVB decoding API did not support. This API has been expanded to support the required features. Both source and binary backwards compatibility is kept intact by these changes. So existing applications are not affected. Signed-off-by: Hans Verkuil <hverkuil@xs4all.nl> Signed-off-by: Ralph Metzler <rjkm@metzlerbros.de> Signed-off-by: Oliver Endriss <o.endriss@gmx.de> Signed-off-by: Mauro Carvalho Chehab <mchehab@infradead.org> The focus of this change was to add support to better control a MPEG2 decoder on an analog TV hardware (IVTV). That didn't bring any uAPI change for av7110. Between 3.1 and 2435be11ae1a ("V4L/DVB (5307): Add support for the cx23415 MPEG decoding features."), there were a couple of changes: +#define AUDIO_GET_PTS _IOR('o', 19, __u64) +#define VIDEO_GET_PTS _IOR('o', 57, __u64) -#define DMX_GET_EVENT _IOR('o', 46, struct dmx_event) +#define FE_TUNE_MODE_ONESHOT 0x01 +#define FE_SET_FRONTEND_TUNE_MODE _IO('o', 81) /* unsigned int */ Those don't cover the main proposed changes at DVBv4. Btw, what it is said there at at chapter 2.2[1] is still true: "Because of the architectual problems of the core, the inconsitency of the API and the lack of zero-copy DMA it’s not possible to simply extend the existing API. A complete new design is inevitable." > > The "modern" there refers to hardware back in 2005! > This is exactly what I wrote just above. Precisely. > Multiple frontends, tuners/demods, CAM's were already supported > There is no encrypt/decrypt hardware, either you have hardware > CAM's or SoftCAM's, which do the decryption for DVB streams. > These already exist with the old API itself. Yes, support for multiple fontends/demux/demods were added, but the A/V API only supports a 1:1 mapping between demux---> demod: typedef enum { VIDEO_SOURCE_DEMUX, /* Select the demux as the main source */ VIDEO_SOURCE_MEMORY /* If this source is selected, the stream comes from the user through the write system call */ } video_stream_source_t; typedef enum { AUDIO_SOURCE_DEMUX, /* Select the demux as the main source */ AUDIO_SOURCE_MEMORY /* Select internal memory as the main source */ } audio_stream_source_t; On other words, zero-copy decoding is only possible with 1:1 mapping: demux0 should route the video PID to video0 codec, demux1 should route the video PID to video1 codec ... demux<n> should route the video PID to video<n> codec There's no way to route a PID from demux3 to be handled by video0. Btw, if demux0 is filtering multiple video channels and the video codec accepts only a single video, with the current API, what channel would be decoded by its video0 codec? There's no way to set it with the existing API. > The S2 6400, KNC S2 Twin and most others do have multiple first > and second generation frontends. > > The DVB demux provides multiple PID's, you can have multiple PiP's > whatever you want. There's no provision at the API to control any parameters of PiP (like setting the position/size of the overlay area). Also, chipsets for TV sets expect zero-copy transfers between video decoders and GPU DRM planes. Most of such hardware are implemented with two separate chips (or two separate areas at the same silicon): - GPU + ISP + video memory; - ARM CPU. On such hardware, copying buffers via CPU is a very expensive operation. So, hardware pipelines should be programmed. For instance: frontend1 -> demux3 -> video0 -> DMA buffer 0 frontend1 -> demux3 --OSD pid--> DMA buffer 2 frontend1 -> demux3 --audio pid--> Audio input 0 frontend0 -> demux0 -> video1 -> DMA buffer 1 frontend0 -> demux0 --OSD pid--> DMA buffer 3 Then, the GPU's compositor will be programmed to properly place each DMA buffer at the composed PiP display, on whatever position the second video will be positioned at the composite screen. > For some SoC's with A/V codecs what you are saying is true. > It does not make sense for PCTV hardware to use the pipelines > you apparently describe. Such SoC's make use the extended API that > you advocate, but the standard PCTV, or standard STB hardware > we all are used to need not use the convoluted API being advocated. > For those SoC's one may use the V4L2 output. But for DVB output > devices, it makes no sense but going many steps backwards to use > the V4L2 output. The existing API works for simple hardware like av7110, but, in order to support what current chipsets provide, it has to be re-designed. As explained said at DVBv4 session 2.2[1]: "Linux DVB API Version 3 was focussed on the popular Siemens PCI DVB card." > > From driver's perspective, it makes no sense to keep support for av7110, > > as TI stopped production of TMS320AV7110 a very long time ago. They > > don't even mention this product number anymore on their website. > > > > What I meant: If there are some users for some hardware, it is impolite > to rip them out, especially when someone is volunteering to maintain them. > Rather than impolite, that's quite rude and arrogant. > > I believe that is the de facto Linux kernel principle still, unless > there is some > real reason to rip it out. > > FWIW, my 2c worth: > a) leave the headers where they belong, already done by Linus. Linus actually asked to copy such headers to the VDR source code. > b) av7110: hand over the maintenance to someone who is happy and has > time to fiddle around with Nobody that actually uses av7110 hardware (if are there any users for such hardware nowadays) ever sent a single patch upstream for quite a long time. See, from 2013 to today, there were 81 patches applied on it: $ git lg --since 2013 --follow -- drivers/media/pci/ttpci/|wc -l 81 None of those were produced by someone actually using av7110. No av7110 users ever replied to any of those patches with a Tested-by. So, nobody has shown any time/interest on maintaining it upstream for quite a long time. > c) Pull in the saa716x driver. I wrote the saa716x driver with NXP support > and with additional help from the community. It would be good to maintain > the credits to the original developers though. > > You can pull the saa716x driver from Soeren, if he needs some help, > I can chip in whatever possible way. Let him have some fun with the driver. It won't make any good to upstream a driver before discussing the API. Even low-end PC hardware (like those with Atom CPUs) nowadays are capable of decoding MPEG2 - and other video codecs - in hardware (at the GPU). This was a reality even back in 2005, as said at the DVBv4 doc, at section 2.1[1]: 'PCs and embedded platforms are divering. For PCs, new cards are only available as ”budget” cards, which means that they only provide the full, raw, unmodified TS to the system and put the burden of handling the data to the main CPU. On embedded platforms, however, dedicated STB/IDTV chipsets demultiplex the data for direct application use and specialized hardware or firmware on DSPs relieves the main CPU greatly.' As Honza pointed, a large amount of users of such API are the ones that have a DreamBox STB and decided to build their own firmware (like opendreambox), running a Kernel based on downstream patches[2]. Clearly, the main usage for a full-feat api is on Embedded hardware. [2] For them, it doesn't matter what API is at the upstream Kernel. All that matters is that the userspace software (like VDR) shall implement whatever API is used to communicate with the Enigma/Enigma2 DVB drivers. Thanks, Mauro ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-08-26 12:26 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-11 12:15 [Regression 5.14] media: dvb userspace api Soeren Moch 2021-08-19 11:31 ` Mauro Carvalho Chehab 2021-08-21 13:58 ` Manu Abraham 2021-08-22 15:21 ` Soeren Moch 2021-08-22 17:47 ` Mauro Carvalho Chehab 2021-08-23 14:59 ` Soeren Moch 2021-08-23 16:58 ` Linus Torvalds 2021-08-23 20:16 ` Soeren Moch 2021-08-24 7:47 ` Mauro Carvalho Chehab 2021-08-24 20:01 ` Honza P 2021-08-25 2:55 ` Manu Abraham 2021-08-25 6:33 ` Mauro Carvalho Chehab 2021-08-25 16:16 ` Manu Abraham 2021-08-26 12:26 ` Mauro Carvalho Chehab
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).