Re: [v4,00/14] ASoC: Sound Open Firmware (SOF) core

Re: [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Xiang Xiao]

Should we utilize official IPC frameowrk instead reinverting the wheel?
1.Load firmware by drivers/remoteproc
  https://www.kernel.org/doc/Documentation/remoteproc.txt
2.Do the comunication through drivers/rpmsg
  https://www.kernel.org/doc/Documentation/rpmsg.txt
Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
Actually, remoteproc/rpmsg is much better than SOF IPC because:
1.Completely isolate the firmware load and message transfer:
  The same rpmsg driver could run on any remote processor
2.Separate the application protocol from transfer layer:
  One remote processor could host many rpmsg services
3.Completely follow kernel driver model(rpsmg_bus, rpmsg_device and rpmsg_driver).
4.Support by many RTOS(Bare Metal, FreeRTOS, Zephyr, NuttX, Nucleus, uC/OS...) for remote side:
  https://github.com/OpenAMP/open-amp
  https://github.com/NXPmicro/rpmsg-lite
5.Maintained by the standard committee:
  https://www.multicore-association.org/workgroup/oamp.php
  https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio
Since the keypoint of SOF is the platform agnostic and modular, please use the standard technique here.

Thanks
Xiang

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Srinivas Kandagatla]

On 18/02/2019 20:03, Xiang Xiao wrote:
> Should we utilize official IPC frameowrk instead reinverting the wheel?
> 1.Load firmware by drivers/remoteproc
>    https://www.kernel.org/doc/Documentation/remoteproc.txt
> 2.Do the comunication through drivers/rpmsg
>    https://www.kernel.org/doc/Documentation/rpmsg.txt
> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?

It definitely makes more sense to use rpmsg for Generic IPC driver here.

Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will 
definitely help everyone in future while immigrating to SOF.

> Actually, remoteproc/rpmsg is much better than SOF IPC because:
> 1.Completely isolate the firmware load and message transfer:
>    The same rpmsg driver could run on any remote processor
> 2.Separate the application protocol from transfer layer:
>    One remote processor could host many rpmsg services
> 3.Completely follow kernel driver model(rpsmg_bus, rpmsg_device and rpmsg_driver).
> 4.Support by many RTOS(Bare Metal, FreeRTOS, Zephyr, NuttX, Nucleus, uC/OS...) for remote side:
>    https://github.com/OpenAMP/open-amp
>    https://github.com/NXPmicro/rpmsg-lite
> 5.Maintained by the standard committee:
>    https://www.multicore-association.org/workgroup/oamp.php
>    https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio
> Since the keypoint of SOF is the platform agnostic and modular, please use the standard technique here.
> 
> Thanks
> Xiang
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
> 
--srini

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[xiang xiao]

On Tue, Feb 19, 2019 at 5:49 PM Srinivas Kandagatla
<srinivas.kandagatla@linaro.org> wrote:
>
>
>
> On 18/02/2019 20:03, Xiang Xiao wrote:
> > Should we utilize official IPC frameowrk instead reinverting the wheel?
> > 1.Load firmware by drivers/remoteproc
> >    https://www.kernel.org/doc/Documentation/remoteproc.txt
> > 2.Do the comunication through drivers/rpmsg
> >    https://www.kernel.org/doc/Documentation/rpmsg.txt
> > Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
>
> It definitely makes more sense to use rpmsg for Generic IPC driver here.
>
> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
> definitely help everyone in future while immigrating to SOF.
>

Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
fully integrate with the ASoC topology framework, and the firmware is
base on FreeRTOS and OpenMAX.
SOF initiative is very good and exciting, our team members(include me)
spend a couple weeks to study the current code base on both firmware
and kernel side, we even port SOF to our DSP/MCU and make it run, but
I have to point out that:
SOF IPC is too simple and rigid, tightly couple with Intel platform
and audio domain, which make:
   a.It's difficult to integrate with other vendor SoC, especially if
other vendor already adopt remote/rpmsg(this is already a trend!).
   b.It's difficult to add other IPC services for example:
      i.Audio DSP talk to power MCU to adjust clock and voltage
      ii.Export ultrasonic distance measurement to IIO subsystem

> > Actually, remoteproc/rpmsg is much better than SOF IPC because:
> > 1.Completely isolate the firmware load and message transfer:
> >    The same rpmsg driver could run on any remote processor
> > 2.Separate the application protocol from transfer layer:
> >    One remote processor could host many rpmsg services
> > 3.Completely follow kernel driver model(rpsmg_bus, rpmsg_device and rpmsg_driver).
> > 4.Support by many RTOS(Bare Metal, FreeRTOS, Zephyr, NuttX, Nucleus, uC/OS...) for remote side:
> >    https://github.com/OpenAMP/open-amp
> >    https://github.com/NXPmicro/rpmsg-lite
> > 5.Maintained by the standard committee:
> >    https://www.multicore-association.org/workgroup/oamp.php
> >    https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio
> > Since the keypoint of SOF is the platform agnostic and modular, please use the standard technique here.
> >
> > Thanks
> > Xiang
> > _______________________________________________
> > Alsa-devel mailing list
> > Alsa-devel@alsa-project.org
> > http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
> >
> --srini

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Pierre-Louis Bossart]

On 2/19/19 9:09 AM, xiang xiao wrote:
> On Tue, Feb 19, 2019 at 5:49 PM Srinivas Kandagatla
> <srinivas.kandagatla@linaro.org> wrote:
>>
>>
>> On 18/02/2019 20:03, Xiang Xiao wrote:
>>> Should we utilize official IPC frameowrk instead reinverting the wheel?
>>> 1.Load firmware by drivers/remoteproc
>>>     https://www.kernel.org/doc/Documentation/remoteproc.txt
>>> 2.Do the comunication through drivers/rpmsg
>>>     https://www.kernel.org/doc/Documentation/rpmsg.txt
>>> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
>> It definitely makes more sense to use rpmsg for Generic IPC driver here.
>>
>> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
>> definitely help everyone in future while immigrating to SOF.
>>
> Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
> fully integrate with the ASoC topology framework, and the firmware is
> base on FreeRTOS and OpenMAX.
> SOF initiative is very good and exciting, our team members(include me)
> spend a couple weeks to study the current code base on both firmware
> and kernel side, we even port SOF to our DSP/MCU and make it run, but
> I have to point out that:
> SOF IPC is too simple and rigid, tightly couple with Intel platform
> and audio domain, which make:
>     a.It's difficult to integrate with other vendor SoC, especially if
> other vendor already adopt remote/rpmsg(this is already a trend!).
>     b.It's difficult to add other IPC services for example:
>        i.Audio DSP talk to power MCU to adjust clock and voltage
>        ii.Export ultrasonic distance measurement to IIO subsystem

The IPC scheme suggested in this patchset is only a first pass that 
works on 3 generations on Intel platforms + the QEMU  parts. There are 
no claims that the current solution is set-in-stone, and this is already 
an area where things are already changing to support notifications and 
low-power transitions.

There will clearly be evolutions to make the IPC more flexible/generic, 
but we've got to start somewhere and bear in mind that we also have to 
support memory-constrained legacy devices where such generic frameworks 
aren't needed or even implementable. Some of your proposals such as 
changing power/clocks with a firmware request aren't necessarily 
possible or recommended on all platforms - i can already hear security 
folks howling, this was already mentioned in the GitHub thread.

There are other paths such as using the mailbox framework, and at the 
end of the day the IPC is likely going to be a configurable element 
where integrators pick what's best for them. If you strongly believe 
that the RPMSG framework is the way to go, there is a public github and 
you can contribute the relevant changes with both kernel and firmware 
patches.

>
>>> Actually, remoteproc/rpmsg is much better than SOF IPC because:
>>> 1.Completely isolate the firmware load and message transfer:
>>>     The same rpmsg driver could run on any remote processor
>>> 2.Separate the application protocol from transfer layer:
>>>     One remote processor could host many rpmsg services
>>> 3.Completely follow kernel driver model(rpsmg_bus, rpmsg_device and rpmsg_driver).
>>> 4.Support by many RTOS(Bare Metal, FreeRTOS, Zephyr, NuttX, Nucleus, uC/OS...) for remote side:
>>>     https://github.com/OpenAMP/open-amp
>>>     https://github.com/NXPmicro/rpmsg-lite
>>> 5.Maintained by the standard committee:
>>>     https://www.multicore-association.org/workgroup/oamp.php
>>>     https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio
>>> Since the keypoint of SOF is the platform agnostic and modular, please use the standard technique here.

As stated above there is no opposition to using fancier/more generic IPC 
solutions, but they have to be considered as evolutions and follow the 
due process of contributions/reviews plus be aligned with product timelines.

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Vinod Koul]

Hi Pierre,

On 19-02-19, 09:55, Pierre-Louis Bossart wrote:
> 
> On 2/19/19 9:09 AM, xiang xiao wrote:
> > On Tue, Feb 19, 2019 at 5:49 PM Srinivas Kandagatla
> > <srinivas.kandagatla@linaro.org> wrote:
> > > 
> > > 
> > > On 18/02/2019 20:03, Xiang Xiao wrote:
> > > > Should we utilize official IPC frameowrk instead reinverting the wheel?
> > > > 1.Load firmware by drivers/remoteproc
> > > >     https://www.kernel.org/doc/Documentation/remoteproc.txt
> > > > 2.Do the comunication through drivers/rpmsg
> > > >     https://www.kernel.org/doc/Documentation/rpmsg.txt
> > > > Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
> > > It definitely makes more sense to use rpmsg for Generic IPC driver here.
> > > 
> > > Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
> > > definitely help everyone in future while immigrating to SOF.
> > > 
> > Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
> > fully integrate with the ASoC topology framework, and the firmware is
> > base on FreeRTOS and OpenMAX.
> > SOF initiative is very good and exciting, our team members(include me)
> > spend a couple weeks to study the current code base on both firmware
> > and kernel side, we even port SOF to our DSP/MCU and make it run, but
> > I have to point out that:
> > SOF IPC is too simple and rigid, tightly couple with Intel platform
> > and audio domain, which make:
> >     a.It's difficult to integrate with other vendor SoC, especially if
> > other vendor already adopt remote/rpmsg(this is already a trend!).
> >     b.It's difficult to add other IPC services for example:
> >        i.Audio DSP talk to power MCU to adjust clock and voltage
> >        ii.Export ultrasonic distance measurement to IIO subsystem
> 
> The IPC scheme suggested in this patchset is only a first pass that works on
> 3 generations on Intel platforms + the QEMU� parts. There are no claims that
> the current solution is set-in-stone, and this is already an area where
> things are already changing to support notifications and low-power
> transitions.
> 
> There will clearly be evolutions to make the IPC more flexible/generic, but
> we've got to start somewhere and bear in mind that we also have to support
> memory-constrained legacy devices where such generic frameworks aren't
> needed or even implementable. Some of your proposals such as changing
> power/clocks with a firmware request aren't necessarily possible or
> recommended on all platforms - i can already hear security folks howling,
> this was already mentioned in the GitHub thread.

Rather than evolve the IPC, i would say it makes more sense that we
"reuse" existing upstream frameworks.. As given below by xiang
this seems to have support for RTOSes (see point 4 below) and looking at
below it seems to have much better coverage across systems.

This should also help in easy adoption of SoF for non Intel people...

Also looking at it, lot of IPC code, DSP loading etc would go away
making SoF code lesser in footprint.

I think benefits outweigh the effort of porting to a framework which is
already upstream and used on many platforms for different vendors!

> There are other paths such as using the mailbox framework, and at the end of
> the day the IPC is likely going to be a configurable element where
> integrators pick what's best for them. If you strongly believe that the
> RPMSG framework is the way to go, there is a public github and you can
> contribute the relevant changes with both kernel and firmware patches.
> 
> > 
> > > > Actually, remoteproc/rpmsg is much better than SOF IPC because:
> > > > 1.Completely isolate the firmware load and message transfer:
> > > >     The same rpmsg driver could run on any remote processor
> > > > 2.Separate the application protocol from transfer layer:
> > > >     One remote processor could host many rpmsg services
> > > > 3.Completely follow kernel driver model(rpsmg_bus, rpmsg_device and rpmsg_driver).
> > > > 4.Support by many RTOS(Bare Metal, FreeRTOS, Zephyr, NuttX, Nucleus, uC/OS...) for remote side:
> > > >     https://github.com/OpenAMP/open-amp
> > > >     https://github.com/NXPmicro/rpmsg-lite

^^^

> > > > 5.Maintained by the standard committee:
> > > >     https://www.multicore-association.org/workgroup/oamp.php
> > > >     https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio
> > > > Since the keypoint of SOF is the platform agnostic and modular, please use the standard technique here.
> 
> As stated above there is no opposition to using fancier/more generic IPC
> solutions, but they have to be considered as evolutions and follow the due
> process of contributions/reviews plus be aligned with product timelines.

Sorry but upstream cares more about doing the right things rather than
vendor timelines..

-- 
~Vinod

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Arnaud Pouliquen]

Hello,

On 2/21/19 5:39 AM, Vinod Koul wrote:
> Hi Pierre,
> 
> On 19-02-19, 09:55, Pierre-Louis Bossart wrote:
>> 
>> On 2/19/19 9:09 AM, xiang xiao wrote:
>> > On Tue, Feb 19, 2019 at 5:49 PM Srinivas Kandagatla
>> > <srinivas.kandagatla@linaro.org> wrote:
>> > > 
>> > > 
>> > > On 18/02/2019 20:03, Xiang Xiao wrote:
>> > > > Should we utilize official IPC frameowrk instead reinverting the wheel?
>> > > > 1.Load firmware by drivers/remoteproc
>> > > >     https://www.kernel.org/doc/Documentation/remoteproc.txt
>> > > > 2.Do the comunication through drivers/rpmsg
>> > > >     https://www.kernel.org/doc/Documentation/rpmsg.txt
>> > > > Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
>> > > It definitely makes more sense to use rpmsg for Generic IPC driver here.
>> > > 
>> > > Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
>> > > definitely help everyone in future while immigrating to SOF.
>> > > 
>> > Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
>> > fully integrate with the ASoC topology framework, and the firmware is
>> > base on FreeRTOS and OpenMAX.
>> > SOF initiative is very good and exciting, our team members(include me)
>> > spend a couple weeks to study the current code base on both firmware
>> > and kernel side, we even port SOF to our DSP/MCU and make it run, but
>> > I have to point out that:
>> > SOF IPC is too simple and rigid, tightly couple with Intel platform
>> > and audio domain, which make:
>> >     a.It's difficult to integrate with other vendor SoC, especially if
>> > other vendor already adopt remote/rpmsg(this is already a trend!).
>> >     b.It's difficult to add other IPC services for example:
>> >        i.Audio DSP talk to power MCU to adjust clock and voltage
>> >        ii.Export ultrasonic distance measurement to IIO subsystem
>> 
>> The IPC scheme suggested in this patchset is only a first pass that works on
>> 3 generations on Intel platforms + the QEMU  parts. There are no claims that
>> the current solution is set-in-stone, and this is already an area where
>> things are already changing to support notifications and low-power
>> transitions.
>> 
>> There will clearly be evolutions to make the IPC more flexible/generic, but
>> we've got to start somewhere and bear in mind that we also have to support
>> memory-constrained legacy devices where such generic frameworks aren't
>> needed or even implementable. Some of your proposals such as changing
>> power/clocks with a firmware request aren't necessarily possible or
>> recommended on all platforms - i can already hear security folks howling,
>> this was already mentioned in the GitHub thread.
> 
> Rather than evolve the IPC, i would say it makes more sense that we
> "reuse" existing upstream frameworks.. As given below by xiang
> this seems to have support for RTOSes (see point 4 below) and looking at
> below it seems to have much better coverage across systems.
> 
> This should also help in easy adoption of SoF for non Intel people...
> 
> Also looking at it, lot of IPC code, DSP loading etc would go away
> making SoF code lesser in footprint.
> 
> I think benefits outweigh the effort of porting to a framework which is
> already upstream and used on many platforms for different vendors!

Just for information... ST Microelectronics plans to port SOF to at
least one of its platforms (based on ARM cores). Today, for the
co-processor  management we use the rpmsg and remoteproc frameworks on
the Linux kernel side and OpenAMP on the remote processor side. We are
therefore interested in xiang's work.
An advantage we see in this generic solution is that compatibility
between the Linux kernel frameworks and the OpenAMP library is ensured
through discussions in the Linux and OpenAMP communities, involving
maintainers and several vendors.

Regards
Arnaud

> 
>> There are other paths such as using the mailbox framework, and at the end of
>> the day the IPC is likely going to be a configurable element where
>> integrators pick what's best for them. If you strongly believe that the
>> RPMSG framework is the way to go, there is a public github and you can
>> contribute the relevant changes with both kernel and firmware patches.
>> 
>> > 
>> > > > Actually, remoteproc/rpmsg is much better than SOF IPC because:
>> > > > 1.Completely isolate the firmware load and message transfer:
>> > > >     The same rpmsg driver could run on any remote processor
>> > > > 2.Separate the application protocol from transfer layer:
>> > > >     One remote processor could host many rpmsg services
>> > > > 3.Completely follow kernel driver model(rpsmg_bus, rpmsg_device and rpmsg_driver).
>> > > > 4.Support by many RTOS(Bare Metal, FreeRTOS, Zephyr, NuttX, Nucleus, uC/OS...) for remote side:
>> > > >     https://github.com/OpenAMP/open-amp
>> > > >     https://github.com/NXPmicro/rpmsg-lite
> 
> ^^^
> 
>> > > > 5.Maintained by the standard committee:
>> > > >     https://www.multicore-association.org/workgroup/oamp.php
>> > > >     https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=virtio
>> > > > Since the keypoint of SOF is the platform agnostic and modular, please use the standard technique here.
>> 
>> As stated above there is no opposition to using fancier/more generic IPC
>> solutions, but they have to be considered as evolutions and follow the due
>> process of contributions/reviews plus be aligned with product timelines.
> 
> Sorry but upstream cares more about doing the right things rather than
> vendor timelines..
> 
> -- 
> ~Vinod

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1696 bytes --]

On Thu, Feb 21, 2019 at 11:42:25AM +0100, Arnaud Pouliquen wrote:
> On 2/21/19 5:39 AM, Vinod Koul wrote:
> > On 19-02-19, 09:55, Pierre-Louis Bossart wrote:
> >> On 2/19/19 9:09 AM, xiang xiao wrote:

> > Rather than evolve the IPC, i would say it makes more sense that we
> > "reuse" existing upstream frameworks.. As given below by xiang
> > this seems to have support for RTOSes (see point 4 below) and looking at
> > below it seems to have much better coverage across systems.

> > This should also help in easy adoption of SoF for non Intel people...

> > Also looking at it, lot of IPC code, DSP loading etc would go away
> > making SoF code lesser in footprint.

> > I think benefits outweigh the effort of porting to a framework which is
> > already upstream and used on many platforms for different vendors!

> Just for information... ST Microelectronics plans to port SOF to at
> least one of its platforms (based on ARM cores). Today, for the
> co-processor  management we use the rpmsg and remoteproc frameworks on
> the Linux kernel side and OpenAMP on the remote processor side. We are
> therefore interested in xiang's work.
> An advantage we see in this generic solution is that compatibility
> between the Linux kernel frameworks and the OpenAMP library is ensured
> through discussions in the Linux and OpenAMP communities, involving
> maintainers and several vendors.

This is a very good point, especially with people actually jumping on it
if we can avoid having multiple ABIs to worry about that would make life
a lot easier.  We should at least figure out if it will be disruptive to
adapt it later on, even if Intel ends up continuing to use a custom
system integration.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Pierre-Louis Bossart]

[-- Attachment #1: Type: text/plain, Size: 4463 bytes --]


>>>>> Should we utilize official IPC frameowrk instead reinverting the wheel?
>>>>> 1.Load firmware by drivers/remoteproc
>>>>>      https://www.kernel.org/doc/Documentation/remoteproc.txt
>>>>> 2.Do the comunication through drivers/rpmsg
>>>>>      https://www.kernel.org/doc/Documentation/rpmsg.txt
>>>>> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
>>>> It definitely makes more sense to use rpmsg for Generic IPC driver here.
>>>>
>>>> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
>>>> definitely help everyone in future while immigrating to SOF.
>>>>
>>> Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
>>> fully integrate with the ASoC topology framework, and the firmware is
>>> base on FreeRTOS and OpenMAX.
>>> SOF initiative is very good and exciting, our team members(include me)
>>> spend a couple weeks to study the current code base on both firmware
>>> and kernel side, we even port SOF to our DSP/MCU and make it run, but
>>> I have to point out that:
>>> SOF IPC is too simple and rigid, tightly couple with Intel platform
>>> and audio domain, which make:
>>>      a.It's difficult to integrate with other vendor SoC, especially if
>>> other vendor already adopt remote/rpmsg(this is already a trend!).
>>>      b.It's difficult to add other IPC services for example:
>>>         i.Audio DSP talk to power MCU to adjust clock and voltage
>>>         ii.Export ultrasonic distance measurement to IIO subsystem
>> The IPC scheme suggested in this patchset is only a first pass that works on
>> 3 generations on Intel platforms + the QEMU  parts. There are no claims that
>> the current solution is set-in-stone, and this is already an area where
>> things are already changing to support notifications and low-power
>> transitions.
>>
>> There will clearly be evolutions to make the IPC more flexible/generic, but
>> we've got to start somewhere and bear in mind that we also have to support
>> memory-constrained legacy devices where such generic frameworks aren't
>> needed or even implementable. Some of your proposals such as changing
>> power/clocks with a firmware request aren't necessarily possible or
>> recommended on all platforms - i can already hear security folks howling,
>> this was already mentioned in the GitHub thread.
> Rather than evolve the IPC, i would say it makes more sense that we
> "reuse" existing upstream frameworks.. As given below by xiang
> this seems to have support for RTOSes (see point 4 below) and looking at
> below it seems to have much better coverage across systems.
>
> This should also help in easy adoption of SoF for non Intel people...
>
> Also looking at it, lot of IPC code, DSP loading etc would go away
> making SoF code lesser in footprint.
>
> I think benefits outweigh the effort of porting to a framework which is
> already upstream and used on many platforms for different vendors!

There is no free lunch. There are 'features' of RPMsg which aren't 
necessarily great for all platforms, e.g. the concepts of virtio-like 
rings for IPC with available/used buffers for both directions are not a 
good match or replacement for the memory-window-based IPC on Intel 
platforms, where there is no DDR access, a small window allocated by 
firmware and only a couple of doorbell registers for essentially serial 
communication. The resources embedded in a firmware file is another 
capability that doesn't align with the way the SOF firmware is 
generated. I also don't know where the topology file would be handled, 
nor how to deal with suspend-resume where the DSP needs to be restarted. 
For folks who need an introduction to RPMsg, the link [1] is the best I 
found to scope out the work required.

In short, I don't mind looking at RPMsg as an option and would welcome 
contributions, but making it the default raises a number of technical 
challenges that can't be dismissed just yet, and such a transition isn't 
going to happen overnight. There are other evolutions that were 
mentioned as well, such as using the MFD framework to split the driver 
in 'core/hardware' support and application-specific parts (audio, 
sensors, etc), and likewise we need time to make it happen - just like 
we need time to move to the modern dailinks, add multi-cpu and SoundWire 
support, add digital domains, etc.

[1] http://processors.wiki.ti.com/index.php/PRU-ICSS_Remoteproc_and_RPMsg



[-- Attachment #2: Type: text/html, Size: 5508 bytes --]

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Pierre-Louis Bossart]

>> Just for information... ST Microelectronics plans to port SOF to at
>> least one of its platforms (based on ARM cores). Today, for the
>> co-processor  management we use the rpmsg and remoteproc frameworks on
>> the Linux kernel side and OpenAMP on the remote processor side. We are
>> therefore interested in xiang's work.
>> An advantage we see in this generic solution is that compatibility
>> between the Linux kernel frameworks and the OpenAMP library is ensured
>> through discussions in the Linux and OpenAMP communities, involving
>> maintainers and several vendors.
Thanks Arnaud for sharing this information. You made my day.� We've had 
to deal with a number of hurdles related to platform and tools 
availability, so it's good news indeed if there are more platforms 
endorsing SOF.
> This is a very good point, especially with people actually jumping on it
> if we can avoid having multiple ABIs to worry about that would make life
> a lot easier.  We should at least figure out if it will be disruptive to
> adapt it later on, even if Intel ends up continuing to use a custom
> system integration.
Yes indeed. At a high level we only have very generic send/receive IPC 
messages, and it really shouldn't matter how the transport happens.
However the actual contents/semantics of the messages do matter, and 
I'll have to run a check to make sure it scales.

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[xiang xiao]

On Thu, Feb 21, 2019 at 11:27 PM Pierre-Louis Bossart
<pierre-louis.bossart@linux.intel.com> wrote:
>
>
> Should we utilize official IPC frameowrk instead reinverting the wheel?
> 1.Load firmware by drivers/remoteproc
>     https://www.kernel.org/doc/Documentation/remoteproc.txt
> 2.Do the comunication through drivers/rpmsg
>     https://www.kernel.org/doc/Documentation/rpmsg.txt
> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
>
> It definitely makes more sense to use rpmsg for Generic IPC driver here.
>
> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
> definitely help everyone in future while immigrating to SOF.
>
> Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
> fully integrate with the ASoC topology framework, and the firmware is
> base on FreeRTOS and OpenMAX.
> SOF initiative is very good and exciting, our team members(include me)
> spend a couple weeks to study the current code base on both firmware
> and kernel side, we even port SOF to our DSP/MCU and make it run, but
> I have to point out that:
> SOF IPC is too simple and rigid, tightly couple with Intel platform
> and audio domain, which make:
>     a.It's difficult to integrate with other vendor SoC, especially if
> other vendor already adopt remote/rpmsg(this is already a trend!).
>     b.It's difficult to add other IPC services for example:
>        i.Audio DSP talk to power MCU to adjust clock and voltage
>        ii.Export ultrasonic distance measurement to IIO subsystem
>
> The IPC scheme suggested in this patchset is only a first pass that works on
> 3 generations on Intel platforms + the QEMU  parts. There are no claims that
> the current solution is set-in-stone, and this is already an area where
> things are already changing to support notifications and low-power
> transitions.
>
> There will clearly be evolutions to make the IPC more flexible/generic, but
> we've got to start somewhere and bear in mind that we also have to support
> memory-constrained legacy devices where such generic frameworks aren't
> needed or even implementable. Some of your proposals such as changing
> power/clocks with a firmware request aren't necessarily possible or
> recommended on all platforms - i can already hear security folks howling,
> this was already mentioned in the GitHub thread.
>
> Rather than evolve the IPC, i would say it makes more sense that we
> "reuse" existing upstream frameworks.. As given below by xiang
> this seems to have support for RTOSes (see point 4 below) and looking at
> below it seems to have much better coverage across systems.
>
> This should also help in easy adoption of SoF for non Intel people...
>
> Also looking at it, lot of IPC code, DSP loading etc would go away
> making SoF code lesser in footprint.
>
> I think benefits outweigh the effort of porting to a framework which is
> already upstream and used on many platforms for different vendors!
>
> There is no free lunch. There are 'features' of RPMsg which aren't necessarily great for all platforms, e.g. the concepts of virtio-like rings for IPC with available/used buffers for both directions are not a good match or replacement for the memory-window-based IPC on Intel platforms, where there is no DDR access, a small window allocated by firmware and only a couple of doorbell registers for essentially serial communication.

rpmsg support to define the custom mechanism(see rpmsg_endpoint_ops in
drivers\rpmsg\rpmsg_internal.h) but keep the upper layer API, qcomm
utilize this for glink and smd actually.

> The resources embedded in a firmware file is another capability that doesn't align with the way the SOF firmware is generated. I also don't know where the topology file would be handled, nor how to deal with suspend-resume where the DSP needs to be restarted. For folks who need an introduction to RPMsg, the link [1] is the best I found to scope out the work required.
>

We can share our rpmsg based topology implementation as reference which:
1.About 2500 lines(much less than SOF)
2.Support pcm and compress playback/capture
3.No any vendor dependence(thanks for rpmsg/remoteproc)

> In short, I don't mind looking at RPMsg as an option and would welcome contributions, but making it the default raises a number of technical challenges that can't be dismissed just yet, and such a transition isn't going to happen overnight. There are other evolutions that were mentioned as well, such as using the MFD framework to split the driver in 'core/hardware' support and application-specific parts (audio, sensors, etc), and likewise we need time to make it happen - just like we need time to move to the modern dailinks, add multi-cpu and SoundWire support, add digital domains, etc.
>

> [1] http://processors.wiki.ti.com/index.php/PRU-ICSS_Remoteproc_and_RPMsg
>
>

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[Keyon Jie]

On 2019/2/22 下午4:32, xiang xiao wrote:
> On Thu, Feb 21, 2019 at 11:27 PM Pierre-Louis Bossart
> <pierre-louis.bossart@linux.intel.com> wrote:
>>
>>
>> Should we utilize official IPC frameowrk instead reinverting the wheel?
>> 1.Load firmware by drivers/remoteproc
>>      https://www.kernel.org/doc/Documentation/remoteproc.txt
>> 2.Do the comunication through drivers/rpmsg
>>      https://www.kernel.org/doc/Documentation/rpmsg.txt
>> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
>>
>> It definitely makes more sense to use rpmsg for Generic IPC driver here.
>>
>> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
>> definitely help everyone in future while immigrating to SOF.
>>
>> Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
>> fully integrate with the ASoC topology framework, and the firmware is
>> base on FreeRTOS and OpenMAX.
>> SOF initiative is very good and exciting, our team members(include me)
>> spend a couple weeks to study the current code base on both firmware
>> and kernel side, we even port SOF to our DSP/MCU and make it run, but
>> I have to point out that:
>> SOF IPC is too simple and rigid, tightly couple with Intel platform
>> and audio domain, which make:
>>      a.It's difficult to integrate with other vendor SoC, especially if
>> other vendor already adopt remote/rpmsg(this is already a trend!).
>>      b.It's difficult to add other IPC services for example:
>>         i.Audio DSP talk to power MCU to adjust clock and voltage
>>         ii.Export ultrasonic distance measurement to IIO subsystem
>>
>> The IPC scheme suggested in this patchset is only a first pass that works on
>> 3 generations on Intel platforms + the QEMU  parts. There are no claims that
>> the current solution is set-in-stone, and this is already an area where
>> things are already changing to support notifications and low-power
>> transitions.
>>
>> There will clearly be evolutions to make the IPC more flexible/generic, but
>> we've got to start somewhere and bear in mind that we also have to support
>> memory-constrained legacy devices where such generic frameworks aren't
>> needed or even implementable. Some of your proposals such as changing
>> power/clocks with a firmware request aren't necessarily possible or
>> recommended on all platforms - i can already hear security folks howling,
>> this was already mentioned in the GitHub thread.
>>
>> Rather than evolve the IPC, i would say it makes more sense that we
>> "reuse" existing upstream frameworks.. As given below by xiang
>> this seems to have support for RTOSes (see point 4 below) and looking at
>> below it seems to have much better coverage across systems.
>>
>> This should also help in easy adoption of SoF for non Intel people...
>>
>> Also looking at it, lot of IPC code, DSP loading etc would go away
>> making SoF code lesser in footprint.
>>
>> I think benefits outweigh the effort of porting to a framework which is
>> already upstream and used on many platforms for different vendors!
>>
>> There is no free lunch. There are 'features' of RPMsg which aren't necessarily great for all platforms, e.g. the concepts of virtio-like rings for IPC with available/used buffers for both directions are not a good match or replacement for the memory-window-based IPC on Intel platforms, where there is no DDR access, a small window allocated by firmware and only a couple of doorbell registers for essentially serial communication.
> 
> rpmsg support to define the custom mechanism(see rpmsg_endpoint_ops in
> drivers\rpmsg\rpmsg_internal.h) but keep the upper layer API, qcomm
> utilize this for glink and smd actually.

Then this looks doable, thanks for sharing Xiang.

I have several questions:
1. Is virtio device and vring buffer access support in remote proc(Audio 
DSP in our case) side mandatory? Which means we have to porting 
libraries like libAMP into FW(not uses Zephyr or freeRTOS yet)?
2. About the resource table in FW binary, is this mandatory or it could 
be empty?

As you may know, we uses SOF-similar IPC framework(which is quite 
different with rpmsg/remoteproc) on Intel SoC platforms for long on tens 
of platforms of several generations(we have enabled and verified SOF on 
most of them), so the transition may take time.

Thanks,
~Keyon

> 
>> The resources embedded in a firmware file is another capability that doesn't align with the way the SOF firmware is generated. I also don't know where the topology file would be handled, nor how to deal with suspend-resume where the DSP needs to be restarted. For folks who need an introduction to RPMsg, the link [1] is the best I found to scope out the work required.
>>
> 
> We can share our rpmsg based topology implementation as reference which:
> 1.About 2500 lines(much less than SOF)
> 2.Support pcm and compress playback/capture
> 3.No any vendor dependence(thanks for rpmsg/remoteproc)
> 
>> In short, I don't mind looking at RPMsg as an option and would welcome contributions, but making it the default raises a number of technical challenges that can't be dismissed just yet, and such a transition isn't going to happen overnight. There are other evolutions that were mentioned as well, such as using the MFD framework to split the driver in 'core/hardware' support and application-specific parts (audio, sensors, etc), and likewise we need time to make it happen - just like we need time to move to the modern dailinks, add multi-cpu and SoundWire support, add digital domains, etc.
>>
> 
>> [1] http://processors.wiki.ti.com/index.php/PRU-ICSS_Remoteproc_and_RPMsg
>>
>>
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> https://mailman.alsa-project.org/mailman/listinfo/alsa-devel
> 

Re: [Sound-open-firmware] [alsa-devel] [v4, 00/14] ASoC: Sound Open Firmware (SOF) core

[Pierre-Louis Bossart]

On 2/22/19 2:32 AM, xiang xiao wrote:
> On Thu, Feb 21, 2019 at 11:27 PM Pierre-Louis Bossart
> <pierre-louis.bossart@linux.intel.com> wrote:
>>
>> Should we utilize official IPC frameowrk instead reinverting the wheel?
>> 1.Load firmware by drivers/remoteproc
>>      https://www.kernel.org/doc/Documentation/remoteproc.txt
>> 2.Do the comunication through drivers/rpmsg
>>      https://www.kernel.org/doc/Documentation/rpmsg.txt
>> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
>>
>> It definitely makes more sense to use rpmsg for Generic IPC driver here.
>>
>> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
>> definitely help everyone in future while immigrating to SOF.
>>
>> Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
>> fully integrate with the ASoC topology framework, and the firmware is
>> base on FreeRTOS and OpenMAX.
>> SOF initiative is very good and exciting, our team members(include me)
>> spend a couple weeks to study the current code base on both firmware
>> and kernel side, we even port SOF to our DSP/MCU and make it run, but
>> I have to point out that:
>> SOF IPC is too simple and rigid, tightly couple with Intel platform
>> and audio domain, which make:
>>      a.It's difficult to integrate with other vendor SoC, especially if
>> other vendor already adopt remote/rpmsg(this is already a trend!).
>>      b.It's difficult to add other IPC services for example:
>>         i.Audio DSP talk to power MCU to adjust clock and voltage
>>         ii.Export ultrasonic distance measurement to IIO subsystem
>>
>> The IPC scheme suggested in this patchset is only a first pass that works on
>> 3 generations on Intel platforms + the QEMU  parts. There are no claims that
>> the current solution is set-in-stone, and this is already an area where
>> things are already changing to support notifications and low-power
>> transitions.
>>
>> There will clearly be evolutions to make the IPC more flexible/generic, but
>> we've got to start somewhere and bear in mind that we also have to support
>> memory-constrained legacy devices where such generic frameworks aren't
>> needed or even implementable. Some of your proposals such as changing
>> power/clocks with a firmware request aren't necessarily possible or
>> recommended on all platforms - i can already hear security folks howling,
>> this was already mentioned in the GitHub thread.
>>
>> Rather than evolve the IPC, i would say it makes more sense that we
>> "reuse" existing upstream frameworks.. As given below by xiang
>> this seems to have support for RTOSes (see point 4 below) and looking at
>> below it seems to have much better coverage across systems.
>>
>> This should also help in easy adoption of SoF for non Intel people...
>>
>> Also looking at it, lot of IPC code, DSP loading etc would go away
>> making SoF code lesser in footprint.
>>
>> I think benefits outweigh the effort of porting to a framework which is
>> already upstream and used on many platforms for different vendors!
>>
>> There is no free lunch. There are 'features' of RPMsg which aren't necessarily great for all platforms, e.g. the concepts of virtio-like rings for IPC with available/used buffers for both directions are not a good match or replacement for the memory-window-based IPC on Intel platforms, where there is no DDR access, a small window allocated by firmware and only a couple of doorbell registers for essentially serial communication.
> rpmsg support to define the custom mechanism(see rpmsg_endpoint_ops in
> drivers\rpmsg\rpmsg_internal.h) but keep the upper layer API, qcomm
> utilize this for glink and smd actually.

That's interesting. Can anyone at Qualcomm/Linaro point to actual 
examples of the implementation, so that we get a better picture of the 
split between 'upper layer API' and 'custom mechanism'?

>
>> The resources embedded in a firmware file is another capability that doesn't align with the way the SOF firmware is generated. I also don't know where the topology file would be handled, nor how to deal with suspend-resume where the DSP needs to be restarted. For folks who need an introduction to RPMsg, the link [1] is the best I found to scope out the work required.
>>
> We can share our rpmsg based topology implementation as reference which:
> 1.About 2500 lines(much less than SOF)
> 2.Support pcm and compress playback/capture
> 3.No any vendor dependence(thanks for rpmsg/remoteproc)

Sure. Where's the code? What's the license?

Most of the SOF code is really in hardware-specific .ops callbacks and 
topology handling, the generic IPC layer is only ~800 lines of code. 
rpmsg would allow for easier portability but a significant reduction of 
the code size is unlikely.

Re: [alsa-devel] [v4,00/14] ASoC: Sound Open Firmware (SOF) core

[xiang xiao]

On Fri, Feb 22, 2019 at 7:13 PM Keyon Jie <yang.jie@linux.intel.com> wrote:
>
>
>
> On 2019/2/22 下午4:32, xiang xiao wrote:
> > On Thu, Feb 21, 2019 at 11:27 PM Pierre-Louis Bossart
> > <pierre-louis.bossart@linux.intel.com> wrote:
> >>
> >>
> >> Should we utilize official IPC frameowrk instead reinverting the wheel?
> >> 1.Load firmware by drivers/remoteproc
> >>      https://www.kernel.org/doc/Documentation/remoteproc.txt
> >> 2.Do the comunication through drivers/rpmsg
> >>      https://www.kernel.org/doc/Documentation/rpmsg.txt
> >> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
> >>
> >> It definitely makes more sense to use rpmsg for Generic IPC driver here.
> >>
> >> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
> >> definitely help everyone in future while immigrating to SOF.
> >>
> >> Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
> >> fully integrate with the ASoC topology framework, and the firmware is
> >> base on FreeRTOS and OpenMAX.
> >> SOF initiative is very good and exciting, our team members(include me)
> >> spend a couple weeks to study the current code base on both firmware
> >> and kernel side, we even port SOF to our DSP/MCU and make it run, but
> >> I have to point out that:
> >> SOF IPC is too simple and rigid, tightly couple with Intel platform
> >> and audio domain, which make:
> >>      a.It's difficult to integrate with other vendor SoC, especially if
> >> other vendor already adopt remote/rpmsg(this is already a trend!).
> >>      b.It's difficult to add other IPC services for example:
> >>         i.Audio DSP talk to power MCU to adjust clock and voltage
> >>         ii.Export ultrasonic distance measurement to IIO subsystem
> >>
> >> The IPC scheme suggested in this patchset is only a first pass that works on
> >> 3 generations on Intel platforms + the QEMU  parts. There are no claims that
> >> the current solution is set-in-stone, and this is already an area where
> >> things are already changing to support notifications and low-power
> >> transitions.
> >>
> >> There will clearly be evolutions to make the IPC more flexible/generic, but
> >> we've got to start somewhere and bear in mind that we also have to support
> >> memory-constrained legacy devices where such generic frameworks aren't
> >> needed or even implementable. Some of your proposals such as changing
> >> power/clocks with a firmware request aren't necessarily possible or
> >> recommended on all platforms - i can already hear security folks howling,
> >> this was already mentioned in the GitHub thread.
> >>
> >> Rather than evolve the IPC, i would say it makes more sense that we
> >> "reuse" existing upstream frameworks.. As given below by xiang
> >> this seems to have support for RTOSes (see point 4 below) and looking at
> >> below it seems to have much better coverage across systems.
> >>
> >> This should also help in easy adoption of SoF for non Intel people...
> >>
> >> Also looking at it, lot of IPC code, DSP loading etc would go away
> >> making SoF code lesser in footprint.
> >>
> >> I think benefits outweigh the effort of porting to a framework which is
> >> already upstream and used on many platforms for different vendors!
> >>
> >> There is no free lunch. There are 'features' of RPMsg which aren't necessarily great for all platforms, e.g. the concepts of virtio-like rings for IPC with available/used buffers for both directions are not a good match or replacement for the memory-window-based IPC on Intel platforms, where there is no DDR access, a small window allocated by firmware and only a couple of doorbell registers for essentially serial communication.
> >
> > rpmsg support to define the custom mechanism(see rpmsg_endpoint_ops in
> > drivers\rpmsg\rpmsg_internal.h) but keep the upper layer API, qcomm
> > utilize this for glink and smd actually.
>
> Then this looks doable, thanks for sharing Xiang.
>
> I have several questions:
> 1. Is virtio device and vring buffer access support in remote proc(Audio
> DSP in our case) side mandatory? Which means we have to porting
> libraries like libAMP into FW(not uses Zephyr or freeRTOS yet)?

If you go through viritio/rpmsg path, virtio/vring are required,
but if you implement the new rpmsg_endpoint_ops, both shouldn't be needed.
But both OpenAMP/RPMsg-Lite support the bare metal environment,
actually since both library define the clean and thin port layer, it's
very easy to port(I just spend 2-3 days for NuttX port).

> 2. About the resource table in FW binary, is this mandatory or it could
> be empty?

Again virtio/rpmsg path is required, but custom implementation mayn't needed.
Actually, remoteproc layer support the custom firmware format, so SOF
can migrate to remoteproc without changing firmware format.

>
> As you may know, we uses SOF-similar IPC framework(which is quite
> different with rpmsg/remoteproc) on Intel SoC platforms for long on tens
> of platforms of several generations(we have enabled and verified SOF on
> most of them), so the transition may take time.
>

Yes, I understand. But Intel still could provide the current version
to the customer before the new IPC version is stable if many people
think the generic IPC is the good direction to go.

> Thanks,
> ~Keyon
>
> >
> >> The resources embedded in a firmware file is another capability that doesn't align with the way the SOF firmware is generated. I also don't know where the topology file would be handled, nor how to deal with suspend-resume where the DSP needs to be restarted. For folks who need an introduction to RPMsg, the link [1] is the best I found to scope out the work required.
> >>
> >
> > We can share our rpmsg based topology implementation as reference which:
> > 1.About 2500 lines(much less than SOF)
> > 2.Support pcm and compress playback/capture
> > 3.No any vendor dependence(thanks for rpmsg/remoteproc)
> >
> >> In short, I don't mind looking at RPMsg as an option and would welcome contributions, but making it the default raises a number of technical challenges that can't be dismissed just yet, and such a transition isn't going to happen overnight. There are other evolutions that were mentioned as well, such as using the MFD framework to split the driver in 'core/hardware' support and application-specific parts (audio, sensors, etc), and likewise we need time to make it happen - just like we need time to move to the modern dailinks, add multi-cpu and SoundWire support, add digital domains, etc.
> >>
> >
> >> [1] http://processors.wiki.ti.com/index.php/PRU-ICSS_Remoteproc_and_RPMsg
> >>
> >>
> > _______________________________________________
> > Alsa-devel mailing list
> > Alsa-devel@alsa-project.org
> > https://mailman.alsa-project.org/mailman/listinfo/alsa-devel
> >

Re: [Sound-open-firmware] [alsa-devel] [v4, 00/14] ASoC: Sound Open Firmware (SOF) core

[xiang xiao]

On Fri, Feb 22, 2019 at 10:48 PM Pierre-Louis Bossart
<pierre-louis.bossart@linux.intel.com> wrote:
>
>
> On 2/22/19 2:32 AM, xiang xiao wrote:
> > On Thu, Feb 21, 2019 at 11:27 PM Pierre-Louis Bossart
> > <pierre-louis.bossart@linux.intel.com> wrote:
> >>
> >> Should we utilize official IPC frameowrk instead reinverting the wheel?
> >> 1.Load firmware by drivers/remoteproc
> >>      https://www.kernel.org/doc/Documentation/remoteproc.txt
> >> 2.Do the comunication through drivers/rpmsg
> >>      https://www.kernel.org/doc/Documentation/rpmsg.txt
> >> Many vendor(TI, Qualcomm, ST, NXP, Xilinx...) migrate to remoteproc/rpmsg, why Intel provide an other IPC mechanism?
> >>
> >> It definitely makes more sense to use rpmsg for Generic IPC driver here.
> >>
> >> Qualcomm DSP audio drivers (non SOF) already use rpmsg. This will
> >> definitely help everyone in future while immigrating to SOF.
> >>
> >> Actually, Xiaomi also build DSP audio driver on top of rpmsg, but
> >> fully integrate with the ASoC topology framework, and the firmware is
> >> base on FreeRTOS and OpenMAX.
> >> SOF initiative is very good and exciting, our team members(include me)
> >> spend a couple weeks to study the current code base on both firmware
> >> and kernel side, we even port SOF to our DSP/MCU and make it run, but
> >> I have to point out that:
> >> SOF IPC is too simple and rigid, tightly couple with Intel platform
> >> and audio domain, which make:
> >>      a.It's difficult to integrate with other vendor SoC, especially if
> >> other vendor already adopt remote/rpmsg(this is already a trend!).
> >>      b.It's difficult to add other IPC services for example:
> >>         i.Audio DSP talk to power MCU to adjust clock and voltage
> >>         ii.Export ultrasonic distance measurement to IIO subsystem
> >>
> >> The IPC scheme suggested in this patchset is only a first pass that works on
> >> 3 generations on Intel platforms + the QEMU  parts. There are no claims that
> >> the current solution is set-in-stone, and this is already an area where
> >> things are already changing to support notifications and low-power
> >> transitions.
> >>
> >> There will clearly be evolutions to make the IPC more flexible/generic, but
> >> we've got to start somewhere and bear in mind that we also have to support
> >> memory-constrained legacy devices where such generic frameworks aren't
> >> needed or even implementable. Some of your proposals such as changing
> >> power/clocks with a firmware request aren't necessarily possible or
> >> recommended on all platforms - i can already hear security folks howling,
> >> this was already mentioned in the GitHub thread.
> >>
> >> Rather than evolve the IPC, i would say it makes more sense that we
> >> "reuse" existing upstream frameworks.. As given below by xiang
> >> this seems to have support for RTOSes (see point 4 below) and looking at
> >> below it seems to have much better coverage across systems.
> >>
> >> This should also help in easy adoption of SoF for non Intel people...
> >>
> >> Also looking at it, lot of IPC code, DSP loading etc would go away
> >> making SoF code lesser in footprint.
> >>
> >> I think benefits outweigh the effort of porting to a framework which is
> >> already upstream and used on many platforms for different vendors!
> >>
> >> There is no free lunch. There are 'features' of RPMsg which aren't necessarily great for all platforms, e.g. the concepts of virtio-like rings for IPC with available/used buffers for both directions are not a good match or replacement for the memory-window-based IPC on Intel platforms, where there is no DDR access, a small window allocated by firmware and only a couple of doorbell registers for essentially serial communication.
> > rpmsg support to define the custom mechanism(see rpmsg_endpoint_ops in
> > drivers\rpmsg\rpmsg_internal.h) but keep the upper layer API, qcomm
> > utilize this for glink and smd actually.
>
> That's interesting. Can anyone at Qualcomm/Linaro point to actual
> examples of the implementation, so that we get a better picture of the
> split between 'upper layer API' and 'custom mechanism'?
>
> >
> >> The resources embedded in a firmware file is another capability that doesn't align with the way the SOF firmware is generated. I also don't know where the topology file would be handled, nor how to deal with suspend-resume where the DSP needs to be restarted. For folks who need an introduction to RPMsg, the link [1] is the best I found to scope out the work required.
> >>
> > We can share our rpmsg based topology implementation as reference which:
> > 1.About 2500 lines(much less than SOF)
> > 2.Support pcm and compress playback/capture
> > 3.No any vendor dependence(thanks for rpmsg/remoteproc)
>
> Sure. Where's the code? What's the license?
>

The code is base on 4.19 kernel, I could upstream the code basing on
the latest kernel in the next couple days for reference.
the license is GPL, of course.

> Most of the SOF code is really in hardware-specific .ops callbacks and
> topology handling, the generic IPC layer is only ~800 lines of code.
> rpmsg would allow for easier portability but a significant reduction of
> the code size is unlikely.
>

The reduce come from:
1.Move firmware load and dsp start/stop to remoteproc layer.
2.Move IPC buffer/mailbox to rpmsg layer.
3.Reuse ASoC topology parser to generate the audio graph.
4.Reuse ASoC DAMP to control the graph node state change(run/stop/pause/resume).
5.Use the general machine driver glue all individual components.

>

Re: [alsa-devel] [Sound-open-firmware] [v4, 00/14] ASoC: Sound Open Firmware (SOF) core

[Pierre-Louis Bossart]

>>> We can share our rpmsg based topology implementation as reference which:
>>> 1.About 2500 lines(much less than SOF)
>>> 2.Support pcm and compress playback/capture
>>> 3.No any vendor dependence(thanks for rpmsg/remoteproc)
>> Sure. Where's the code? What's the license?
>>
> The code is base on 4.19 kernel, I could upstream the code basing on
> the latest kernel in the next couple days for reference.
> the license is GPL, of course.
I'll be looking forward to the code. My turn to provide comments :-)
>
>> Most of the SOF code is really in hardware-specific .ops callbacks and
>> topology handling, the generic IPC layer is only ~800 lines of code.
>> rpmsg would allow for easier portability but a significant reduction of
>> the code size is unlikely.
>>
> The reduce come from:
> 1.Move firmware load and dsp start/stop to remoteproc layer.
> 2.Move IPC buffer/mailbox to rpmsg layer.

You are not going to see a lot of code reduction here, at the end of the 
day most of the code comes from hardware-specific register access...

> 3.Reuse ASoC topology parser to generate the audio graph.
> 4.Reuse ASoC DAMP to control the graph node state change(run/stop/pause/resume).
> 5.Use the general machine driver glue all individual components
the last 3 points are already how SOF works, wondering if there is a 
misunderstanding... We are using topology/DAPM directly without any 
reinvention or duplication, as measured by the multiple fixes we 
provided to the frameworks in the last few weeks. Also when 
dedicated/custom topology tokens are needed, you still need code to deal 
with them and send the relevant configuration to firmware, whatever the 
transport format might be.

Re: [alsa-devel] [Sound-open-firmware] [v4, 00/14] ASoC: Sound Open Firmware (SOF) core

[xiang xiao]

On Sat, Feb 23, 2019 at 5:52 AM Pierre-Louis Bossart
<pierre-louis.bossart@linux.intel.com> wrote:
>
>
> >>> We can share our rpmsg based topology implementation as reference which:
> >>> 1.About 2500 lines(much less than SOF)
> >>> 2.Support pcm and compress playback/capture
> >>> 3.No any vendor dependence(thanks for rpmsg/remoteproc)
> >> Sure. Where's the code? What's the license?
> >>
> > The code is base on 4.19 kernel, I could upstream the code basing on
> > the latest kernel in the next couple days for reference.
> > the license is GPL, of course.
> I'll be looking forward to the code. My turn to provide comments :-)
> >
> >> Most of the SOF code is really in hardware-specific .ops callbacks and
> >> topology handling, the generic IPC layer is only ~800 lines of code.
> >> rpmsg would allow for easier portability but a significant reduction of
> >> the code size is unlikely.
> >>
> > The reduce come from:
> > 1.Move firmware load and dsp start/stop to remoteproc layer.
> > 2.Move IPC buffer/mailbox to rpmsg layer.
>
> You are not going to see a lot of code reduction here, at the end of the
> day most of the code comes from hardware-specific register access...
>
> > 3.Reuse ASoC topology parser to generate the audio graph.
> > 4.Reuse ASoC DAMP to control the graph node state change(run/stop/pause/resume).
> > 5.Use the general machine driver glue all individual components
> the last 3 points are already how SOF works, wondering if there is a
> misunderstanding... We are using topology/DAPM directly without any
> reinvention or duplication, as measured by the multiple fixes we
> provided to the frameworks in the last few weeks. Also when
> dedicated/custom topology tokens are needed, you still need code to deal
> with them and send the relevant configuration to firmware, whatever the
> transport format might be.

Here is the patch(~2800 lines) just for reference only:
https://github.com/xiaoxiang781216/linux/commit/077aad71ef153212ddb25cba71082d3ecd684f02

I attach a presentation in the commit to help understand the whole picture:
1.There is a tplg rpmsg driver for each machine:
   a.DSP create a rpmsg channel named "rpmsg-audio"
   b.rpmsg subsystem create a new rpmsg device and bind to tplg rpmsg driver
   c.tplg rpmsg probe callback load the topology file from user space
   d.Forward widget create and route to DSP for graph generation
   e.Forward kcontrol get/set to DSP for widget parameter operation
   Note: widget name is used to locate the widget in DSP.
2.For each FE/BE/COMPR, a new rpmsg based platform driver is created:
   a.Each platform driver create a new rpmsg channel to talk with the
remote peer
   b.Forward snd_pcm_ops/snd_compr_ops to the remote peer
   Note: widget name is used here as the channel name
Basically, 1(TPLG)+n(FE/BE/COMPR) rpmsg channels exist.

Re: [Sound-open-firmware] [alsa-devel] [v4, 00/14] ASoC: Sound Open Firmware (SOF) core

[Keyon Jie]

On 2019/2/23 上午2:21, xiang xiao wrote:
> On Fri, Feb 22, 2019 at 7:13 PM Keyon Jie <yang.jie@linux.intel.com> wrote:
>> Then this looks doable, thanks for sharing Xiang.
>>
>> I have several questions:
>> 1. Is virtio device and vring buffer access support in remote proc(Audio
>> DSP in our case) side mandatory? Which means we have to porting
>> libraries like libAMP into FW(not uses Zephyr or freeRTOS yet)?
> 
> If you go through viritio/rpmsg path, virtio/vring are required,
> but if you implement the new rpmsg_endpoint_ops, both shouldn't be needed.
> But both OpenAMP/RPMsg-Lite support the bare metal environment,
> actually since both library define the clean and thin port layer, it's
> very easy to port(I just spend 2-3 days for NuttX port).

Thanks for sharing, that's good if it is possible to go without virtio 
support at the first step, we will do investigation based on code you 
shared.

> 
>> 2. About the resource table in FW binary, is this mandatory or it could
>> be empty?
> 
> Again virtio/rpmsg path is required, but custom implementation mayn't needed.
> Actually, remoteproc layer support the custom firmware format, so SOF
> can migrate to remoteproc without changing firmware format.

Good to know this, thanks.

Thanks,
~Keyon

> 
>>
>> As you may know, we uses SOF-similar IPC framework(which is quite
>> different with rpmsg/remoteproc) on Intel SoC platforms for long on tens
>> of platforms of several generations(we have enabled and verified SOF on
>> most of them), so the transition may take time.
>>
> 
> Yes, I understand. But Intel still could provide the current version
> to the customer before the new IPC version is stable if many people
> think the generic IPC is the good direction to go.
> 
>> Thanks,
>> ~Keyon
>>
>>>
>>>> The resources embedded in a firmware file is another capability that doesn't align with the way the SOF firmware is generated. I also don't know where the topology file would be handled, nor how to deal with suspend-resume where the DSP needs to be restarted. For folks who need an introduction to RPMsg, the link [1] is the best I found to scope out the work required.
>>>>

Re: [Sound-open-firmware] [alsa-devel] [v4, 00/14] ASoC: Sound Open Firmware (SOF) core

[Srinivas Kandagatla]

On 22/02/2019 14:48, Pierre-Louis Bossart wrote:
>> drivers\rpmsg\rpmsg_internal.h) but keep the upper layer API, qcomm
>> utilize this for glink and smd actually.
> 
> That's interesting. Can anyone at Qualcomm/Linaro point to actual 
> examples of the implementation, so that we get a better picture of the 
> split between 'upper layer API' and 'custom mechanism'?
> 
Here is an example of Qualcomm Audio Hexagon DSP :

Remoteproc driver to start dsp is at:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/remoteproc/qcom_common.c?h=v5.0-rc8

Which registers rpmsg channels based of glink or SMD transport once dsp 
is booted at:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/rpmsg/qcom_glink_smem.c?h=v5.0-rc8 
and few other files in drivers/rpmsg/

This rpmsg driver driver will create and destroy 
endpoints/edges/channels/services based on dsp services and dsp 
notifications. These notifications are passed to service drivers via 
rpmsg device driver model.

One example of audio service driver is:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soc/qcom/apr.c?h=v5.0-rc8

and the audio drivers are at:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/sound/soc/qcom/qdsp6?h=v5.0-rc8

An example Device Tree of audio and DSP compute offload would look like 
this:

adsp-pil {
	compatible = "qcom,msm8996-adsp-pil";
	smd-edge {
		label = "lpass";
                 mboxes = <&apcs_glb 8>;
                 qcom,smd-edge = <1>;
                 qcom,remote-pid = <2>;

		apr {
                 	compatible = "qcom,apr-v2";
                         qcom,smd-channels = "apr_audio_svc";
			...
		};

                 fastrpc {
			qcom,smd-channels = "fastrpcsmd-apps-dsp";
			compatible = "qcom,fastrpc";
			...
		}
		...
	};
};

thanks,
srini