[PATCH 0/3] ALSA USB MIDI: Predictable low latency for USB MIDI output

[PATCH 0/3] ALSA USB MIDI: Predictable low latency for USB MIDI output

[Andreas Steinmetz]

The following patch series fixes USB MIDI output starvation and allows
to configure a predictable worst case output latency with a minimum of 15ms
for 16x16 devices and 3.8ms for 4x4 devices (excluding USB transfer time
and latencies of the interface device itself).

The reasoning behind this patch series is first to fix the possible
indefinite latency and then to allow to configure a predictable
low latency to be able to use a cheap embedded Linux system (e.g.
Raspberry Pi) in combination with a multiport USB MIDI interface
as a drop in replacement for commercial hardware based MIDI routing
and filtering products. This includes "real time" (live) usage.

For this usage the guaranteed latency within the kernel must be as
low as possible, depending on the requirements of the USB MIDI
interface in use.

Though the current state of the kernel code is acceptable in a
fully controlled by a single user environment, it is not usable
for uncontrolled standalone use as the driver behaviour then
depends on unpredictable MIDI data streams and thus is unpredictable
in behaviour itself.

After applying this patch series the USB MIDI driver can be used
as usual or it can be tuned to low latency when this is desired.

From a developer perspective the actual output rate of USB MIDI
interfaces is typically slower than the expected 320us and even
varies depending on how many ports are busy (e.g. a M-Audio
Midisport 4x4 AE interface exhibits a varying output rate of
357us to 515us per byte depending on device load). Thus the driver
needs a predictable low latency to allow userspace to use the
rawmidi buffer fill state to estimate the expected output latency
and schedule data accordingly.

The following table shows relative processing time for varying driver
configurations using the M-Audio Midisport 4x4 AE as the test device.
For the parameters "outqueue" and "events" do see the descriptions
of the following patches.

	| unpatched	| outqueue=7	| outqueue=1	| outqueue=1	|
	| kernel	| events=0	| events=0	| events=4	|
--------+---------------+---------------+---------------+---------------+
1 Port	|      100%	|      100%	|      100%	|    100.05%	|
--------+---------------+---------------+---------------+---------------+
4 Ports	|      n/a	|      100%	|     99.97%	|    100.36%	|
--------+---------------+---------------+---------------+---------------+

As can be seen from the above table the changes to output latency do
affect throughput only marginally.

Re: [PATCH 0/3] ALSA USB MIDI: Predictable low latency for USB MIDI output

[Takashi Iwai]

On Sat, 14 Mar 2020 09:11:17 +0100,
Andreas Steinmetz wrote:
> 
> The following patch series fixes USB MIDI output starvation and allows
> to configure a predictable worst case output latency with a minimum of 15ms
> for 16x16 devices and 3.8ms for 4x4 devices (excluding USB transfer time
> and latencies of the interface device itself).
> 
> The reasoning behind this patch series is first to fix the possible
> indefinite latency and then to allow to configure a predictable
> low latency to be able to use a cheap embedded Linux system (e.g.
> Raspberry Pi) in combination with a multiport USB MIDI interface
> as a drop in replacement for commercial hardware based MIDI routing
> and filtering products. This includes "real time" (live) usage.
> 
> For this usage the guaranteed latency within the kernel must be as
> low as possible, depending on the requirements of the USB MIDI
> interface in use.
> 
> Though the current state of the kernel code is acceptable in a
> fully controlled by a single user environment, it is not usable
> for uncontrolled standalone use as the driver behaviour then
> depends on unpredictable MIDI data streams and thus is unpredictable
> in behaviour itself.
> 
> After applying this patch series the USB MIDI driver can be used
> as usual or it can be tuned to low latency when this is desired.
> 
> >From a developer perspective the actual output rate of USB MIDI
> interfaces is typically slower than the expected 320us and even
> varies depending on how many ports are busy (e.g. a M-Audio
> Midisport 4x4 AE interface exhibits a varying output rate of
> 357us to 515us per byte depending on device load). Thus the driver
> needs a predictable low latency to allow userspace to use the
> rawmidi buffer fill state to estimate the expected output latency
> and schedule data accordingly.
> 
> The following table shows relative processing time for varying driver
> configurations using the M-Audio Midisport 4x4 AE as the test device.
> For the parameters "outqueue" and "events" do see the descriptions
> of the following patches.
> 
> 	| unpatched	| outqueue=7	| outqueue=1	| outqueue=1	|
> 	| kernel	| events=0	| events=0	| events=4	|
> --------+---------------+---------------+---------------+---------------+
> 1 Port	|      100%	|      100%	|      100%	|    100.05%	|
> --------+---------------+---------------+---------------+---------------+
> 4 Ports	|      n/a	|      100%	|     99.97%	|    100.36%	|
> --------+---------------+---------------+---------------+---------------+
> 
> As can be seen from the above table the changes to output latency do
> affect throughput only marginally.

Thanks for the patches and the analysis.

So, judging from the experiments above, outqueue=1 has no measurable
performance regression?  For 4 ports, it's even an improvement.
Then we may change the default value.

I guess you kept the old value just to be conservative, but if the
number tells a better story and it's no part or ABI, we may consider
changing the default for a better performance.

Or, is there any other potential risk by changing this?


thanks,

Takashi

Re: [PATCH 0/3] ALSA USB MIDI: Predictable low latency for USB MIDI output

[Andreas Steinmetz]

On Mon, 2020-03-16 at 09:42 +0100, Takashi Iwai wrote:
> Thanks for the patches and the analysis.
> 
> So, judging from the experiments above, outqueue=1 has no measurable
> performance regression?  For 4 ports, it's even an improvement.
> Then we may change the default value.
> 
> I guess you kept the old value just to be conservative, but if the
> number tells a better story and it's no part or ABI, we may consider
> changing the default for a better performance.
> 
> Or, is there any other potential risk by changing this?
> 
I'm very careful here. MIDI based equipment (and that includes MIDI
interfaces) is typically very long lived (I do have equipment that is
more than 20 years old). Thus I cannot guarantee that changed defaults
will not harm performance of older equipment.

I do prefer to keep the current defaults and allow users to, well,
experiment if modifications to the defaults do work and actually help
for their hardware. I would not want to break usability of older
hardware.

Regards
-- 
Andreas Steinmetz

Re: [PATCH 0/3] ALSA USB MIDI: Predictable low latency for USB MIDI output

[Takashi Iwai]

On Mon, 16 Mar 2020 11:45:41 +0100,
Andreas Steinmetz wrote:
> 
> On Mon, 2020-03-16 at 09:42 +0100, Takashi Iwai wrote:
> > Thanks for the patches and the analysis.
> > 
> > So, judging from the experiments above, outqueue=1 has no measurable
> > performance regression?  For 4 ports, it's even an improvement.
> > Then we may change the default value.
> > 
> > I guess you kept the old value just to be conservative, but if the
> > number tells a better story and it's no part or ABI, we may consider
> > changing the default for a better performance.
> > 
> > Or, is there any other potential risk by changing this?
> > 
> I'm very careful here. MIDI based equipment (and that includes MIDI
> interfaces) is typically very long lived (I do have equipment that is
> more than 20 years old). Thus I cannot guarantee that changed defaults
> will not harm performance of older equipment.
> 
> I do prefer to keep the current defaults and allow users to, well,
> experiment if modifications to the defaults do work and actually help
> for their hardware. I would not want to break usability of older
> hardware.

OK.  Unless Clemens has any objection, I'm going to take your patches
as-is, then.


thanks,

Takashi