[B.A.T.M.A.N.] is the multicast performance throttled?

[B.A.T.M.A.N.] is the multicast performance throttled?

[Ignacio Quezada]

Hello,
I am trying to build an application using UDP multicast sockets based
on a network mesh built with devices connected in IBSS mode. 

The kernel documentation says that this was the place to ask questions
or anything :).

Testing the application in the mentioned environment, it suffers some
kind of throttling or buffering limiting the throughput. Testing with
my application, without the batman interface, just 2 IBSS connected
devices, the multicast rate goes around 100kb/s (not saying that the
wifi link actually tops at that speed), but then just to add the
batman-adv interface and use that instead, the multicast rate becomes
unstable between 0-20kb/s.

Disabling multicast optimization, it changes a bit the behaviour but
still the same speed.

If I change the multicast packet to a unicast packet, both setups
achieve the same speed (~100kb/s), so the batman interface is working
but the multicast packets are being handled in a different way. Is it
supposed to be like that?

The packets are sent right away according to the iptables counter of
packets as well as tcpdump, but the receiver only gets them in a very
slow way (counting but not calculating, I think it is around 8 packets
per second).
The MTU is set as advised, 1532 and the payload for the UDP packets is
as big as it can be so the packet does not get fragmented.

Any way to increase the rate or disabling the throttling?
-- 

MVH
Ignacio Quezada

Re: [B.A.T.M.A.N.] is the multicast performance throttled?

[Linus Lüssing]

Hi Ignacio,

Could you describe your setup a little more, just two devices?
What hardware are you using, which wifi cards? Which operating
system and version?

Which version of batman-adv are you using? What is your kernel
version?

What is a typical bitrate your wifi cards are agreeing on (iw
station dump)? What multicast rate have you configured for your
wifi cards?

What software are you using for testing? What is the bitrate you
have configured in there? Is the unicast test UDP as well? What
bitrates are you able to achieve via TCP? Does a lower payload
length make any difference?

Regards, Linus


On Thu, Aug 11, 2016 at 05:13:01PM +0200, Ignacio Quezada wrote:
> Hello,
> I am trying to build an application using UDP multicast sockets based
> on a network mesh built with devices connected in IBSS mode. 
> 
> The kernel documentation says that this was the place to ask questions
> or anything :).
> 
> Testing the application in the mentioned environment, it suffers some
> kind of throttling or buffering limiting the throughput. Testing with
> my application, without the batman interface, just 2 IBSS connected
> devices, the multicast rate goes around 100kb/s (not saying that the
> wifi link actually tops at that speed), but then just to add the
> batman-adv interface and use that instead, the multicast rate becomes
> unstable between 0-20kb/s.
> 
> Disabling multicast optimization, it changes a bit the behaviour but
> still the same speed.
> 
> If I change the multicast packet to a unicast packet, both setups
> achieve the same speed (~100kb/s), so the batman interface is working
> but the multicast packets are being handled in a different way. Is it
> supposed to be like that?
> 
> The packets are sent right away according to the iptables counter of
> packets as well as tcpdump, but the receiver only gets them in a very
> slow way (counting but not calculating, I think it is around 8 packets
> per second).
> The MTU is set as advised, 1532 and the payload for the UDP packets is
> as big as it can be so the packet does not get fragmented.
> 
> Any way to increase the rate or disabling the throttling?
> -- 
> 
> MVH
> Ignacio Quezada

Re: [B.A.T.M.A.N.] is the multicast performance throttled?

[Ignacio Quezada]

Hi Linus,

On tor, 2016-08-11 at 17:47 +0200, Linus Lüssing wrote:
> Hi Ignacio,
> 
> Could you describe your setup a little more, just two devices?
> What hardware are you using, which wifi cards? Which operating
> system and version?
> 
> Which version of batman-adv are you using? What is your kernel
> version?

The test environment is 2 android phones with Android 5.1 running
kernel 3.4 where I backported batman 2016.2, same wifi chip (same soc)
BCM4339.

> What is a typical bitrate your wifi cards are agreeing on (iw
> station dump)? What multicast rate have you configured for your
> wifi cards?

Sadly, the support of iw for these kind of chipsets is very limited and
iw dev wlan0 station dump returns nothing. I don't know how to get you
that information.

> What software are you using for testing? What is the bitrate you
> have configured in there? Is the unicast test UDP as well? What
> bitrates are you able to achieve via TCP? Does a lower payload
> length make any difference?
> 

The software used is a java program with 1 socket each where one sends
and the other receives. The unicast is UDP as well, yes. I haven't
tried with TCP but I can do it if it is necessary, my application only
relies on UDP.

I just tried a lower payload (504 bytes) and the amount of packets per
second becomes considerably higher but the amount of data is still
almost the same.

> Regards, Linus
> 
> 
> On Thu, Aug 11, 2016 at 05:13:01PM +0200, Ignacio Quezada wrote:
> > 
> > Hello,
> > I am trying to build an application using UDP multicast sockets
> > based
> > on a network mesh built with devices connected in IBSS mode. 
> > 
> > The kernel documentation says that this was the place to ask
> > questions
> > or anything :).
> > 
> > Testing the application in the mentioned environment, it suffers
> > some
> > kind of throttling or buffering limiting the throughput. Testing
> > with
> > my application, without the batman interface, just 2 IBSS connected
> > devices, the multicast rate goes around 100kb/s (not saying that
> > the
> > wifi link actually tops at that speed), but then just to add the
> > batman-adv interface and use that instead, the multicast rate
> > becomes
> > unstable between 0-20kb/s.
> > 
> > Disabling multicast optimization, it changes a bit the behaviour
> > but
> > still the same speed.
> > 
> > If I change the multicast packet to a unicast packet, both setups
> > achieve the same speed (~100kb/s), so the batman interface is
> > working
> > but the multicast packets are being handled in a different way. Is
> > it
> > supposed to be like that?
> > 
> > The packets are sent right away according to the iptables counter
> > of
> > packets as well as tcpdump, but the receiver only gets them in a
> > very
> > slow way (counting but not calculating, I think it is around 8
> > packets
> > per second).
> > The MTU is set as advised, 1532 and the payload for the UDP packets
> > is
> > as big as it can be so the packet does not get fragmented.
> > 
> > Any way to increase the rate or disabling the throttling?
> > -- 
> > 
> > MVH
> > Ignacio Quezada

--

MVH
Ignacio Quezada

Re: [B.A.T.M.A.N.] is the multicast performance throttled?

[Linus Lüssing]

On Thu, Aug 11, 2016 at 06:09:34PM +0200, Ignacio Quezada wrote:
> Hi Linus,
> 
> On tor, 2016-08-11 at 17:47 +0200, Linus Lüssing wrote:
> > Hi Ignacio,
> > 
> > Could you describe your setup a little more, just two devices?
> > What hardware are you using, which wifi cards? Which operating
> > system and version?
> > 
> > Which version of batman-adv are you using? What is your kernel
> > version?
> 
> The test environment is 2 android phones with Android 5.1 running
> kernel 3.4 where I backported batman 2016.2, same wifi chip (same soc)
> BCM4339.

Note sure how usable IBSS on Android is. Last time I heard someone
talking about it, it was supposedly unusable. But maybe someone
else has some more current information :-).

> 
> > What is a typical bitrate your wifi cards are agreeing on (iw
> > station dump)? What multicast rate have you configured for your
> > wifi cards?
> 
> Sadly, the support of iw for these kind of chipsets is very limited and
> iw dev wlan0 station dump returns nothing. I don't know how to get you
> that information.

And the multicast rate is still left at the 1MBit/s default?
Unfortunately, usually multicast packets are sent at the lowest
rate available as there are no wifi ACKs. To maximize the probability of
reception. There are some tweaks in OpenWRT though.

Also, often people change the bitrate from 1MBit/s to more like
12MBit/s or even above. That gives more free airtime and enhances the
batman-adv estimations (BATMAN IV).

> 
> > What software are you using for testing? What is the bitrate you
> > have configured in there? Is the unicast test UDP as well? What
> > bitrates are you able to achieve via TCP? Does a lower payload
> > length make any difference?
> > 
> 
> The software used is a java program with 1 socket each where one sends
> and the other receives. The unicast is UDP as well, yes. I haven't
> tried with TCP but I can do it if it is necessary, my application only
> relies on UDP.

UDP has no flow control. That's why I was asking about the bitrate
you have configured in the application. Sending with "as fast as
possible" will basically jam your (and your neighbors) wifi. And
will create trouble for batman-adv to find usable routes too
without any airtime available.

Regards, Linus

Re: [B.A.T.M.A.N.] is the multicast performance throttled?

[Ignacio Quezada]

-- 

MVH
Ignacio Quezada

On tor, 2016-08-11 at 18:31 +0200, Linus Lüssing wrote:
> On Thu, Aug 11, 2016 at 06:09:34PM +0200, Ignacio Quezada wrote:
> > 
> > Hi Linus,
> > 
> > On tor, 2016-08-11 at 17:47 +0200, Linus Lüssing wrote:
> > > 
> > > Hi Ignacio,
> > > 
> > > Could you describe your setup a little more, just two devices?
> > > What hardware are you using, which wifi cards? Which operating
> > > system and version?
> > > 
> > > Which version of batman-adv are you using? What is your kernel
> > > version?
> > 
> > The test environment is 2 android phones with Android 5.1 running
> > kernel 3.4 where I backported batman 2016.2, same wifi chip (same
> > soc)
> > BCM4339.
> 
> Note sure how usable IBSS on Android is. Last time I heard someone
> talking about it, it was supposedly unusable. But maybe someone
> else has some more current information :-).
> 
After some tweaking, it works good enough.
> > 
> > 
> > > 
> > > What is a typical bitrate your wifi cards are agreeing on (iw
> > > station dump)? What multicast rate have you configured for your
> > > wifi cards?
> > 
> > Sadly, the support of iw for these kind of chipsets is very limited
> > and
> > iw dev wlan0 station dump returns nothing. I don't know how to get
> > you
> > that information.
> 
> And the multicast rate is still left at the 1MBit/s default?
> Unfortunately, usually multicast packets are sent at the lowest
> rate available as there are no wifi ACKs. To maximize the probability
> of
> reception. There are some tweaks in OpenWRT though.
> 
> Also, often people change the bitrate from 1MBit/s to more like
> 12MBit/s or even above. That gives more free airtime and enhances the
> batman-adv estimations (BATMAN IV).
> 
Do you mean that with 1MBit/s for multicast, batman protocol needs
80kb/s to set up the connections? 1MBit/s for my application is more
than enough. 
> > 
> > 
> > > 
> > > What software are you using for testing? What is the bitrate you
> > > have configured in there? Is the unicast test UDP as well? What
> > > bitrates are you able to achieve via TCP? Does a lower payload
> > > length make any difference?
> > > 
> > 
> > The software used is a java program with 1 socket each where one
> > sends
> > and the other receives. The unicast is UDP as well, yes. I haven't
> > tried with TCP but I can do it if it is necessary, my application
> > only
> > relies on UDP.
> 
> UDP has no flow control. That's why I was asking about the bitrate
> you have configured in the application. Sending with "as fast as
> possible" will basically jam your (and your neighbors) wifi. And
> will create trouble for batman-adv to find usable routes too
> without any airtime available.
> 
> Regards, Linus

What I don't understand is that you are explaining standard behavior of
WiFi networks, but using plain WiFi without batman works ok and the
multicast goes up to 100kb/s (I think I wrote that in my first email). 
The tests have been like this so far:
- wlan0 (no batman),  multicast: 100kb/s
- wlan0 (no batman),  unicast: 100kb/s
- bat0,  unicast: 100kb/s
- bat0,  multicast: 20kb/s


PS: Emails from the Gmail Android App get rejected by the mailinglist,
I guess it is because of the HTML format? I wasn't able to disable it,
back at the desktop.

MVH 
Ignacio Quezada

Re: [B.A.T.M.A.N.] is the multicast performance throttled?

[Linus Lüssing]

On Fri, Aug 12, 2016 at 09:50:56AM +0200, Ignacio Quezada wrote:
> > UDP has no flow control. That's why I was asking about the bitrate
> > you have configured in the application. Sending with "as fast as
> > possible" will basically jam your (and your neighbors) wifi. And
> > will create trouble for batman-adv to find usable routes too
> > without any airtime available.
> > 
> > Regards, Linus
> 
> What I don't understand is that you are explaining standard behavior of
> WiFi networks, but using plain WiFi without batman works ok and the
> multicast goes up to 100kb/s (I think I wrote that in my first email). 
> The tests have been like this so far:
> - wlan0 (no batman),  multicast: 100kb/s
> - wlan0 (no batman),  unicast: 100kb/s
> - bat0,  unicast: 100kb/s
> - bat0,  multicast: 20kb/s

Yes, read your first email and that's the oddity I'm trying to
figure out :-).

Let me try to rephrase. One guess I'm having (which might be
wrong), that the multicast traffic on bat0 is interfering with the
control packets ("Originator Messages" in batman speak). And that
this breaks even one-hop routes. Hm, ok, on the other hand, even
with no routes, flooded multicast would "work" because it doesn't
use the routes discovered by OGMs. Next guess. (although the
output of the originator table, "batctl o", before and during your
application transmitting would be interesting to check whether
there is an effect/interference)


Second guess (which is probably it :-) ): In general, batman-adv
(re)broadcasts a multicast packet three times on an interface. To
ensure proper reception even over multiple hops.

That might even add up with your numbers: Initial sender
broadcasts three times, neighbor rebroadcasts three times, so
6*16kb/s ~ 100kb/s. 100kb/s => 0.800Mbit/s, which is already
oversaturating the gross 1MBit/s wifi multicast rate (@1MBit/s gross
it's usually more like 0.5MBit/s net at least for unicast,
multicast might be something between 1MBit/s and 0.5MBit/s net).

> 
> 
> PS: Emails from the Gmail Android App get rejected by the mailinglist,
> I guess it is because of the HTML format? I wasn't able to disable it,
> back at the desktop.

Yes, this mailinglist rejects HTML mail. To cope with spam,
I think.

Regards, Linus

Re: [B.A.T.M.A.N.] is the multicast performance throttled?

[Linus Lüssing]

On Sun, Aug 14, 2016 at 07:21:28PM +0200, Linus Lüssing wrote:
> Second guess (which is probably it :-) ): In general, batman-adv
> (re)broadcasts a multicast packet three times on an interface. To
> ensure proper reception even over multiple hops.

PS: As you are compiling batman-adv yourself, you might want to
try different values for BATADV_NUM_BCASTS_WIRELESS in main.h to
(in)validate this hypothesis.

Re: [B.A.T.M.A.N.] is the multicast performance throttled?

[Ignacio Quezada]

Hi,


> > Second guess (which is probably it :-) ): In general, batman-adv
> > (re)broadcasts a multicast packet three times on an interface. To
> > ensure proper reception even over multiple hops.
> 
> PS: As you are compiling batman-adv yourself, you might want to
> try different values for BATADV_NUM_BCASTS_WIRELESS in main.h to
> (in)validate this hypothesis.

This was indeed "the problem". Setting that value to 1 allowed much
more throughput. Is the value 3 just an arbitrary number to "ensure"
reliability? Because that would mean that if dropping some multicast
packets now and then is not really an issue, I could just set this to 1
in my case.

--
MVH
Ignacio Quezada