Bluez blotoothctl scan vs hcitool scan

Bluez blotoothctl scan vs hcitool scan

[chris baker]

I'm running bluez 5.50 on a Raspberry Pi (both Buster and Stretch). I
have a ble sensor device that advertises data only when a button on
the sensor device is pressed. So advertisements are asynchronous and
there are no periodic advertisements in between (and all packets are
unique, no duplicates). I'm having an issue with Bluez though where
once a packet is received, Bluez seems to not report any additional
packets from the device for the next approximately 11 seconds (very
occasionally the interval is shorter). This is with the bluetoothctl
line command tool as well as my own c++ application (based off the
bluez client/main.c example). In both cases before starting a scan I
clear the scan filter, set transport to le, and set duplicate-data
reporting on. Conversely, when running "hcitool lescan" I see all the
packets from the sensor (it even seems to be reporting up to all 3
copies broadcast on the different advertisement channels).

Below is a scan covering about 20 seconds (editing out all unrelated
packets), where I'm pressing down the button on the sensor about every
2 seconds and then holding it down for another 2 seconds before
letting it up. The first chunk is from bluetoothctl, the second from
"hcitool lescan"/"hcidump --raw" (taken at the same time, on a second
raspberry pi). The first four bytes in the bluetoothctl packet data is
a little endian packet seq number incremented by the sensor for each
new packet. The next byte indicates a button up/down action. You can
see bluetoothctl reported only packets numbered 05df, 05e5, 05e9 (3
total). In the hci raw dump the seq number is at the end of the top
line. There you can see all packets are in order, reported 1 to 3
times (I assume it is reporting all advertising channels it catches).
All packets are present, 05df to 05e9, in the hcidump scan (11
packets).

So my question is, is there a way to get those missing advertisements
through the dbus api, possibly some additional setting somewhere? If
not, is the hci api okay to use from c++ and should it do the trick?
Is there another/better approach?

------ bluetoothctl
                                .
[NEW] Device E2:15:00:01:73:96 E2-15-00-01-73-96

[CHG] Device E2:15:00:01:73:96 RSSI: -46
[CHG] Device E2:15:00:01:73:96 ManufacturerData Key: 0x03da
[CHG] Device E2:15:00:01:73:96 ManufacturerData Value:
  df 05 00 00 10 a1 ac 8a b4                       .........

[CHG] Device E2:15:00:01:73:96 RSSI: -45
[CHG] Device E2:15:00:01:73:96 ManufacturerData Key: 0x03da
[CHG] Device E2:15:00:01:73:96 ManufacturerData Value:
  e5 05 00 00 10 e7 4f 67 6e                       ......Ogn
                                             .
[CHG] Device E2:15:00:01:73:96 RSSI: -65
[CHG] Device E2:15:00:01:73:96 ManufacturerData Key: 0x03da
[CHG] Device E2:15:00:01:73:96 ManufacturerData Value:
  e9 05 00 00 10 f4 f9 f8 7d                       ........}

---------- hcidump --raw

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 DF 05
  00 00 10 A1 AC 8A B4 C3
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 DF 05
  00 00 10 A1 AC 8A B4 BE

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E0 05
  00 00 11 11 0F 3E 24 B6

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E1 05
  00 00 10 F7 68 07 50 BE
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E1 05
  00 00 10 F7 68 07 50 CF
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E1 05
  00 00 10 F7 68 07 50 BA

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E2 05
  00 00 11 1D 18 A8 2A BF
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E2 05
  00 00 11 1D 18 A8 2A C0
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E2 05
  00 00 11 1D 18 A8 2A B8

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E3 05
  00 00 10 E2 29 C7 F7 BB

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E4 05
  00 00 11 57 F0 5C 76 BD
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E4 05
  00 00 11 57 F0 5C 76 C1

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E5 05
  00 00 10 E7 4F 67 6E CA

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E6 05
  00 00 11 77 63 92 CE C0
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E6 05
  00 00 11 77 63 92 CE BA
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E6 05
  00 00 11 77 63 92 CE BE

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E7 05
  00 00 10 2D 52 48 C2 BD

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E8 05
  00 00 11 EE 32 20 9D BD
> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E8 05
  00 00 11 EE 32 20 9D C1

> 04 3E 19 02 01 03 01 96 73 01 00 15 E2 0D 0C FF DA 03 E9 05
  00 00 10 F4 F9 F8 7D BC

Re: Bluez blotoothctl scan vs hcitool scan

[Barry Byford]

Hi Chris,

On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
>

> So my question is, is there a way to get those missing advertisements
> through the dbus api, possibly some additional setting somewhere?

Duplicates are disabled by default with the DBus API. This can be
controlled with the DuplicateData setting:
https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107

Regards,
Barry

Re: Bluez blotoothctl scan vs hcitool scan

[chris baker]

On Mon, Feb 24, 2020 at 6:08 AM Barry Byford <31baz66@gmail.com> wrote:
>
> Hi Chris,
>
> On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
> >
>
> > So my question is, is there a way to get those missing advertisements
> > through the dbus api, possibly some additional setting somewhere?
>
> Duplicates are disabled by default with the DBus API. This can be
> controlled with the DuplicateData setting:
> https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107
>
> Regards,
> Barry


My apologies, I guess I wasn't clear (long post) but, I turned
duplicate data on when using the bluetoothctl command (via the "scan"
submenu) and also used the flag, "hcitool lescan --duplicates", when
running the hcitool command. So both scans should have included any
duplicates (unless I've misunderstood something). Additionally, none
of the missing packets were duplicates (again, unless I'm
misunderstanding what "duplicates" means). each packet had a unique
sequence numbers as well as the button data field toggling.

Re: Bluez blotoothctl scan vs hcitool scan

[Barry Byford]

If the DBus API is not cutting it for you then using the MGMT API
https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/mgmt-api.txt
is what has been recommended in the past:
https://www.spinics.net/lists/linux-bluetooth/msg68959.html

On Mon, 24 Feb 2020 at 16:37, chris baker <chrisbkr2020@gmail.com> wrote:
>
> On Mon, Feb 24, 2020 at 6:08 AM Barry Byford <31baz66@gmail.com> wrote:
> >
> > Hi Chris,
> >
> > On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
> > >
> >
> > > So my question is, is there a way to get those missing advertisements
> > > through the dbus api, possibly some additional setting somewhere?
> >
> > Duplicates are disabled by default with the DBus API. This can be
> > controlled with the DuplicateData setting:
> > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107
> >
> > Regards,
> > Barry
>
>
> My apologies, I guess I wasn't clear (long post) but, I turned
> duplicate data on when using the bluetoothctl command (via the "scan"
> submenu) and also used the flag, "hcitool lescan --duplicates", when
> running the hcitool command. So both scans should have included any
> duplicates (unless I've misunderstood something). Additionally, none
> of the missing packets were duplicates (again, unless I'm
> misunderstanding what "duplicates" means). each packet had a unique
> sequence numbers as well as the button data field toggling.

Re: Bluez blotoothctl scan vs hcitool scan

[chris baker]

On Mon, Feb 24, 2020 at 9:13 AM Barry Byford <31baz66@gmail.com> wrote:
>
> If the DBus API is not cutting it for you then using the MGMT API
> https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/mgmt-api.txt
> is what has been recommended in the past:
> https://www.spinics.net/lists/linux-bluetooth/msg68959.html
>
> On Mon, 24 Feb 2020 at 16:37, chris baker <chrisbkr2020@gmail.com> wrote:
> >
> > On Mon, Feb 24, 2020 at 6:08 AM Barry Byford <31baz66@gmail.com> wrote:
> > >
> > > Hi Chris,
> > >
> > > On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
> > > >
> > >
> > > > So my question is, is there a way to get those missing advertisements
> > > > through the dbus api, possibly some additional setting somewhere?
> > >
> > > Duplicates are disabled by default with the DBus API. This can be
> > > controlled with the DuplicateData setting:
> > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107
> > >
> > > Regards,
> > > Barry
> >
> >
> > My apologies, I guess I wasn't clear (long post) but, I turned
> > duplicate data on when using the bluetoothctl command (via the "scan"
> > submenu) and also used the flag, "hcitool lescan --duplicates", when
> > running the hcitool command. So both scans should have included any
> > duplicates (unless I've misunderstood something). Additionally, none
> > of the missing packets were duplicates (again, unless I'm
> > misunderstanding what "duplicates" means). each packet had a unique
> > sequence numbers as well as the button data field toggling.

Great, thank you. I'll look into the MGMT api in the coming days. That
said, is it a problem to use both api's (MGMT/DBus) concurrently from
the same app? My application supports both connected BLE sensors as
well as BLE beacon type sensors. If possible I can handle them in two
different threads, but the DBus thread for connected sensors would
still occasionally need to scan for new sensors (using the DBus
discovery call) and would still need to get characteristic changed
callbacks as well.

Out of curiosity though, is the behavior I'm seeing normal? Or is the
sensor doing something improper possibly? Seeing as the packets aren't
duplicates why would they be filtered (or are they just not being
received to begin with for some reason)? The 11 second interval seems
kind of strange. Anyway, thanks again for the help! Chris

Re: Bluez blotoothctl scan vs hcitool scan

[Luiz Augusto von Dentz]

Hi Chris,

On Mon, Feb 24, 2020 at 1:56 PM chris baker <chrisbkr2020@gmail.com> wrote:
>
> On Mon, Feb 24, 2020 at 9:13 AM Barry Byford <31baz66@gmail.com> wrote:
> >
> > If the DBus API is not cutting it for you then using the MGMT API
> > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/mgmt-api.txt
> > is what has been recommended in the past:
> > https://www.spinics.net/lists/linux-bluetooth/msg68959.html
> >
> > On Mon, 24 Feb 2020 at 16:37, chris baker <chrisbkr2020@gmail.com> wrote:
> > >
> > > On Mon, Feb 24, 2020 at 6:08 AM Barry Byford <31baz66@gmail.com> wrote:
> > > >
> > > > Hi Chris,
> > > >
> > > > On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
> > > > >
> > > >
> > > > > So my question is, is there a way to get those missing advertisements
> > > > > through the dbus api, possibly some additional setting somewhere?
> > > >
> > > > Duplicates are disabled by default with the DBus API. This can be
> > > > controlled with the DuplicateData setting:
> > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107
> > > >
> > > > Regards,
> > > > Barry
> > >
> > >
> > > My apologies, I guess I wasn't clear (long post) but, I turned
> > > duplicate data on when using the bluetoothctl command (via the "scan"
> > > submenu) and also used the flag, "hcitool lescan --duplicates", when
> > > running the hcitool command. So both scans should have included any
> > > duplicates (unless I've misunderstood something). Additionally, none
> > > of the missing packets were duplicates (again, unless I'm
> > > misunderstanding what "duplicates" means). each packet had a unique
> > > sequence numbers as well as the button data field toggling.
>
> Great, thank you. I'll look into the MGMT api in the coming days. That
> said, is it a problem to use both api's (MGMT/DBus) concurrently from
> the same app? My application supports both connected BLE sensors as
> well as BLE beacon type sensors. If possible I can handle them in two
> different threads, but the DBus thread for connected sensors would
> still occasionally need to scan for new sensors (using the DBus
> discovery call) and would still need to get characteristic changed
> callbacks as well.

Have a look at the other thread subject: Adding support for
DuplicateData into the kernel

We are discussing adding support to disable duplicate via MGMT since
DuplicateData does not currently remove it but we might want to change
that, or at least have some alternative API to do that. We could for
example have a socket that enables a more direct access to the
advertisments, that way protocol that work over advertisement would
have a way to do this, in fact this might be better for things like
mesh so it can coexist with bluetoothd.

> Out of curiosity though, is the behavior I'm seeing normal? Or is the
> sensor doing something improper possibly? Seeing as the packets aren't
> duplicates why would they be filtered (or are they just not being
> received to begin with for some reason)? The 11 second interval seems
> kind of strange. Anyway, thanks again for the help! Chris



--
Luiz Augusto von Dentz

Re: Bluez blotoothctl scan vs hcitool scan

[chris baker]

On Mon, Feb 24, 2020 at 4:33 PM Luiz Augusto von Dentz
<luiz.dentz@gmail.com> wrote:
>
> Hi Chris,
>
> On Mon, Feb 24, 2020 at 1:56 PM chris baker <chrisbkr2020@gmail.com> wrote:
> >
> > On Mon, Feb 24, 2020 at 9:13 AM Barry Byford <31baz66@gmail.com> wrote:
> > >
> > > If the DBus API is not cutting it for you then using the MGMT API
> > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/mgmt-api.txt
> > > is what has been recommended in the past:
> > > https://www.spinics.net/lists/linux-bluetooth/msg68959.html
> > >
> > > On Mon, 24 Feb 2020 at 16:37, chris baker <chrisbkr2020@gmail.com> wrote:
> > > >
> > > > On Mon, Feb 24, 2020 at 6:08 AM Barry Byford <31baz66@gmail.com> wrote:
> > > > >
> > > > > Hi Chris,
> > > > >
> > > > > On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
> > > > > >
> > > > >
> > > > > > So my question is, is there a way to get those missing advertisements
> > > > > > through the dbus api, possibly some additional setting somewhere?
> > > > >
> > > > > Duplicates are disabled by default with the DBus API. This can be
> > > > > controlled with the DuplicateData setting:
> > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107
> > > > >
> > > > > Regards,
> > > > > Barry
> > > >
> > > >
> > > > My apologies, I guess I wasn't clear (long post) but, I turned
> > > > duplicate data on when using the bluetoothctl command (via the "scan"
> > > > submenu) and also used the flag, "hcitool lescan --duplicates", when
> > > > running the hcitool command. So both scans should have included any
> > > > duplicates (unless I've misunderstood something). Additionally, none
> > > > of the missing packets were duplicates (again, unless I'm
> > > > misunderstanding what "duplicates" means). each packet had a unique
> > > > sequence numbers as well as the button data field toggling.
> >
> > Great, thank you. I'll look into the MGMT api in the coming days. That
> > said, is it a problem to use both api's (MGMT/DBus) concurrently from
> > the same app? My application supports both connected BLE sensors as
> > well as BLE beacon type sensors. If possible I can handle them in two
> > different threads, but the DBus thread for connected sensors would
> > still occasionally need to scan for new sensors (using the DBus
> > discovery call) and would still need to get characteristic changed
> > callbacks as well.
>
> Have a look at the other thread subject: Adding support for
> DuplicateData into the kernel
>
> We are discussing adding support to disable duplicate via MGMT since
> DuplicateData does not currently remove it but we might want to change
> that, or at least have some alternative API to do that. We could for
> example have a socket that enables a more direct access to the
> advertisments, that way protocol that work over advertisement would
> have a way to do this, in fact this might be better for things like
> mesh so it can coexist with bluetoothd.
>
> > Out of curiosity though, is the behavior I'm seeing normal? Or is the
> > sensor doing something improper possibly? Seeing as the packets aren't
> > duplicates why would they be filtered (or are they just not being
> > received to begin with for some reason)? The 11 second interval seems
> > kind of strange. Anyway, thanks again for the help! Chris
>
>
>
> --
> Luiz Augusto von Dentz


Thanks Luiz, I don't want to sound dense, and I really appreciate you
and Barry's help, but these aren't duplicate packets (unless, again,
I'm misunderstanding the term). Each packet payload is completely
unique. I'll have a look at the other thread for sure, but I'm really
just trying to understand if the missing packets I identified in the
trace should be there (in the DBus/bluetoothctl trace) or if there's a
reason they were excluded. Again, they weren't duplicates and I'm
reasonably sure I had the duplicate data flags set correctly each
time. Also, whatever is going on is not transient, I can duplicate it
with the senor I'm testing every time (both in my app or via
bluetoothctl). More important for sure is to find a work-around
(hopefully the MGMT api Barry pointed me to) but still just curious
why these packets are getting dropped/filtered... Anyway, thanks again
to you both!

Re: Bluez blotoothctl scan vs hcitool scan

[Luiz Augusto von Dentz]

Hi Chris,

On Mon, Feb 24, 2020 at 5:36 PM chris baker <chrisbkr2020@gmail.com> wrote:
>
> On Mon, Feb 24, 2020 at 4:33 PM Luiz Augusto von Dentz
> <luiz.dentz@gmail.com> wrote:
> >
> > Hi Chris,
> >
> > On Mon, Feb 24, 2020 at 1:56 PM chris baker <chrisbkr2020@gmail.com> wrote:
> > >
> > > On Mon, Feb 24, 2020 at 9:13 AM Barry Byford <31baz66@gmail.com> wrote:
> > > >
> > > > If the DBus API is not cutting it for you then using the MGMT API
> > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/mgmt-api.txt
> > > > is what has been recommended in the past:
> > > > https://www.spinics.net/lists/linux-bluetooth/msg68959.html
> > > >
> > > > On Mon, 24 Feb 2020 at 16:37, chris baker <chrisbkr2020@gmail.com> wrote:
> > > > >
> > > > > On Mon, Feb 24, 2020 at 6:08 AM Barry Byford <31baz66@gmail.com> wrote:
> > > > > >
> > > > > > Hi Chris,
> > > > > >
> > > > > > On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
> > > > > > >
> > > > > >
> > > > > > > So my question is, is there a way to get those missing advertisements
> > > > > > > through the dbus api, possibly some additional setting somewhere?
> > > > > >
> > > > > > Duplicates are disabled by default with the DBus API. This can be
> > > > > > controlled with the DuplicateData setting:
> > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107
> > > > > >
> > > > > > Regards,
> > > > > > Barry
> > > > >
> > > > >
> > > > > My apologies, I guess I wasn't clear (long post) but, I turned
> > > > > duplicate data on when using the bluetoothctl command (via the "scan"
> > > > > submenu) and also used the flag, "hcitool lescan --duplicates", when
> > > > > running the hcitool command. So both scans should have included any
> > > > > duplicates (unless I've misunderstood something). Additionally, none
> > > > > of the missing packets were duplicates (again, unless I'm
> > > > > misunderstanding what "duplicates" means). each packet had a unique
> > > > > sequence numbers as well as the button data field toggling.
> > >
> > > Great, thank you. I'll look into the MGMT api in the coming days. That
> > > said, is it a problem to use both api's (MGMT/DBus) concurrently from
> > > the same app? My application supports both connected BLE sensors as
> > > well as BLE beacon type sensors. If possible I can handle them in two
> > > different threads, but the DBus thread for connected sensors would
> > > still occasionally need to scan for new sensors (using the DBus
> > > discovery call) and would still need to get characteristic changed
> > > callbacks as well.
> >
> > Have a look at the other thread subject: Adding support for
> > DuplicateData into the kernel
> >
> > We are discussing adding support to disable duplicate via MGMT since
> > DuplicateData does not currently remove it but we might want to change
> > that, or at least have some alternative API to do that. We could for
> > example have a socket that enables a more direct access to the
> > advertisments, that way protocol that work over advertisement would
> > have a way to do this, in fact this might be better for things like
> > mesh so it can coexist with bluetoothd.
> >
> > > Out of curiosity though, is the behavior I'm seeing normal? Or is the
> > > sensor doing something improper possibly? Seeing as the packets aren't
> > > duplicates why would they be filtered (or are they just not being
> > > received to begin with for some reason)? The 11 second interval seems
> > > kind of strange. Anyway, thanks again for the help! Chris
> >
> >
> >
> > --
> > Luiz Augusto von Dentz
>
>
> Thanks Luiz, I don't want to sound dense, and I really appreciate you
> and Barry's help, but these aren't duplicate packets (unless, again,
> I'm misunderstanding the term). Each packet payload is completely
> unique. I'll have a look at the other thread for sure, but I'm really
> just trying to understand if the missing packets I identified in the
> trace should be there (in the DBus/bluetoothctl trace) or if there's a
> reason they were excluded. Again, they weren't duplicates and I'm
> reasonably sure I had the duplicate data flags set correctly each
> time. Also, whatever is going on is not transient, I can duplicate it
> with the senor I'm testing every time (both in my app or via
> bluetoothctl). More important for sure is to find a work-around
> (hopefully the MGMT api Barry pointed me to) but still just curious
> why these packets are getting dropped/filtered... Anyway, thanks again
> to you both!

Afaik the HCI duplicate filter flag is not very granular, so the
controller may be dropping any reports found during a sort period
since it might not have enough memory to store everything, include
actual data in the advertisement, to compare with.

-- 
Luiz Augusto von Dentz

Re: Bluez blotoothctl scan vs hcitool scan

[chris baker]

Thanks Luiz, that makes sense and would explain why it ignores the
sensor's advertisements at pretty regular 11 second intervals. And
again, I don't want to keep bugging you guys but my last question is,
is there any issue using the MGMT api in conjunction with the normal
DBus BLE api's? For that matter if I end up having to go to the hci
level can that be mixed as well or will they step on each other
somehow? I'm envisioning putting the normal, "connected" sensors using
the DBus api calls (for scanning, connecting, receiving characteristic
data) in one thread and the beacon devices in a completely separate
thread, both running independently and using different api's (though
possibly sharing some data). If that's just not going to work it'll
help to know up front and I can look at alternatives. Thanks again,
chris

On Mon, Feb 24, 2020 at 5:52 PM Luiz Augusto von Dentz
<luiz.dentz@gmail.com> wrote:
>
> Hi Chris,
>
> On Mon, Feb 24, 2020 at 5:36 PM chris baker <chrisbkr2020@gmail.com> wrote:
> >
> > On Mon, Feb 24, 2020 at 4:33 PM Luiz Augusto von Dentz
> > <luiz.dentz@gmail.com> wrote:
> > >
> > > Hi Chris,
> > >
> > > On Mon, Feb 24, 2020 at 1:56 PM chris baker <chrisbkr2020@gmail.com> wrote:
> > > >
> > > > On Mon, Feb 24, 2020 at 9:13 AM Barry Byford <31baz66@gmail.com> wrote:
> > > > >
> > > > > If the DBus API is not cutting it for you then using the MGMT API
> > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/mgmt-api.txt
> > > > > is what has been recommended in the past:
> > > > > https://www.spinics.net/lists/linux-bluetooth/msg68959.html
> > > > >
> > > > > On Mon, 24 Feb 2020 at 16:37, chris baker <chrisbkr2020@gmail.com> wrote:
> > > > > >
> > > > > > On Mon, Feb 24, 2020 at 6:08 AM Barry Byford <31baz66@gmail.com> wrote:
> > > > > > >
> > > > > > > Hi Chris,
> > > > > > >
> > > > > > > On Mon, 24 Feb 2020 at 10:12, chris baker <chrisbkr2020@gmail.com> wrote:
> > > > > > > >
> > > > > > >
> > > > > > > > So my question is, is there a way to get those missing advertisements
> > > > > > > > through the dbus api, possibly some additional setting somewhere?
> > > > > > >
> > > > > > > Duplicates are disabled by default with the DBus API. This can be
> > > > > > > controlled with the DuplicateData setting:
> > > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt#n107
> > > > > > >
> > > > > > > Regards,
> > > > > > > Barry
> > > > > >
> > > > > >
> > > > > > My apologies, I guess I wasn't clear (long post) but, I turned
> > > > > > duplicate data on when using the bluetoothctl command (via the "scan"
> > > > > > submenu) and also used the flag, "hcitool lescan --duplicates", when
> > > > > > running the hcitool command. So both scans should have included any
> > > > > > duplicates (unless I've misunderstood something). Additionally, none
> > > > > > of the missing packets were duplicates (again, unless I'm
> > > > > > misunderstanding what "duplicates" means). each packet had a unique
> > > > > > sequence numbers as well as the button data field toggling.
> > > >
> > > > Great, thank you. I'll look into the MGMT api in the coming days. That
> > > > said, is it a problem to use both api's (MGMT/DBus) concurrently from
> > > > the same app? My application supports both connected BLE sensors as
> > > > well as BLE beacon type sensors. If possible I can handle them in two
> > > > different threads, but the DBus thread for connected sensors would
> > > > still occasionally need to scan for new sensors (using the DBus
> > > > discovery call) and would still need to get characteristic changed
> > > > callbacks as well.
> > >
> > > Have a look at the other thread subject: Adding support for
> > > DuplicateData into the kernel
> > >
> > > We are discussing adding support to disable duplicate via MGMT since
> > > DuplicateData does not currently remove it but we might want to change
> > > that, or at least have some alternative API to do that. We could for
> > > example have a socket that enables a more direct access to the
> > > advertisments, that way protocol that work over advertisement would
> > > have a way to do this, in fact this might be better for things like
> > > mesh so it can coexist with bluetoothd.
> > >
> > > > Out of curiosity though, is the behavior I'm seeing normal? Or is the
> > > > sensor doing something improper possibly? Seeing as the packets aren't
> > > > duplicates why would they be filtered (or are they just not being
> > > > received to begin with for some reason)? The 11 second interval seems
> > > > kind of strange. Anyway, thanks again for the help! Chris
> > >
> > >
> > >
> > > --
> > > Luiz Augusto von Dentz
> >
> >
> > Thanks Luiz, I don't want to sound dense, and I really appreciate you
> > and Barry's help, but these aren't duplicate packets (unless, again,
> > I'm misunderstanding the term). Each packet payload is completely
> > unique. I'll have a look at the other thread for sure, but I'm really
> > just trying to understand if the missing packets I identified in the
> > trace should be there (in the DBus/bluetoothctl trace) or if there's a
> > reason they were excluded. Again, they weren't duplicates and I'm
> > reasonably sure I had the duplicate data flags set correctly each
> > time. Also, whatever is going on is not transient, I can duplicate it
> > with the senor I'm testing every time (both in my app or via
> > bluetoothctl). More important for sure is to find a work-around
> > (hopefully the MGMT api Barry pointed me to) but still just curious
> > why these packets are getting dropped/filtered... Anyway, thanks again
> > to you both!
>
> Afaik the HCI duplicate filter flag is not very granular, so the
> controller may be dropping any reports found during a sort period
> since it might not have enough memory to store everything, include
> actual data in the advertisement, to compare with.
>
> --
> Luiz Augusto von Dentz