BLE Midi problem with mixed 16/128Bit UUIDs in characteristics

BLE Midi problem with mixed 16/128Bit UUIDs in characteristics

[Johannes Deisenhofer]

[-- Attachment #1: Type: text/plain, Size: 3109 bytes --]

Hi,

I could use some help understanding and hopefully fixing a BLE Midi problem.

I have bought a new MIDI device (Roland GO:Keys). This device connects 
fine via BLE, but does not send any key events. I have another keyboard 
working fine via BLE. Both devices work with android.

After digging into code and specs, I think that the chain of events is:
  - The MIDI Service is found
  - The characteristics are parsed
  - The code thinks that all characteristics in the PDU have 16bit UUIDS 
(len 7), while 3 of them. are 128bit
  - Thus it parses nonsense, and does not find the Midi I/O 
characteristic, no data transfer possible.

src/device.c:gatt_debug() MTU exchange complete, with MTU: 96
src/device.c:gatt_debug() Primary services found: 4
src/device.c:gatt_debug() start: 0x0001, end: 0x0003, uuid: 
00001800-0000-1000-8000-00805f9b34fb
src/device.c:gatt_debug() start: 0x0007, end: 0x000f, uuid: 
03b80e5a-ede8-4b33-a751-6ce34ec4c700
src/device.c:gatt_debug() start: 0x0021, end: 0x0023, uuid: 
00001801-0000-1000-8000-00805f9b34fb
src/device.c:gatt_debug() start: 0x0031, end: 0x0041, uuid: 
0000180a-0000-1000-8000-00805f9b34fb
src/device.c:gatt_debug() Characteristics found: 19
src/device.c:gatt_debug() start: 0x0002, end: 0x0007, value: 0x0003, 
props: 0x0a, uuid: 00002a00-0000-1
src/device.c:gatt_debug() start: 0x0008, end: 0x28a5, value: 0x0009, 
props: 0x18, uuid: 00000318-0000-1
src/device.c:gatt_debug() start: 0x28a6, end: 0xa347, value: 0xecd8, 
props: 0x5e, uuid: 00001c91-0000-1
src/device.c:gatt_debug() start: 0xa348, end: 0x000a, value: 0x5343, 
props: 0xac, uuid: 00004953-0000-1
src/device.c:gatt_debug() start: 0x000b, end: 0x9d0f, value: 0x000c, 
props: 0x1e, uuid: 00006bf3-0000-1
src/device.c:gatt_debug() start: 0x9d10, end: 0x6840, value: 0xa9f2, 
props: 0x66, uuid: 000012a1-0000-1
...

 From parsing handle 8, he parses 3 16 bit UUIDs instead of one 128bit.

wireshark has the same problem and misparses the buffer.

The buffer is the result of a
ATT_OP_READ_BY_TYPE_RESP opcode, which contains a unit length for the 
whole buffer, which is correct for some, but no all characteristics in 
the PDU.

Comparing the attributs from my working with my not working device, I find:
- The not working ROLAND has a total of 3 characteristics with 128bit 
UUIDS for the MIDI Service, the one needed ist the second one.
- The device also contains an audio sink, which I think does not matter

Now for the things that I don't know and where I could use some help:
  - Bug in the bluez stack or should I write to Roland?
  - I have no idea why my other MIDI Device works, mixed 16bit and 
128bit UUIDS is needed in both cases
  - How can I make sure that an attribute response only contains UUIDs 
of one kind? That seems to be required, since there is only one 'len' 
element.
  - Any tips on what needs to be changed?

Also attached is a wireshark dissection and hexdump of the "Read by Type 
Response":

I have a very ugly workaround, but having spent way to much time on 
debugging this, I could also invest the work for a proper patch.

Thanks,
Jo
-- 
Johannes Deisenhofer

[-- Attachment #2: badrolandpdu.txt --]
[-- Type: text/plain, Size: 8835 bytes --]

Frame 1034: 20 bytes on wire (160 bits), 20 bytes captured (160 bits) on interface bluetooth-monitor, id 0
Bluetooth
Bluetooth Linux Monitor Transport
Bluetooth HCI ACL Packet
Bluetooth L2CAP Protocol
Bluetooth Attribute Protocol
    Opcode: Read By Type Response (0x09)
    Length: 7
    Attribute Data, Handle: 0x0002, Characteristic Handle: 0x0003, UUID: Device Name
        Handle: 0x0002 (Generic Access Profile: GATT Characteristic Declaration)
            [Service UUID: Generic Access Profile (0x1800)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x0a, Write, Read
        Characteristic Value Handle: 0x0003 (Generic Access Profile: Device Name)
            [Service UUID: Generic Access Profile (0x1800)]
            [UUID: Device Name (0x2a00)]
        UUID: Device Name (0x2a00)
    Attribute Data, Handle: 0x0008, Characteristic Handle: 0x0009, UUID: Unknown
        Handle: 0x0008 (Unknown: GATT Characteristic Declaration)
            [Service UUID: 03b80e5aede84b33a7516ce34ec4c700]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x18, Notify, Write
        Characteristic Value Handle: 0x0009 (Unknown: Unknown)
            [Service UUID: 03b80e5aede84b33a7516ce34ec4c700]
            [UUID: Unknown (0x0318)]
        UUID: Unknown (0x0318)
    Attribute Data, Handle: 0x28a6, Characteristic Handle: 0xecd8, UUID: Unknown
        Handle: 0x28a6 (Device Information: Model Number String: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [Characteristic UUID: Model Number String (0x2a24)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x5e, Authenticated Signed Writes, Notify, Write, Write without Response, Read
        Characteristic Value Handle: 0xecd8 (Device Information: Unknown)
            [Service UUID: Device Information (0x180a)]
            [UUID: Unknown (0x1c91)]
        UUID: Unknown (0x1c91)
    Attribute Data, Handle: 0xa348, Characteristic Handle: 0x5343, UUID: Unknown
        Handle: 0xa348 (Device Information: Unknown: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [Characteristic UUID: Unknown (0x4953)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0xac, Extended Properties, Indicate, Write, Write without Response
        Characteristic Value Handle: 0x5343 (Device Information: Unknown)
            [Service UUID: Device Information (0x180a)]
            [UUID: Unknown (0x4953)]
        UUID: Unknown (0x4953)
    Attribute Data, Handle: 0x000b, Characteristic Handle: 0x000c, UUID: Unknown
        Handle: 0x000b (Unknown: Unknown: GATT Characteristic Declaration)
            [Service UUID: 03b80e5aede84b33a7516ce34ec4c700]
            [Characteristic UUID: Unknown (0x0318)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x1e, Notify, Write, Write without Response, Read
        Characteristic Value Handle: 0x000c (Unknown: Unknown)
            [Service UUID: 03b80e5aede84b33a7516ce34ec4c700]
            [UUID: Unknown (0x6bf3)]
        UUID: Unknown (0x6bf3)
    Attribute Data, Handle: 0x9d10, Characteristic Handle: 0xa9f2, UUID: Unknown
        Handle: 0x9d10 (Device Information: Unknown: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [Characteristic UUID: Unknown (0x4953)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x66, Authenticated Signed Writes, Indicate, Write without Response, Read
        Characteristic Value Handle: 0xa9f2 (Device Information: Unknown)
            [Service UUID: Device Information (0x180a)]
            [UUID: Unknown (0x12a1)]
        UUID: Unknown (0x12a1)
    Attribute Data, Handle: 0x6841, Characteristic Handle: 0xe5db, UUID: Unknown
        Handle: 0x6841 (Device Information: Unknown: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [Characteristic UUID: Unknown (0x4953)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x38, Indicate, Notify, Write
        Characteristic Value Handle: 0xe5db (Device Information: Unknown)
            [Service UUID: Device Information (0x180a)]
            [UUID: Unknown (0x7772)]
        UUID: Unknown (0x7772)
    Attribute Data, Handle: 0x000e, Characteristic Handle: 0x000f, UUID: Unknown
        Handle: 0x000e (Unknown: Unknown: GATT Characteristic Declaration)
            [Service UUID: 03b80e5aede84b33a7516ce34ec4c700]
            [Characteristic UUID: Unknown (0x6bf3)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x08, Write
        Characteristic Value Handle: 0x000f (Unknown: Unknown)
            [Service UUID: 03b80e5aede84b33a7516ce34ec4c700]
            [UUID: Unknown (0x9bb3)]
        UUID: Unknown (0x9bb3)
    Attribute Data, Handle: 0x3472, Characteristic Handle: 0xd4ec, UUID: Unknown
        Handle: 0x3472 (Device Information: Model Number String: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [Characteristic UUID: Model Number String (0x2a24)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0xbe, Extended Properties, Indicate, Notify, Write, Write without Response, Read
        Characteristic Value Handle: 0xd4ec (Device Information: Unknown)
            [Service UUID: Device Information (0x180a)]
            [UUID: Unknown (0xf4a8)]
        UUID: Unknown (0xf4a8)
    Attribute Data, Handle: 0x4143, Characteristic Handle: 0x5343, UUID: Unknown
        Handle: 0x4143 (Device Information: Model Number String: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [Characteristic UUID: Model Number String (0x2a24)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x88, Extended Properties, Write
        Characteristic Value Handle: 0x5343 (Device Information: Unknown)
            [Service UUID: Device Information (0x180a)]
            [UUID: Unknown (0x4953)]
        UUID: Unknown (0x4953)
    Attribute Data, Handle: 0x0022, Characteristic Handle: 0x0023, UUID: Service Changed
        Handle: 0x0022 (Generic Attribute Profile: GATT Characteristic Declaration)
            [Service UUID: Generic Attribute Profile (0x1801)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x26, Indicate, Write without Response, Read
        Characteristic Value Handle: 0x0023 (Generic Attribute Profile: Service Changed)
            [Service UUID: Generic Attribute Profile (0x1801)]
            [UUID: Service Changed (0x2a05)]
        UUID: Service Changed (0x2a05)
    Attribute Data, Handle: 0x0032, Characteristic Handle: 0x0033, UUID: Manufacturer Name String
        Handle: 0x0032 (Device Information: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x02, Read
        Characteristic Value Handle: 0x0033 (Device Information: Manufacturer Name String)
            [Service UUID: Device Information (0x180a)]
            [UUID: Manufacturer Name String (0x2a29)]
        UUID: Manufacturer Name String (0x2a29)
    Attribute Data, Handle: 0x0034, Characteristic Handle: 0x0035, UUID: Model Number String
        Handle: 0x0034 (Device Information: Manufacturer Name String: GATT Characteristic Declaration)
            [Service UUID: Device Information (0x180a)]
            [Characteristic UUID: Manufacturer Name String (0x2a29)]
            [UUID: GATT Characteristic Declaration (0x2803)]
        Characteristic Properties: 0x02, Read
        Characteristic Value Handle: 0x0035 (Device Information: Model Number String)
            [Service UUID: Device Information (0x180a)]
            [UUID: Model Number String (0x2a24)]
        UUID: Model Number String (0x2a24)
    [UUID: GATT Characteristic Declaration (0x2803)]
    [Request in Frame: 1028]


0000   5d 00 04 00 09 07 02 00 0a 03 00 00 2a 08 00 18   ]...........*...
0010   09 00 18 03 a6 28 5e d8 ec 91 1c 48 a3 ac 43 53   .....(^....H..CS
0020   53 49 0b 00 1e 0c 00 f3 6b 10 9d 66 f2 a9 a1 12   SI......k..f....
0030   41 68 38 db e5 72 77 0e 00 08 0f 00 b3 9b 72 34   Ah8..rw.......r4
0040   be ec d4 a8 f4 43 41 88 43 53 53 49 22 00 26 23   .....CA.CSSI".&#
0050   00 05 2a 32 00 02 33 00 29 2a 34 00 02 35 00 24   ..*2..3.)*4..5.$
0060   2a                                                *

Re: BLE Midi problem with mixed 16/128Bit UUIDs in characteristics

[Luiz Augusto von Dentz]

Hi Johannes,

On Tue, Dec 15, 2020 at 6:43 AM Johannes Deisenhofer
<jo.deisenhofer@gmail.com> wrote:
>
> Hi,
>
> I could use some help understanding and hopefully fixing a BLE Midi problem.
>
> I have bought a new MIDI device (Roland GO:Keys). This device connects
> fine via BLE, but does not send any key events. I have another keyboard
> working fine via BLE. Both devices work with android.
>
> After digging into code and specs, I think that the chain of events is:
>   - The MIDI Service is found
>   - The characteristics are parsed
>   - The code thinks that all characteristics in the PDU have 16bit UUIDS
> (len 7), while 3 of them. are 128bit

The spec doesn't allow mixing values of different sizes, or does it
first return the 16 bits one and then later 3 are return in a
different response?

>   - Thus it parses nonsense, and does not find the Midi I/O
> characteristic, no data transfer possible.
>
> src/device.c:gatt_debug() MTU exchange complete, with MTU: 96
> src/device.c:gatt_debug() Primary services found: 4
> src/device.c:gatt_debug() start: 0x0001, end: 0x0003, uuid:
> 00001800-0000-1000-8000-00805f9b34fb
> src/device.c:gatt_debug() start: 0x0007, end: 0x000f, uuid:
> 03b80e5a-ede8-4b33-a751-6ce34ec4c700
> src/device.c:gatt_debug() start: 0x0021, end: 0x0023, uuid:
> 00001801-0000-1000-8000-00805f9b34fb
> src/device.c:gatt_debug() start: 0x0031, end: 0x0041, uuid:
> 0000180a-0000-1000-8000-00805f9b34fb
> src/device.c:gatt_debug() Characteristics found: 19
> src/device.c:gatt_debug() start: 0x0002, end: 0x0007, value: 0x0003,
> props: 0x0a, uuid: 00002a00-0000-1
> src/device.c:gatt_debug() start: 0x0008, end: 0x28a5, value: 0x0009,
> props: 0x18, uuid: 00000318-0000-1
> src/device.c:gatt_debug() start: 0x28a6, end: 0xa347, value: 0xecd8,
> props: 0x5e, uuid: 00001c91-0000-1
> src/device.c:gatt_debug() start: 0xa348, end: 0x000a, value: 0x5343,
> props: 0xac, uuid: 00004953-0000-1
> src/device.c:gatt_debug() start: 0x000b, end: 0x9d0f, value: 0x000c,
> props: 0x1e, uuid: 00006bf3-0000-1
> src/device.c:gatt_debug() start: 0x9d10, end: 0x6840, value: 0xa9f2,
> props: 0x66, uuid: 000012a1-0000-1
> ...
>
>  From parsing handle 8, he parses 3 16 bit UUIDs instead of one 128bit.
>
> wireshark has the same problem and misparses the buffer.
>
> The buffer is the result of a
> ATT_OP_READ_BY_TYPE_RESP opcode, which contains a unit length for the
> whole buffer, which is correct for some, but no all characteristics in
> the PDU.
>
> Comparing the attributs from my working with my not working device, I find:
> - The not working ROLAND has a total of 3 characteristics with 128bit
> UUIDS for the MIDI Service, the one needed ist the second one.
> - The device also contains an audio sink, which I think does not matter
>
> Now for the things that I don't know and where I could use some help:
>   - Bug in the bluez stack or should I write to Roland?
>   - I have no idea why my other MIDI Device works, mixed 16bit and
> 128bit UUIDS is needed in both cases
>   - How can I make sure that an attribute response only contains UUIDs
> of one kind? That seems to be required, since there is only one 'len'
> element.
>   - Any tips on what needs to be changed?

Well if the device is returning mixed UUID sizes then there is nothing
we can do to figure out since as you said there is only one len so all
elements should be of the same length, perhaps Android doesn't use
Read By Type procedure and discover them, anyway it is perhaps worth
notifying them about this problem given that it doesn't seem to
conform to the spec.

> Also attached is a wireshark dissection and hexdump of the "Read by Type
> Response":
>
> I have a very ugly workaround, but having spent way to much time on
> debugging this, I could also invest the work for a proper patch.
>
> Thanks,
> Jo
> --
> Johannes Deisenhofer



-- 
Luiz Augusto von Dentz

Re: BLE Midi problem with mixed 16/128Bit UUIDs in characteristics

[Johannes Deisenhofer]

On 12/15/20 6:51 PM, Luiz Augusto von Dentz wrote:
> Hi Johannes,
> 

Hi,

> 
> The spec doesn't allow mixing values of different sizes, or does it
> first return the 16 bits one and then later 3 are return in a
> different response?
>

No, all in one response (which is re-assembled from two HCI packets).

My working device (DIY, arduino, but probably based on nordic semi code)
does the right thing:
It returns all handles with 16 bit, the client requests a continuation,
which results in the MIDI I/O characteristic, 128Bit, in another response.

[ cut ]

> 
> Well if the device is returning mixed UUID sizes then there is nothing
> we can do to figure out since as you said there is only one len so all
> elements should be of the same length, perhaps Android doesn't use
> Read By Type procedure and discover them, anyway it is perhaps worth
> notifying them about this problem given that it doesn't seem to
> conform to the spec.
>

Thanks for clarifying. So quite obvious a bug in their (Roland's) 
implementation. I hope they care enough.
I contacted them through their customer support forum, but I don't have 
much hope getting by the first-level support there.
If anybody has a better contact...

In this case, it would help to fetch the characteristics service by 
service instead of all in one. All characteristic UUIDs for the MIDI 
service are 128bit, the rest is all 16 bit. Could be the reason it works 
with Windows, Android, OSX, and whatever else they test with.

 From my limited understandig, that could probably be changed, but needs 
to be done in the general code, slowing everybodys pairing time down. A 
non-starter, I guess, for a single buggy device.

So I'll keep a fork with my super-ugly workaround and hope for roland.
I have to rebuild bluez anyway because my distro does not use 
--enable-midi.

Thanks!
Jo
-- 
Johannes Deisenhofer

Re: BLE Midi problem with mixed 16/128Bit UUIDs in characteristics

[Luiz Augusto von Dentz]

Hi Johannes,

On Tue, Dec 15, 2020 at 11:58 AM Johannes Deisenhofer
<jo.deisenhofer@gmail.com> wrote:
>
> On 12/15/20 6:51 PM, Luiz Augusto von Dentz wrote:
> > Hi Johannes,
> >
>
> Hi,
>
> >
> > The spec doesn't allow mixing values of different sizes, or does it
> > first return the 16 bits one and then later 3 are return in a
> > different response?
> >
>
> No, all in one response (which is re-assembled from two HCI packets).
>
> My working device (DIY, arduino, but probably based on nordic semi code)
> does the right thing:
> It returns all handles with 16 bit, the client requests a continuation,
> which results in the MIDI I/O characteristic, 128Bit, in another response.
>
> [ cut ]
>
> >
> > Well if the device is returning mixed UUID sizes then there is nothing
> > we can do to figure out since as you said there is only one len so all
> > elements should be of the same length, perhaps Android doesn't use
> > Read By Type procedure and discover them, anyway it is perhaps worth
> > notifying them about this problem given that it doesn't seem to
> > conform to the spec.
> >
>
> Thanks for clarifying. So quite obvious a bug in their (Roland's)
> implementation. I hope they care enough.
> I contacted them through their customer support forum, but I don't have
> much hope getting by the first-level support there.
> If anybody has a better contact...
>
> In this case, it would help to fetch the characteristics service by
> service instead of all in one. All characteristic UUIDs for the MIDI
> service are 128bit, the rest is all 16 bit. Could be the reason it works
> with Windows, Android, OSX, and whatever else they test with.

Yep, it is quite possible others OSes don't take advantage of big
UUIDs like we do since we can discover more than one service at time
that speeds up the discovery procedure quite a lot depending on MTU
size.

>  From my limited understandig, that could probably be changed, but needs
> to be done in the general code, slowing everybodys pairing time down. A
> non-starter, I guess, for a single buggy device.
>
> So I'll keep a fork with my super-ugly workaround and hope for roland.
> I have to rebuild bluez anyway because my distro does not use
> --enable-midi.

We should at least attempt to validate the response since it appears
we don't detect there is more data than expected but if the total
length is actually a multiple of the elements len it would still
create invalid attributes in the database. Anyway I would reach to the
manufacturer since they are clearly not following the Bluetooth Core
spec to the letter here.

> Thanks!
> Jo
> --
> Johannes Deisenhofer



-- 
Luiz Augusto von Dentz