Thank you so much for your reply, Luiz. We have a small device which runs a Linux system in it. In this embedded Linux system, there is GATT server runs BLE service which provides the services with the characteristics for reading and writing from the mobile device such as iPhone. The folder "/var/lib/bluetooth" is in the embedded Linux system. We call this embedded Linux as BLE Server, and the iPhone as the BLE Client. During the testing a few weeks ago, not only our mobile app crashed, but the nRF also crashed as the service was changed. Since we have no way to tell whether the ServiceChanged indication works or not. We disabled the bluetooth cache. Then we have this new issue. I never saw the "[ServiceChanged]" section from the "info" file. I tried to change the attributes of some characteristics. For example, adding notification in some characteristics. Then restart the BlE service from the Linux system. I did check the "info" file after that, but I didn't see the "[ServiceChanged]" section. Is there another way to have the "[ServiceChanged]" showing up? Today, I ran the testing again by doing the firmware upgrade. Right after the firmware upgrade, I used the btmon to capture the characteristics reading by using our mobile app. Please see the attached log. As you can see, there are a lot of errors because there are 5 services which only have the service itself listed(I checked it by using nRF). For about 30 characteristics under those services can't be seen. That's why there are errors when the mobile app tries to read the data. I double checked, the BLE python files between these two versions of the firmware are exactly the same. So there are no ServiceChanged. As the reading didn't work, I had to remove the "/var/lib/bluetooth" folder. Then this folder was recreated again. I compared the file "attributes" and "info" before and after removal of the "/var/lib/bluetooth" folder. The "attributes" files are exactly the same. The "info" files are different. But it makes sense because only the "[SlaveLongTermKey]" and "[LinkKey]" are different which is caused by re-pairing from the mobile side, I guess. So in this case, the BLE service python code is exactly the same. The "attributes" and "info" files are also the same or slightly different because of the re-pairing. How come some characteristics cannot be seen from nRF? As those characteristics are not listed in nRF, neither nRF or our mobile app can read the data. Honestly, this situation is pretty much the same as there is no firmware upgrade because the service file itself and those files under "/var/lib/bluetooth" are almost the same. How come the normal reading from the mobile app works fine? But after the firmware upgrade, some characteristics can't be seen? Is there anything wrong about advertisement? In fact, the firmware upgrade only copy over the "/var/lib/bluetooth" in order to keep the pairing information. I have no idea something went wrong. Are there anything else to look into? Thanks in advance! On Wed, Mar 17, 2021 at 11:15 AM Luiz Augusto von Dentz wrote: > > Hi Kenny, > > On Wed, Mar 17, 2021 at 12:40 AM Kenny Bian wrote: > > > > In order to keep the pairing information, we carry over the > > "/var/lib/bluetooth" folder from an older build to the new build in > > the Linux system when doing firmware upgrade. This way, our end user > > doesn't have to choose "Forget This Device" each time after the > > firmware upgrade. Otherwise, our end user has to "Forget This Device" > > and re-pair because the pairing info can't be found in the Linux > > device. We also set the GATT cache as "no" in > > "/etc/bluetooth/main.conf". Then the mobile app won't crash when there > > is any "service changed" such as adding notification to some existing > > characteristics. > > But the cache only applies to client, service changed is for the > server not sure why your mobile would be crashing with that, or is it > disabling ServiceChanged indications with that? If there is no > indication that would explain why the services are not updated since > the mobile OS won't notice any changes on the database. > > > We organize our characteristics under the services. There are 10 > > services on the Linux device. Each service has 5 to 10 > > characteristics. However, we found a new problem recently. The problem > > is: the first 8 services are fine. But the last one or two services > > only have the service UUID listed in nRF or our mobile app. There are > > no characteristics showing up under these one or two services. This > > caused our mobile app can't read the characteristics data under these > > problematic services. I checked the nRF. It's the same thing. nRF only > > shows the last one or two services with the service UUIDs. There are > > no characteristics listed under those 2 services. > > > > In order to fix this issue, we have to do these(let's call it "The Steps"): > > - Remove the "/var/lib/bluetooth" folder from the Linux system > > - Restart our BlueZ based service in Python in the Linux system > > - Choose "Forget This Device" on the iPhone. After this operation, the > > "/var/lib/bluetooth" is generated in the Linux device. > > - Connect to the device from our mobile app or from nRF. We can see > > all of the 10 services in the list. Each characteristic under the > > service can be seen. > > - We are able to read the data by using our mobile app or by using nRF > > > > I compared the files in the "/var/lib/bluetooth" folder. As we > > disabled the GATT cache, there is no "cache" folder. There are only 2 > > files in the folder. Here is the folder structure: > > /var/lib/bluetooth: > > | > > +----00:11:22:33:44:55/ > > | > > +----6C:92:11:87:11:25/ > > | > > +----attributes > > +----info > > > > I kept one copy of the files after the firmware upgrade. I also kept a > > copy of the files after the folder was re-generated. I compared the > > file "attributes" and "info". For the attributes, they are the same. > > For the "info" file, there are only 3 differences: > > [General] > > Name=PhoneX > > . . . > > > > [SlaveLongTermKey] > > Key=252256E19E26F3ED9FD0B99B7B5B0193 > > . . . > > > > [LinkKey] > > Key=1B593296571E2C48CF6952E60395E304 > > . . . > > > > > > I don't think these differences could cause the issue. But the fact > > is, if I delete the "/var/lib/bluetooth" folder and go thru "The > > Steps" above, the problem will be gone. All of the services will have > > their characteristics showing up in nRF. The data can be read from the > > characteristics. This means the folder "/var/lib/bluetooth" is related > > to the issue. But I don't know why. > > > > Here is also another thing. I subscribed to the "PropertiesChanged" > > signal. In the callback function "properties_changed", I print out the > > "changed". Here is the code to subscribe the "PropertiesChanged": > > > > bus.add_signal_receiver( > > properties_changed, > > dbus_interface="org.freedesktop.DBus.Properties", > > signal_name="PropertiesChanged", > > arg0="org.bluez.Device1", > > path_keyword="path", > > ) > > > > If the folder "/var/lib/bluetooth" is removed and go thru "The Steps", > > I saw the log like this: > > dbus.Dictionary({dbus.String('UUIDs'): > > dbus.Array([dbus.String('00000000-deca-fade-deca-deafdecacafe'), > > dbus.String('00001000-0000-1000-8000-00805f9b34fb'), > > dbus.String('0000110a-0000-1000-8000-00805f9b34fb'), > > dbus.String('0000110c-0000-1000-8000-00805f9b34fb'), > > dbus.String('0000110e-0000-1000-8000-00805f9b34fb'), > > dbus.String('00001116-0000-1000-8000-00805f9b34fb'), > > dbus.String('0000111f-0000-1000-8000-00805f9b34fb'), > > dbus.String('0000112f-0000-1000-8000-00805f9b34fb'), > > dbus.String('00001132-0000-1000-8000-00805f9b34fb'), > > dbus.String('00001200-0000-1000-8000-00805f9b34fb'), > > dbus.String('00001800-0000-1000-8000-00805f9b34fb'), > > dbus.String('00001801-0000-1000-8000-00805f9b34fb'), > > dbus.String('00001805-0000-1000-8000-00805f9b34fb'), > > dbus.String('0000180a-0000-1000-8000-00805f9b34fb'), > > dbus.String('0000180f-0000-1000-8000-00805f9b34fb'), > > dbus.String('02030302-1d19-415f-86f2-22a2106a0a77'), > > dbus.String('7905f431-b5ce-4e99-a40f-4b1e122d00d0'), > > dbus.String('89d3502b-0f36-433a-8ef4-c502ad55f8dc'), > > dbus.String('9fa480e0-4967-4542-9390-d343dc5d04ae'), > > dbus.String('d0611e78-bbb4-4591-a5f8-487910ae4366')], > > signature=dbus.Signature('s'), variant_level=1), > > dbus.String('ServicesResolved'): dbus.Boolean(True, variant_level=1)}, > > signature=dbus.Signature('sv')) > > > > If I check the log after the firmware is upgraded without deleting the > > "/var/lib/bluetooth" folder, the log looks like this: > > dbus.Dictionary({dbus.String('ServicesResolved'): dbus.Boolean(True, > > variant_level=1)}, signature=dbus.Signature('sv')) > > > > As we can see, they are different. Is it something to do with D-Bus? > > > > We understand that if we don't carry over the "/var/lib/bluetooth" > > folder, then there is no this issue. But our end user will have to > > choose "Forget This Device" each time after the firmware is upgraded. > > Otherwise, the end user can't connect to the Linux device because the > > pairing info can't be found. Is there a way to fix this issue? Namely, > > is there a way to keep the pairing info and also avoid this issue? > > This sounds like ServiceChanged is not being properly propagated when > you do the firmware update, do you have ServiceChanged entry on the > storage: > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/settings-storage.txt#n321 > > Is there any indication on btmon when you first connect after doing > the firmware update? > > -- > Luiz Augusto von Dentz