* Background scanning and white list usage @ 2014-02-28 3:20 Marcel Holtmann 2014-02-28 6:44 ` Johan Hedberg 0 siblings, 1 reply; 7+ messages in thread From: Marcel Holtmann @ 2014-02-28 3:20 UTC (permalink / raw) To: linux-bluetooth Hi Andre, with the background scanning and auto-connect patches now merged, I want to move into the direction of utilizing the white list for background scanning. For every devices where we have an auto-connect entry, the device should be placed into the white list. From our management side, it makes sense to only allow auto-connect entries for identity addresses (meaning public or static addresses). If our identity addresses are not using LE Privacy this is pretty much straight forward. We just put the identity address into the white list and scan with the white list filter policy. This should be our first target. Start using the white list and populate it with our identity addresses. The complicated part comes into play when we have devices with LE Privacy enabled and when they are using resolvable private addresses. Meaning when our IRK list is populated with identity addresses and their IRKs. The only way to make this work with the current available controller features is if we program the RPA into the white list. Since that RPA is going to change over time, we need to stop scanning with the white list filter every now and then, scan for all devices and resolve the RPA. If we see a new RPA for a know IRK, we have to replace the old RPA in the white list with the new RPA. And then we go back to scanning with the white list filter policy. Now the important question is what are good enough intervals to make this work smoothly. Devices using LE Privacy will take a hit in their re-connection time, but that is what we have to trade in for compared to waking up the host for every single advertising packet. My initial idea is to scan 5 minutes using the white list, then scan 10 seconds without the white list, then back to 5 minutes using the white list and so on. The default value for the PRA lifetime according to the specification is 15 minutes. I timed recent iOS devices which seem to be using 9 minutes intervals. So we have to play a little bit with this and see what are good values. Maybe 3 minutes white list scan and 5 seconds without white list is better. Things to try out. First task now should be to track the white list inside the kernel. I want a debugfs entry that tells us exactly what devices are in the white list. And second step is to make sure that when adding devices with auto-connect enabled, they get added to the white list, same as they get removed when connected or no longer have auto-connect enabled. Regards Marcel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Background scanning and white list usage 2014-02-28 3:20 Background scanning and white list usage Marcel Holtmann @ 2014-02-28 6:44 ` Johan Hedberg 2014-02-28 6:51 ` Marcel Holtmann 0 siblings, 1 reply; 7+ messages in thread From: Johan Hedberg @ 2014-02-28 6:44 UTC (permalink / raw) To: Marcel Holtmann; +Cc: linux-bluetooth Hi Marcel, On Thu, Feb 27, 2014, Marcel Holtmann wrote: > The complicated part comes into play when we have devices with LE > Privacy enabled and when they are using resolvable private addresses. > Meaning when our IRK list is populated with identity addresses and > their IRKs. The only way to make this work with the current available > controller features is if we program the RPA into the white list. > Since that RPA is going to change over time, we need to stop scanning > with the white list filter every now and then, scan for all devices > and resolve the RPA. If we see a new RPA for a know IRK, we have to > replace the old RPA in the white list with the new RPA. And then we go > back to scanning with the white list filter policy. > > Now the important question is what are good enough intervals to make > this work smoothly. Devices using LE Privacy will take a hit in their > re-connection time, but that is what we have to trade in for compared > to waking up the host for every single advertising packet. > > My initial idea is to scan 5 minutes using the white list, then scan > 10 seconds without the white list, then back to 5 minutes using the > white list and so on. > > The default value for the PRA lifetime according to the specification > is 15 minutes. I timed recent iOS devices which seem to be using 9 > minutes intervals. So we have to play a little bit with this and see > what are good values. > > Maybe 3 minutes white list scan and 5 seconds without white list is > better. Things to try out. I don't think this is a good idea at all. With LE starting advertising is typically seen as the initiating action of connection creation (unlike with BR/EDR where HCI_Create_Connection is the initiating action). Typically peripherals mean "connect to me now!" when they start connectable advertising. Let's stay you switch on your peripheral device, or it comes into range you haven't used it for some time (hours or even days). If it's using RPAs it's pretty much guaranteed to have a different one than what we know of and even if we're using 3 minutes white list scanning the user is on average going to have to wait for 1.5 minutes for the device to get connected which is not acceptable behavior (the extreme example would be if this is a keyboard or a mouse which you start using for the first time in the morning - moving the mouse or pressing a key on the keyboard should certainly get you a connection in less than 1 second). I fully understand the desire to use the white list as it's a very nice power saving feature, but I don't think we can win here as long as we don't have a way to have the controller do the resolving for us. One thing we could potentially try to do (but which I doubt is really worth it in the end) is to track the age of resolved RPAs. If we have an RPA which was resolved say less than 5 minutes ago we consider it appropriate to place into the white list. Otherwise we skip using the white list. Johan ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Background scanning and white list usage 2014-02-28 6:44 ` Johan Hedberg @ 2014-02-28 6:51 ` Marcel Holtmann 2014-03-06 21:15 ` Andre Guedes 0 siblings, 1 reply; 7+ messages in thread From: Marcel Holtmann @ 2014-02-28 6:51 UTC (permalink / raw) To: Johan Hedberg; +Cc: linux-bluetooth Hi Johan, >> The complicated part comes into play when we have devices with LE >> Privacy enabled and when they are using resolvable private addresses. >> Meaning when our IRK list is populated with identity addresses and >> their IRKs. The only way to make this work with the current available >> controller features is if we program the RPA into the white list. >> Since that RPA is going to change over time, we need to stop scanning >> with the white list filter every now and then, scan for all devices >> and resolve the RPA. If we see a new RPA for a know IRK, we have to >> replace the old RPA in the white list with the new RPA. And then we go >> back to scanning with the white list filter policy. >> >> Now the important question is what are good enough intervals to make >> this work smoothly. Devices using LE Privacy will take a hit in their >> re-connection time, but that is what we have to trade in for compared >> to waking up the host for every single advertising packet. >> >> My initial idea is to scan 5 minutes using the white list, then scan >> 10 seconds without the white list, then back to 5 minutes using the >> white list and so on. >> >> The default value for the PRA lifetime according to the specification >> is 15 minutes. I timed recent iOS devices which seem to be using 9 >> minutes intervals. So we have to play a little bit with this and see >> what are good values. >> >> Maybe 3 minutes white list scan and 5 seconds without white list is >> better. Things to try out. > > I don't think this is a good idea at all. With LE starting advertising > is typically seen as the initiating action of connection creation > (unlike with BR/EDR where HCI_Create_Connection is the initiating > action). Typically peripherals mean "connect to me now!" when they start > connectable advertising. > > Let's stay you switch on your peripheral device, or it comes into range > you haven't used it for some time (hours or even days). If it's using > RPAs it's pretty much guaranteed to have a different one than what we > know of and even if we're using 3 minutes white list scanning the user > is on average going to have to wait for 1.5 minutes for the device to > get connected which is not acceptable behavior (the extreme example > would be if this is a keyboard or a mouse which you start using for the > first time in the morning - moving the mouse or pressing a key on the > keyboard should certainly get you a connection in less than 1 second). HID devices would suffer the most here. Fully agree here. However since neither Microsoft Windows nor OS X can deal with RPAs at the moment, I do not think we are entering a dangerous zone here from an interoperability point of view. Actually Windows 8.1 is not able to connect to any random address for that matter. My point is that we certainly not make it worse. > I fully understand the desire to use the white list as it's a very nice > power saving feature, but I don't think we can win here as long as we > don't have a way to have the controller do the resolving for us. > > One thing we could potentially try to do (but which I doubt is really > worth it in the end) is to track the age of resolved RPAs. If we have an > RPA which was resolved say less than 5 minutes ago we consider it > appropriate to place into the white list. Otherwise we skip using the > white list. I was thinking about this as well. Over time we could learn the age of a RPA and thus schedule the scanning without white list times most efficient. Frankly, I want to get the white list usage going first. As long as you only have public or static addresses, it is the way to go. And then we optimize it when we have to deal with RPAs. We can not close our eyes to iOS devices using RPAs and I am not willing to take the power hit of getting flooded with advertising reports. Regards Marcel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Background scanning and white list usage 2014-02-28 6:51 ` Marcel Holtmann @ 2014-03-06 21:15 ` Andre Guedes 2014-03-07 7:04 ` Johan Hedberg 2014-03-07 7:30 ` Marcel Holtmann 0 siblings, 2 replies; 7+ messages in thread From: Andre Guedes @ 2014-03-06 21:15 UTC (permalink / raw) To: Marcel Holtmann, Johan Hedberg; +Cc: linux-bluetooth Hi Johan/Marcel, On 02/28/2014 03:51 AM, Marcel Holtmann wrote: > Hi Johan, > >>> The complicated part comes into play when we have devices with >>> LE Privacy enabled and when they are using resolvable private >>> addresses. Meaning when our IRK list is populated with identity >>> addresses and their IRKs. The only way to make this work with the >>> current available controller features is if we program the RPA >>> into the white list. Since that RPA is going to change over time, >>> we need to stop scanning with the white list filter every now and >>> then, scan for all devices and resolve the RPA. If we see a new >>> RPA for a know IRK, we have to replace the old RPA in the white >>> list with the new RPA. And then we go back to scanning with the >>> white list filter policy. >>> >>> Now the important question is what are good enough intervals to >>> make this work smoothly. Devices using LE Privacy will take a hit >>> in their re-connection time, but that is what we have to trade in >>> for compared to waking up the host for every single advertising >>> packet. >>> >>> My initial idea is to scan 5 minutes using the white list, then >>> scan 10 seconds without the white list, then back to 5 minutes >>> using the white list and so on. >>> >>> The default value for the PRA lifetime according to the >>> specification is 15 minutes. I timed recent iOS devices which >>> seem to be using 9 minutes intervals. So we have to play a little >>> bit with this and see what are good values. >>> >>> Maybe 3 minutes white list scan and 5 seconds without white list >>> is better. Things to try out. >> >> I don't think this is a good idea at all. With LE starting >> advertising is typically seen as the initiating action of >> connection creation (unlike with BR/EDR where HCI_Create_Connection >> is the initiating action). Typically peripherals mean "connect to >> me now!" when they start connectable advertising. >> >> Let's stay you switch on your peripheral device, or it comes into >> range you haven't used it for some time (hours or even days). If >> it's using RPAs it's pretty much guaranteed to have a different one >> than what we know of and even if we're using 3 minutes white list >> scanning the user is on average going to have to wait for 1.5 >> minutes for the device to get connected which is not acceptable >> behavior (the extreme example would be if this is a keyboard or a >> mouse which you start using for the first time in the morning - >> moving the mouse or pressing a key on the keyboard should certainly >> get you a connection in less than 1 second). > > HID devices would suffer the most here. Fully agree here. > > However since neither Microsoft Windows nor OS X can deal with RPAs > at the moment, I do not think we are entering a dangerous zone here > from an interoperability point of view. Actually Windows 8.1 is not > able to connect to any random address for that matter. > > My point is that we certainly not make it worse. HoG is not the only problem. I'm afraid delaying re-connection 3-5 minutes we might become others profiles impractical (e.g. Fob keeps beeping even if it is beside the cellphone). >> I fully understand the desire to use the white list as it's a very >> nice power saving feature, but I don't think we can win here as >> long as we don't have a way to have the controller do the resolving >> for us. >> >> One thing we could potentially try to do (but which I doubt is >> really worth it in the end) is to track the age of resolved RPAs. >> If we have an RPA which was resolved say less than 5 minutes ago we >> consider it appropriate to place into the white list. Otherwise we >> skip using the white list. > > I was thinking about this as well. Over time we could learn the age > of a RPA and thus schedule the scanning without white list times most > efficient. Frankly, I want to get the white list usage going first. > As long as you only have public or static addresses, it is the way to > go. And then we optimize it when we have to deal with RPAs. I agree with Johan about this aging approach, I seems not worthy. Honestly, I don't see how we would properly deal with white list and LE Privacy issue by now. > We can not close our eyes to iOS devices using RPAs and I am not > willing to take the power hit of getting flooded with advertising > reports. To fix this other issue (advertising flooding), filtering duplicates is just fine. The problem with enabling filter duplicates in background scan happens in the following scenario: * Background scan is running. * A device disconnects and starts advertising. * Before host gets the disconnect event, the advertising is reported to host. Since there is no pending LE connection at that time, nothing happens. * Host gets the disconnection event and adds a pending connection. * No advertising is reported (since controller is filtering) and the connection is never established. To address this scenario, all we have to do is: always restart background scan when a new LE pending connection is added. This way, we unsure that we don't miss the advertising report. If you guys agree with this approach I can write the patches. Regards, Andre ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Background scanning and white list usage 2014-03-06 21:15 ` Andre Guedes @ 2014-03-07 7:04 ` Johan Hedberg 2014-03-07 7:30 ` Marcel Holtmann 1 sibling, 0 replies; 7+ messages in thread From: Johan Hedberg @ 2014-03-07 7:04 UTC (permalink / raw) To: Andre Guedes; +Cc: Marcel Holtmann, linux-bluetooth Hi Andre, On Thu, Mar 06, 2014, Andre Guedes wrote: > The problem with enabling filter duplicates in background scan > happens in the following scenario: > * Background scan is running. > * A device disconnects and starts advertising. > * Before host gets the disconnect event, the advertising is reported > to host. Since there is no pending LE connection at that time, > nothing happens. > * Host gets the disconnection event and adds a pending connection. > * No advertising is reported (since controller is filtering) and the > connection is never established. > > To address this scenario, all we have to do is: always restart > background scan when a new LE pending connection is added. This way, > we unsure that we don't miss the advertising report. > > If you guys agree with this approach I can write the patches. Sounds fairly reasonable to me, if it actually end up working ok. Johan ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Background scanning and white list usage 2014-03-06 21:15 ` Andre Guedes 2014-03-07 7:04 ` Johan Hedberg @ 2014-03-07 7:30 ` Marcel Holtmann 2014-03-10 14:21 ` Andre Guedes 1 sibling, 1 reply; 7+ messages in thread From: Marcel Holtmann @ 2014-03-07 7:30 UTC (permalink / raw) To: Andre Guedes; +Cc: Johan Hedberg, linux-bluetooth Hi Andre, >>>> The complicated part comes into play when we have devices with >>>> LE Privacy enabled and when they are using resolvable private >>>> addresses. Meaning when our IRK list is populated with identity >>>> addresses and their IRKs. The only way to make this work with the >>>> current available controller features is if we program the RPA >>>> into the white list. Since that RPA is going to change over time, >>>> we need to stop scanning with the white list filter every now and >>>> then, scan for all devices and resolve the RPA. If we see a new >>>> RPA for a know IRK, we have to replace the old RPA in the white >>>> list with the new RPA. And then we go back to scanning with the >>>> white list filter policy. >>>> >>>> Now the important question is what are good enough intervals to >>>> make this work smoothly. Devices using LE Privacy will take a hit >>>> in their re-connection time, but that is what we have to trade in >>>> for compared to waking up the host for every single advertising >>>> packet. >>>> >>>> My initial idea is to scan 5 minutes using the white list, then >>>> scan 10 seconds without the white list, then back to 5 minutes >>>> using the white list and so on. >>>> >>>> The default value for the PRA lifetime according to the >>>> specification is 15 minutes. I timed recent iOS devices which >>>> seem to be using 9 minutes intervals. So we have to play a little >>>> bit with this and see what are good values. >>>> >>>> Maybe 3 minutes white list scan and 5 seconds without white list >>>> is better. Things to try out. >>> >>> I don't think this is a good idea at all. With LE starting >>> advertising is typically seen as the initiating action of >>> connection creation (unlike with BR/EDR where HCI_Create_Connection >>> is the initiating action). Typically peripherals mean "connect to >>> me now!" when they start connectable advertising. >>> >>> Let's stay you switch on your peripheral device, or it comes into >>> range you haven't used it for some time (hours or even days). If >>> it's using RPAs it's pretty much guaranteed to have a different one >>> than what we know of and even if we're using 3 minutes white list >>> scanning the user is on average going to have to wait for 1.5 >>> minutes for the device to get connected which is not acceptable >>> behavior (the extreme example would be if this is a keyboard or a >>> mouse which you start using for the first time in the morning - >>> moving the mouse or pressing a key on the keyboard should certainly >>> get you a connection in less than 1 second). >> >> HID devices would suffer the most here. Fully agree here. >> >> However since neither Microsoft Windows nor OS X can deal with RPAs >> at the moment, I do not think we are entering a dangerous zone here >> from an interoperability point of view. Actually Windows 8.1 is not >> able to connect to any random address for that matter. >> >> My point is that we certainly not make it worse. > > HoG is not the only problem. I'm afraid delaying re-connection 3-5 > minutes we might become others profiles impractical (e.g. Fob keeps > beeping even if it is beside the cellphone). to be honest not every bonded device needs to auto-reconnect. So this might all work out and when resolution in the controller becomes available, it can be nicely switched over. >>> I fully understand the desire to use the white list as it's a very >>> nice power saving feature, but I don't think we can win here as >>> long as we don't have a way to have the controller do the resolving >>> for us. >>> >>> One thing we could potentially try to do (but which I doubt is >>> really worth it in the end) is to track the age of resolved RPAs. >>> If we have an RPA which was resolved say less than 5 minutes ago we >>> consider it appropriate to place into the white list. Otherwise we >>> skip using the white list. >> >> I was thinking about this as well. Over time we could learn the age >> of a RPA and thus schedule the scanning without white list times most >> efficient. Frankly, I want to get the white list usage going first. >> As long as you only have public or static addresses, it is the way to >> go. And then we optimize it when we have to deal with RPAs. > > I agree with Johan about this aging approach, I seems not worthy. Honestly, I don't see how we would properly deal with white list and LE Privacy issue by now. As I said, we try out best and see how it works out. The interval can be tweaked of course. >> We can not close our eyes to iOS devices using RPAs and I am not >> willing to take the power hit of getting flooded with advertising >> reports. > > To fix this other issue (advertising flooding), filtering duplicates is just fine. > > The problem with enabling filter duplicates in background scan happens in the following scenario: > * Background scan is running. > * A device disconnects and starts advertising. > * Before host gets the disconnect event, the advertising is reported > to host. Since there is no pending LE connection at that time, > nothing happens. > * Host gets the disconnection event and adds a pending connection. > * No advertising is reported (since controller is filtering) and the > connection is never established. that is why we have to disable and re-enable scanning from time to time. > To address this scenario, all we have to do is: always restart background scan when a new LE pending connection is added. This way, we unsure that we don't miss the advertising report. > > If you guys agree with this approach I can write the patches. I want to start using the white list for scanning. That is our next step. Regards Marcel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Background scanning and white list usage 2014-03-07 7:30 ` Marcel Holtmann @ 2014-03-10 14:21 ` Andre Guedes 0 siblings, 0 replies; 7+ messages in thread From: Andre Guedes @ 2014-03-10 14:21 UTC (permalink / raw) To: Marcel Holtmann; +Cc: Johan Hedberg, linux-bluetooth Hi Johan/Marcel, On 03/07/2014 04:30 AM, Marcel Holtmann wrote: > Hi Andre, > >>>>> The complicated part comes into play when we have devices with >>>>> LE Privacy enabled and when they are using resolvable private >>>>> addresses. Meaning when our IRK list is populated with identity >>>>> addresses and their IRKs. The only way to make this work with the >>>>> current available controller features is if we program the RPA >>>>> into the white list. Since that RPA is going to change over time, >>>>> we need to stop scanning with the white list filter every now and >>>>> then, scan for all devices and resolve the RPA. If we see a new >>>>> RPA for a know IRK, we have to replace the old RPA in the white >>>>> list with the new RPA. And then we go back to scanning with the >>>>> white list filter policy. >>>>> >>>>> Now the important question is what are good enough intervals to >>>>> make this work smoothly. Devices using LE Privacy will take a hit >>>>> in their re-connection time, but that is what we have to trade in >>>>> for compared to waking up the host for every single advertising >>>>> packet. >>>>> >>>>> My initial idea is to scan 5 minutes using the white list, then >>>>> scan 10 seconds without the white list, then back to 5 minutes >>>>> using the white list and so on. >>>>> >>>>> The default value for the PRA lifetime according to the >>>>> specification is 15 minutes. I timed recent iOS devices which >>>>> seem to be using 9 minutes intervals. So we have to play a little >>>>> bit with this and see what are good values. >>>>> >>>>> Maybe 3 minutes white list scan and 5 seconds without white list >>>>> is better. Things to try out. >>>> >>>> I don't think this is a good idea at all. With LE starting >>>> advertising is typically seen as the initiating action of >>>> connection creation (unlike with BR/EDR where HCI_Create_Connection >>>> is the initiating action). Typically peripherals mean "connect to >>>> me now!" when they start connectable advertising. >>>> >>>> Let's stay you switch on your peripheral device, or it comes into >>>> range you haven't used it for some time (hours or even days). If >>>> it's using RPAs it's pretty much guaranteed to have a different one >>>> than what we know of and even if we're using 3 minutes white list >>>> scanning the user is on average going to have to wait for 1.5 >>>> minutes for the device to get connected which is not acceptable >>>> behavior (the extreme example would be if this is a keyboard or a >>>> mouse which you start using for the first time in the morning - >>>> moving the mouse or pressing a key on the keyboard should certainly >>>> get you a connection in less than 1 second). >>> >>> HID devices would suffer the most here. Fully agree here. >>> >>> However since neither Microsoft Windows nor OS X can deal with RPAs >>> at the moment, I do not think we are entering a dangerous zone here >>> from an interoperability point of view. Actually Windows 8.1 is not >>> able to connect to any random address for that matter. >>> >>> My point is that we certainly not make it worse. >> >> HoG is not the only problem. I'm afraid delaying re-connection 3-5 >> minutes we might become others profiles impractical (e.g. Fob keeps >> beeping even if it is beside the cellphone). > > to be honest not every bonded device needs to auto-reconnect. So this might all work out and when resolution in the controller becomes available, it can be nicely switched over. > >>>> I fully understand the desire to use the white list as it's a very >>>> nice power saving feature, but I don't think we can win here as >>>> long as we don't have a way to have the controller do the resolving >>>> for us. >>>> >>>> One thing we could potentially try to do (but which I doubt is >>>> really worth it in the end) is to track the age of resolved RPAs. >>>> If we have an RPA which was resolved say less than 5 minutes ago we >>>> consider it appropriate to place into the white list. Otherwise we >>>> skip using the white list. >>> >>> I was thinking about this as well. Over time we could learn the age >>> of a RPA and thus schedule the scanning without white list times most >>> efficient. Frankly, I want to get the white list usage going first. >>> As long as you only have public or static addresses, it is the way to >>> go. And then we optimize it when we have to deal with RPAs. >> >> I agree with Johan about this aging approach, I seems not worthy. Honestly, I don't see how we would properly deal with white list and LE Privacy issue by now. > > As I said, we try out best and see how it works out. The interval can be tweaked of course. > >>> We can not close our eyes to iOS devices using RPAs and I am not >>> willing to take the power hit of getting flooded with advertising >>> reports. >> >> To fix this other issue (advertising flooding), filtering duplicates is just fine. >> >> The problem with enabling filter duplicates in background scan happens in the following scenario: >> * Background scan is running. >> * A device disconnects and starts advertising. >> * Before host gets the disconnect event, the advertising is reported >> to host. Since there is no pending LE connection at that time, >> nothing happens. >> * Host gets the disconnection event and adds a pending connection. >> * No advertising is reported (since controller is filtering) and the >> connection is never established. > > that is why we have to disable and re-enable scanning from time to time. > >> To address this scenario, all we have to do is: always restart background scan when a new LE pending connection is added. This way, we unsure that we don't miss the advertising report. >> >> If you guys agree with this approach I can write the patches. > > I want to start using the white list for scanning. That is our next step. So, I think it would be fine we have first this advertising flooding issue fixed by enabling duplicate filter. Then we improve the background scan by using white list. At the end, after experiments, if the white list approach doesn't work well, we simply fall back to scan with duplicate filter enabled. I'll send patches soon. Regards, Andre ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2014-03-10 14:21 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-02-28 3:20 Background scanning and white list usage Marcel Holtmann 2014-02-28 6:44 ` Johan Hedberg 2014-02-28 6:51 ` Marcel Holtmann 2014-03-06 21:15 ` Andre Guedes 2014-03-07 7:04 ` Johan Hedberg 2014-03-07 7:30 ` Marcel Holtmann 2014-03-10 14:21 ` Andre Guedes
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.