* noise issues when recording sound on i.MX28 @ 2016-01-27 10:56 Uwe Kleine-König 2016-01-27 11:53 ` Fabio Estevam 0 siblings, 1 reply; 13+ messages in thread From: Uwe Kleine-König @ 2016-01-27 10:56 UTC (permalink / raw) To: linux-arm-kernel Hello, I currently work with an i.MX28 based machine and occasionally when recording sound with arecord but no microphone connected the result contains much noise. I found commits 90ce77d4468e ENGR00285446-3 [MX28] SAIF: Bit Shift in SAIF RX Data 1ea685a12f19 ENGR00285446-2 [MX28] SAIF: Bit Shift in SAIF RX Data 1ca899221d8b ENGR00285446-1 [MX28] SAIF: Bit Shift in SAIF RX Data in the Freescale vendor kernel (branch imx_2.6.35_maintain at git://git.freescale.com/imx/linux-2.6-imx.git). The kernel running on the machine in question is based on 3.10 with an impressive (that's negative) patch stack on top. I think patches -2 and -3 are not relevant for my setup because the two saif clocks are configured identically if I'm not mistaken. However implementing the soft reset as is done in patch -1 doesn't work here. When resetting saif1 (that is used for recording with saif0 being it's master) the CLKGATE bit gets never set and when breaking out of the loop the noise issue happend 5 out of 5 test runs in the first iteration. Does this ring a bell for you? Even if there is something bad hidden in the patch stack (or 3.10?) a reset should never fail, right? Best regards Uwe -- Pengutronix e.K. | Uwe Kleine-K?nig | Industrial Linux Solutions | http://www.pengutronix.de/ | ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-01-27 10:56 noise issues when recording sound on i.MX28 Uwe Kleine-König @ 2016-01-27 11:53 ` Fabio Estevam 2016-01-27 14:43 ` Uwe Kleine-König 0 siblings, 1 reply; 13+ messages in thread From: Fabio Estevam @ 2016-01-27 11:53 UTC (permalink / raw) To: linux-arm-kernel Hi Uwe, On Wed, Jan 27, 2016 at 8:56 AM, Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de> wrote: > Hello, > > I currently work with an i.MX28 based machine and occasionally when > recording sound with arecord but no microphone connected the result > contains much noise. > > I found commits > > 90ce77d4468e ENGR00285446-3 [MX28] SAIF: Bit Shift in SAIF RX Data > 1ea685a12f19 ENGR00285446-2 [MX28] SAIF: Bit Shift in SAIF RX Data > 1ca899221d8b ENGR00285446-1 [MX28] SAIF: Bit Shift in SAIF RX Data > > in the Freescale vendor kernel (branch imx_2.6.35_maintain at > git://git.freescale.com/imx/linux-2.6-imx.git). The kernel running on > the machine in question is based on 3.10 with an impressive (that's > negative) patch stack on top. I think patches -2 and -3 are not relevant > for my setup because the two saif clocks are configured identically if > I'm not mistaken. However implementing the soft reset as is done in Please make sure that the two saif clocks are configured identically. This was the most important part of the fix when we worked on this problem on 2.6.35. Unfortunately I have never had a chance to upstream this fix. Regards, Fabio Estevam ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-01-27 11:53 ` Fabio Estevam @ 2016-01-27 14:43 ` Uwe Kleine-König 2016-01-28 6:38 ` Lothar Waßmann 2016-02-02 13:00 ` Fabio Estevam 0 siblings, 2 replies; 13+ messages in thread From: Uwe Kleine-König @ 2016-01-27 14:43 UTC (permalink / raw) To: linux-arm-kernel Hello Fabio, [dropping Jack Lee from Cc: as his address doesn't exist.] On Wed, Jan 27, 2016 at 09:53:18AM -0200, Fabio Estevam wrote: > On Wed, Jan 27, 2016 at 8:56 AM, Uwe Kleine-K?nig > <u.kleine-koenig@pengutronix.de> wrote: > > I currently work with an i.MX28 based machine and occasionally when > > recording sound with arecord but no microphone connected the result > > contains much noise. > > > > I found commits > > > > 90ce77d4468e ENGR00285446-3 [MX28] SAIF: Bit Shift in SAIF RX Data > > 1ea685a12f19 ENGR00285446-2 [MX28] SAIF: Bit Shift in SAIF RX Data > > 1ca899221d8b ENGR00285446-1 [MX28] SAIF: Bit Shift in SAIF RX Data > > > > in the Freescale vendor kernel (branch imx_2.6.35_maintain at > > git://git.freescale.com/imx/linux-2.6-imx.git). The kernel running on > > the machine in question is based on 3.10 with an impressive (that's > > negative) patch stack on top. I think patches -2 and -3 are not relevant > > for my setup because the two saif clocks are configured identically if > > I'm not mistaken. However implementing the soft reset as is done in > > Please make sure that the two saif clocks are configured identically. I think I have that. According to $debugfs/clk the two saif clocks have the same parent and frequency. > This was the most important part of the fix when we worked on this > problem on 2.6.35. So you didn't hit the problem that resetting a saif didn't work, right? Do you have a few more technical details here? A usecase of my machine is recording and playback at the same time (an intercom system). So if it's necessary to reset the sending side to start receiving that's unfortunate. Best regards Uwe -- Pengutronix e.K. | Uwe Kleine-K?nig | Industrial Linux Solutions | http://www.pengutronix.de/ | ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-01-27 14:43 ` Uwe Kleine-König @ 2016-01-28 6:38 ` Lothar Waßmann 2016-02-02 13:00 ` Fabio Estevam 1 sibling, 0 replies; 13+ messages in thread From: Lothar Waßmann @ 2016-01-28 6:38 UTC (permalink / raw) To: linux-arm-kernel Hi, On Wed, 27 Jan 2016 15:43:40 +0100 Uwe Kleine-K?nig wrote: > Hello Fabio, > > [dropping Jack Lee from Cc: as his address doesn't exist.] > > On Wed, Jan 27, 2016 at 09:53:18AM -0200, Fabio Estevam wrote: > > On Wed, Jan 27, 2016 at 8:56 AM, Uwe Kleine-K?nig > > <u.kleine-koenig@pengutronix.de> wrote: > > > I currently work with an i.MX28 based machine and occasionally when > > > recording sound with arecord but no microphone connected the result > > > contains much noise. > > > > > > I found commits > > > > > > 90ce77d4468e ENGR00285446-3 [MX28] SAIF: Bit Shift in SAIF RX Data > > > 1ea685a12f19 ENGR00285446-2 [MX28] SAIF: Bit Shift in SAIF RX Data > > > 1ca899221d8b ENGR00285446-1 [MX28] SAIF: Bit Shift in SAIF RX Data > > > > > > in the Freescale vendor kernel (branch imx_2.6.35_maintain at > > > git://git.freescale.com/imx/linux-2.6-imx.git). The kernel running on > > > the machine in question is based on 3.10 with an impressive (that's > > > negative) patch stack on top. I think patches -2 and -3 are not relevant > > > for my setup because the two saif clocks are configured identically if > > > I'm not mistaken. However implementing the soft reset as is done in > > > > Please make sure that the two saif clocks are configured identically. > > I think I have that. According to $debugfs/clk the two saif clocks have > the same parent and frequency. > > > This was the most important part of the fix when we worked on this > > problem on 2.6.35. > > So you didn't hit the problem that resetting a saif didn't work, right? > According to the i.MX28 Ref. Manual CLKGATE of the SAIF block does NOT automatically assert upon asserting SFTRST: |35.3.1 SAIF Control Register (HW_SAIF_CTRL) [...] |HW_SAIF_CTRL field descriptions | Field Description | 31 | Setting this bit to 1 forces a reset to the entire block. SFTRST has no effect on CLKGATE. Also, the |SFTRST | SFTRST bit may be written when CLKGATE=1. This bit must be cleared to 0 for normal operation. So the standard stmp_reset_block() function won't work for the SAIF block. BTW: I'm currently investigating the same problem... Lothar Wa?mann ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-01-27 14:43 ` Uwe Kleine-König 2016-01-28 6:38 ` Lothar Waßmann @ 2016-02-02 13:00 ` Fabio Estevam 2016-02-10 15:03 ` Uwe Kleine-König 1 sibling, 1 reply; 13+ messages in thread From: Fabio Estevam @ 2016-02-02 13:00 UTC (permalink / raw) To: linux-arm-kernel Hi Uwe, On Wed, Jan 27, 2016 at 12:43 PM, Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de> wrote: > So you didn't hit the problem that resetting a saif didn't work, right? No, we haven't seen this reset issue. > Do you have a few more technical details here? A usecase of my machine Let me find out as it has been several years I worked on this problem. Will let you know. Regards, Fabio Estevam ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-02 13:00 ` Fabio Estevam @ 2016-02-10 15:03 ` Uwe Kleine-König 2016-02-10 20:10 ` Fabio Estevam 0 siblings, 1 reply; 13+ messages in thread From: Uwe Kleine-König @ 2016-02-10 15:03 UTC (permalink / raw) To: linux-arm-kernel Hello Fabio, On Tue, Feb 02, 2016 at 11:00:33AM -0200, Fabio Estevam wrote: > On Wed, Jan 27, 2016 at 12:43 PM, Uwe Kleine-K?nig > <u.kleine-koenig@pengutronix.de> wrote: > > > So you didn't hit the problem that resetting a saif didn't work, right? > > No, we haven't seen this reset issue. > > > Do you have a few more technical details here? A usecase of my machine > > Let me find out as it has been several years I worked on this problem. > Will let you know. Anything new on your end? I know from Lothar that he invested quite some time to debug his hardware setup, tested the codec instead of the i.MX28 as clock provider and several more things. I also spend some time on this issue and we both hope that Freescale/NXP can help in better understanding the issue and hopefully can come up with a recipe to make concurrent recording and playback on i.MX28 work. Lothar is in contact with Freescale, too, but up to now with little success. The company I'm working for at the moment would also welcome a solution here. Best regards Uwe -- Pengutronix e.K. | Uwe Kleine-K?nig | Industrial Linux Solutions | http://www.pengutronix.de/ | ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-10 15:03 ` Uwe Kleine-König @ 2016-02-10 20:10 ` Fabio Estevam 2016-02-11 8:40 ` Uwe Kleine-König 0 siblings, 1 reply; 13+ messages in thread From: Fabio Estevam @ 2016-02-10 20:10 UTC (permalink / raw) To: linux-arm-kernel Hi Uwe, On Wed, Feb 10, 2016 at 1:03 PM, Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de> wrote: > Hello Fabio, > > On Tue, Feb 02, 2016 at 11:00:33AM -0200, Fabio Estevam wrote: >> On Wed, Jan 27, 2016 at 12:43 PM, Uwe Kleine-K?nig >> <u.kleine-koenig@pengutronix.de> wrote: >> >> > So you didn't hit the problem that resetting a saif didn't work, right? >> >> No, we haven't seen this reset issue. >> >> > Do you have a few more technical details here? A usecase of my machine >> >> Let me find out as it has been several years I worked on this problem. >> Will let you know. > > Anything new on your end? I know from Lothar that he invested quite some > time to debug his hardware setup, tested the codec instead of the i.MX28 > as clock provider and several more things. I also spend some time on > this issue and we both hope that Freescale/NXP can help in better > understanding the issue and hopefully can come up with a recipe to make > concurrent recording and playback on i.MX28 work. > > Lothar is in contact with Freescale, too, but up to now with little > success. The company I'm working for at the moment would also welcome a > solution here. Sorry for the delay. I have been on holidays. Can you confirm you have both SAIF interfaces being clocked from ref_pll? ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-10 20:10 ` Fabio Estevam @ 2016-02-11 8:40 ` Uwe Kleine-König 2016-02-11 14:25 ` Lothar Waßmann 2016-02-16 15:01 ` Fabio Estevam 0 siblings, 2 replies; 13+ messages in thread From: Uwe Kleine-König @ 2016-02-11 8:40 UTC (permalink / raw) To: linux-arm-kernel On Wed, Feb 10, 2016 at 06:10:18PM -0200, Fabio Estevam wrote: > Hi Uwe, > > On Wed, Feb 10, 2016 at 1:03 PM, Uwe Kleine-K?nig > <u.kleine-koenig@pengutronix.de> wrote: > > Hello Fabio, > > > > On Tue, Feb 02, 2016 at 11:00:33AM -0200, Fabio Estevam wrote: > >> On Wed, Jan 27, 2016 at 12:43 PM, Uwe Kleine-K?nig > >> <u.kleine-koenig@pengutronix.de> wrote: > >> > >> > So you didn't hit the problem that resetting a saif didn't work, right? > >> > >> No, we haven't seen this reset issue. > >> > >> > Do you have a few more technical details here? A usecase of my machine > >> > >> Let me find out as it has been several years I worked on this problem. > >> Will let you know. > > > > Anything new on your end? I know from Lothar that he invested quite some > > time to debug his hardware setup, tested the codec instead of the i.MX28 > > as clock provider and several more things. I also spend some time on > > this issue and we both hope that Freescale/NXP can help in better > > understanding the issue and hopefully can come up with a recipe to make > > concurrent recording and playback on i.MX28 work. > > > > Lothar is in contact with Freescale, too, but up to now with little > > success. The company I'm working for at the moment would also welcome a > > solution here. > > Sorry for the delay. I have been on holidays. I wonder who granted you to go on holidays given that we have problems :-) > Can you confirm you have both SAIF interfaces being clocked from ref_pll? root at hostname:/sys/kernel/debug/clk find -name saif?_sel ./ref_xtal/pll0/saif0_sel ./ref_xtal/pll0/saif1_sel root at hostname:/sys/kernel/debug/clk cat ./ref_xtal/pll0/saif?_sel/clk_rate 480000000 480000000 This is based on 3.10 (but I hope to update soon to 4.4ish). I know that Lothar uses 4.4. Also Lothar noticed that disabling one of these saif_sel clocks doesn't make an impression on the interfaces, they just continue to run. Maybe a clock is routed in a wrong way? Best regards Uwe -- Pengutronix e.K. | Uwe Kleine-K?nig | Industrial Linux Solutions | http://www.pengutronix.de/ | ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-11 8:40 ` Uwe Kleine-König @ 2016-02-11 14:25 ` Lothar Waßmann 2016-02-16 15:01 ` Fabio Estevam 1 sibling, 0 replies; 13+ messages in thread From: Lothar Waßmann @ 2016-02-11 14:25 UTC (permalink / raw) To: linux-arm-kernel Hi, On Thu, 11 Feb 2016 09:40:29 +0100 Uwe Kleine-K?nig wrote: > On Wed, Feb 10, 2016 at 06:10:18PM -0200, Fabio Estevam wrote: > > Hi Uwe, > > > > On Wed, Feb 10, 2016 at 1:03 PM, Uwe Kleine-K?nig > > <u.kleine-koenig@pengutronix.de> wrote: > > > Hello Fabio, > > > > > > On Tue, Feb 02, 2016 at 11:00:33AM -0200, Fabio Estevam wrote: > > >> On Wed, Jan 27, 2016 at 12:43 PM, Uwe Kleine-K?nig > > >> <u.kleine-koenig@pengutronix.de> wrote: > > >> > > >> > So you didn't hit the problem that resetting a saif didn't work, right? > > >> > > >> No, we haven't seen this reset issue. > > >> > > >> > Do you have a few more technical details here? A usecase of my machine > > >> > > >> Let me find out as it has been several years I worked on this problem. > > >> Will let you know. > > > > > > Anything new on your end? I know from Lothar that he invested quite some > > > time to debug his hardware setup, tested the codec instead of the i.MX28 > > > as clock provider and several more things. I also spend some time on > > > this issue and we both hope that Freescale/NXP can help in better > > > understanding the issue and hopefully can come up with a recipe to make > > > concurrent recording and playback on i.MX28 work. > > > > > > Lothar is in contact with Freescale, too, but up to now with little > > > success. The company I'm working for at the moment would also welcome a > > > solution here. > > > > Sorry for the delay. I have been on holidays. > > I wonder who granted you to go on holidays given that we have > problems :-) > > > Can you confirm you have both SAIF interfaces being clocked from ref_pll? > > root at hostname:/sys/kernel/debug/clk find -name saif?_sel > ./ref_xtal/pll0/saif0_sel > ./ref_xtal/pll0/saif1_sel > > root at hostname:/sys/kernel/debug/clk cat ./ref_xtal/pll0/saif?_sel/clk_rate > 480000000 > 480000000 > > This is based on 3.10 (but I hope to update soon to 4.4ish). I know that > Lothar uses 4.4. Also Lothar noticed that disabling one of these > saif_sel clocks doesn't make an impression on the interfaces, they just > continue to run. Maybe a clock is routed in a wrong way? > That's obviously some misunderstanding. What I observed is, that by selectively disabling and re-enabling one of the SAIF clocks in any way (toggling the CLKGATE bit in the CLKCTRL_SAIF# register, toggling the RUN bit in the SAIF_CTRL register or even toggling the SLAVE_MODE bit in the SAIF_CTRL register) the SAIF interfaces get out of sync (if they happened to be in sync in the first place) with no way to deterministically get them in sync again. Since the SAIF0 clock is routed to the codec, sampling the data on the SAIF1 interface is not in sync with the data sent by the codec. I also tried tying the BITCLK resp. LRCLK of both SAIFs together and selecting the SAIF1 inputs as source via the SAIF_CLKMUX_SEL field in the DIGCTL_CTRL register. Even with this setup there is no way to get consistent data sampled on the SAIF1 interface. Lothar Wa?mann ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-11 8:40 ` Uwe Kleine-König 2016-02-11 14:25 ` Lothar Waßmann @ 2016-02-16 15:01 ` Fabio Estevam 2016-02-16 19:35 ` Uwe Kleine-König 2016-02-17 7:55 ` Lothar Waßmann 1 sibling, 2 replies; 13+ messages in thread From: Fabio Estevam @ 2016-02-16 15:01 UTC (permalink / raw) To: linux-arm-kernel On Thu, Feb 11, 2016 at 6:40 AM, Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de> wrote: > root at hostname:/sys/kernel/debug/clk find -name saif?_sel > ./ref_xtal/pll0/saif0_sel > ./ref_xtal/pll0/saif1_sel > > root at hostname:/sys/kernel/debug/clk cat ./ref_xtal/pll0/saif?_sel/clk_rate > 480000000 > 480000000 Ok, what about the SAIF0 and SAIF1 sampling rates: are they always the same? ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-16 15:01 ` Fabio Estevam @ 2016-02-16 19:35 ` Uwe Kleine-König 2016-02-17 7:55 ` Lothar Waßmann 1 sibling, 0 replies; 13+ messages in thread From: Uwe Kleine-König @ 2016-02-16 19:35 UTC (permalink / raw) To: linux-arm-kernel Hello Fabio, On Tue, Feb 16, 2016 at 01:01:19PM -0200, Fabio Estevam wrote: > On Thu, Feb 11, 2016 at 6:40 AM, Uwe Kleine-K?nig > <u.kleine-koenig@pengutronix.de> wrote: > > > root at hostname:/sys/kernel/debug/clk find -name saif?_sel > > ./ref_xtal/pll0/saif0_sel > > ./ref_xtal/pll0/saif1_sel > > > > root at hostname:/sys/kernel/debug/clk cat ./ref_xtal/pll0/saif?_sel/clk_rate > > 480000000 > > 480000000 > > Ok, what about the SAIF0 and SAIF1 sampling rates: are they always the same? Lothar found a recipe to reproduce. I didn't try, but it's something along the lines of (Lothar please correct if I'm wrong): - start a recording on one SAIF - start playback on the other SAIF - using direct access to the registers stop and restart the playback SAIF via the RUN bit. The program I have doesn't provide a reliable reproduction recipe, it runs in a loop and sometimes need several (>1000) iterations to trigger. In my setup the sampling rates don't change. The values of BITCLK_MULT_RATE differ however in my setup, I don't see why though. Best regards Uwe -- Pengutronix e.K. | Uwe Kleine-K?nig | Industrial Linux Solutions | http://www.pengutronix.de/ | ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-16 15:01 ` Fabio Estevam 2016-02-16 19:35 ` Uwe Kleine-König @ 2016-02-17 7:55 ` Lothar Waßmann 2017-01-10 16:17 ` Jörg Krause 1 sibling, 1 reply; 13+ messages in thread From: Lothar Waßmann @ 2016-02-17 7:55 UTC (permalink / raw) To: linux-arm-kernel Hi, On Tue, 16 Feb 2016 13:01:19 -0200 Fabio Estevam wrote: > On Thu, Feb 11, 2016 at 6:40 AM, Uwe Kleine-K?nig > <u.kleine-koenig@pengutronix.de> wrote: > > > root at hostname:/sys/kernel/debug/clk find -name saif?_sel > > ./ref_xtal/pll0/saif0_sel > > ./ref_xtal/pll0/saif1_sel > > > > root at hostname:/sys/kernel/debug/clk cat ./ref_xtal/pll0/saif?_sel/clk_rate > > 480000000 > > 480000000 > > Ok, what about the SAIF0 and SAIF1 sampling rates: are they always the same? > Yes, of course. The driver wouldn't allow using both interfaces with different sample rates any way. Lothar Wa?mann ^ permalink raw reply [flat|nested] 13+ messages in thread
* noise issues when recording sound on i.MX28 2016-02-17 7:55 ` Lothar Waßmann @ 2017-01-10 16:17 ` Jörg Krause 0 siblings, 0 replies; 13+ messages in thread From: Jörg Krause @ 2017-01-10 16:17 UTC (permalink / raw) To: linux-arm-kernel Hi all, On Wed, 2016-02-17 at 08:55 +0100, Lothar Wa?mann wrote: > Hi, > > On Tue, 16 Feb 2016 13:01:19 -0200 Fabio Estevam wrote: > > On Thu, Feb 11, 2016 at 6:40 AM, Uwe Kleine-K?nig > > <u.kleine-koenig@pengutronix.de> wrote: > > > > > root at hostname:/sys/kernel/debug/clk find -name saif?_sel > > > ./ref_xtal/pll0/saif0_sel > > > ./ref_xtal/pll0/saif1_sel > > > > > > root at hostname:/sys/kernel/debug/clk cat > > > ./ref_xtal/pll0/saif?_sel/clk_rate > > > 480000000 > > > 480000000 > > > > Ok, what about the SAIF0 and SAIF1 sampling rates: are they always > > the same? > > > > Yes, of course. The driver wouldn't allow using both interfaces with > different sample rates any way. Are there any updates about this issue? I am facing the same problem when using SAIF1 for capture and SAIF0 for playback. SAIF1 uses SAIF0 as the master and I do set MXS_DIGCTL_SAIF_CLKMUX_EXTMSTR0 when bringing up the board. However, whenever I record with $ arecord -D hw:0,1 -t wav -c 2 -r 48000 -f S16_LE -d 5 /tmp/out.wav and playback afterwards: $ aplay -D hw:0,0 /tmp/out.wav I only get noise. Note, that I am using the Audio Codec Board from Mikroelectronika, but replaced the on-board 12.288MHz quarz with the MCLK line from SAIF0. Best regards, J?rg Krause ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2017-01-10 16:17 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-01-27 10:56 noise issues when recording sound on i.MX28 Uwe Kleine-König 2016-01-27 11:53 ` Fabio Estevam 2016-01-27 14:43 ` Uwe Kleine-König 2016-01-28 6:38 ` Lothar Waßmann 2016-02-02 13:00 ` Fabio Estevam 2016-02-10 15:03 ` Uwe Kleine-König 2016-02-10 20:10 ` Fabio Estevam 2016-02-11 8:40 ` Uwe Kleine-König 2016-02-11 14:25 ` Lothar Waßmann 2016-02-16 15:01 ` Fabio Estevam 2016-02-16 19:35 ` Uwe Kleine-König 2016-02-17 7:55 ` Lothar Waßmann 2017-01-10 16:17 ` Jörg Krause
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.