* Samsung 840 EVO
@ 2015-05-04 12:38 Günter Waller
2015-05-04 15:28 ` Martin K. Petersen
2015-05-04 15:45 ` Sven Köhler
0 siblings, 2 replies; 11+ messages in thread
From: Günter Waller @ 2015-05-04 12:38 UTC (permalink / raw)
To: Tejun Heo; +Cc: linux-ide
Dear Tejun,
the subject Samsung SSD is having performance issues with "old data"
which Samsung now addresses with a new firmware. As often is the case,
Linux users find it a bit more difficult to apply that fix. However, I
also have a Windows system, so it should be no big deal for me.
But there is also a big concern for Linux users which this FW version
seems to introduce. The link below points to a warning (backed by
others, elsewhere on the web) that fstrim in combination with this FW
will wreck your drive.
http://www.overclock.net/t/1507897/samsung-840-evo-read-speed-drops-on-old-written-data-in-the-drive/2640%23post_23827674
(search for fstrim)
I raised this in a discussion thread of the Ubuntu MATE community and
was pointed to the SSD blacklist that is part of the module
libata-core.c which names you as the maintainer. I wonder if you could
take a look at this issue - maybe you want to add the device to the
blacklist thereafter if need be.
If you find the time and find this worthwhile to investigate I would
appreciate a brief notification of the outcome. For now I am holding
back the FW update.
Best regards
Guenter Waller
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 12:38 Samsung 840 EVO Günter Waller
@ 2015-05-04 15:28 ` Martin K. Petersen
2015-05-04 15:49 ` Sven Köhler
2015-05-04 16:19 ` Günter Waller
2015-05-04 15:45 ` Sven Köhler
1 sibling, 2 replies; 11+ messages in thread
From: Martin K. Petersen @ 2015-05-04 15:28 UTC (permalink / raw)
To: Günter Waller; +Cc: Tejun Heo, linux-ide
>>>>> "Günter" == Günter Waller <g.wal@web.de> writes:
Günter> the subject Samsung SSD is having performance issues with "old
Günter> data" which Samsung now addresses with a new firmware.
I wonder if they enabled the same botched queued discard that they did
in 850 Pro?
If someone has an 840 Evo with the new firmware it would be helpful for
us to receive the output of:
# hdparm -I /dev/sdN
# smartctl -l gplog,0x13 /dev/sdN
I think I might have an 840 in the lab. Will head down and update it
later this week unless somebody beats me to providing the information
above.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 12:38 Samsung 840 EVO Günter Waller
2015-05-04 15:28 ` Martin K. Petersen
@ 2015-05-04 15:45 ` Sven Köhler
1 sibling, 0 replies; 11+ messages in thread
From: Sven Köhler @ 2015-05-04 15:45 UTC (permalink / raw)
To: linux-ide
Am 04.05.2015 um 15:38 schrieb Günter Waller:
> But there is also a big concern for Linux users which this FW version
> seems to introduce. The link below points to a warning (backed by
> others, elsewhere on the web) that fstrim in combination with this FW
> will wreck your drive.
> http://www.overclock.net/t/1507897/samsung-840-evo-read-speed-drops-on-old-written-data-in-the-drive/2640%23post_23827674
> (search for fstrim)
> I raised this in a discussion thread of the Ubuntu MATE community and
> was pointed to the SSD blacklist that is part of the module
> libata-core.c which names you as the maintainer. I wonder if you could
> take a look at this issue - maybe you want to add the device to the
> blacklist thereafter if need be.
> If you find the time and find this worthwhile to investigate I would
> appreciate a brief notification of the outcome. For now I am holding
> back the FW update.
Here is a discussion of the issue and a patch:
https://bugs.launchpad.net/ubuntu/+source/fstrim/+bug/1449005
Apparently, TRIM in combination with NCQ is broken with the new
firmware. Having the discard mount option enabled didn't wreck my drive,
but it kept my machine from booting. I had to disable that option and
the patch given at the URL above seems to disable NCQ specifically for
TRIM only. So the performance hit might be minimal.
The patch should really be applied asap and it should be backported to
all stable branched as far as possible.
Regards,
Sven
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 15:28 ` Martin K. Petersen
@ 2015-05-04 15:49 ` Sven Köhler
2015-05-04 16:21 ` Martin K. Petersen
2015-05-04 16:19 ` Günter Waller
1 sibling, 1 reply; 11+ messages in thread
From: Sven Köhler @ 2015-05-04 15:49 UTC (permalink / raw)
To: linux-ide
[-- Attachment #1: Type: text/plain, Size: 369 bytes --]
Am 04.05.2015 um 18:28 schrieb Martin K. Petersen:
> If someone has an 840 Evo with the new firmware it would be helpful for
> us to receive the output of:
>
> # hdparm -I /dev/sdN
> # smartctl -l gplog,0x13 /dev/sdN
Find the output attached. But I'm not sure the smartctl is helpful,
since my machine has been running with discard disabled for about half a
day now.
[-- Attachment #2: hdparm.txt --]
[-- Type: text/plain, Size: 3236 bytes --]
/dev/sda:
ATA device, with non-removable media
Model Number: Samsung SSD 840 EVO 250GB
Serial Number: deleted
Firmware Revision: EXT0DB6Q
Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x0039)
Supported: 9 8 7 6 5
Likely used: 9
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 488397168
Logical Sector size: 512 bytes
Physical Sector size: 512 bytes
Logical Sector-0 offset: 0 bytes
device size with M = 1024*1024: 238475 MBytes
device size with M = 1000*1000: 250059 MBytes (250 GB)
cache/buffer size = unknown
Nominal Media Rotation Rate: Solid State Device
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 1 Current = 1
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* WRITE_{DMA|MULTIPLE}_FUA_EXT
* 64-bit World wide name
Write-Read-Verify feature set
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Phy event counters
* unknown 76[15]
* DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Asynchronous notification (eg. media change)
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT LBA Segment Access (AC2)
* SCT Error Recovery Control (AC3)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
* reserved 69[4]
* DOWNLOAD MICROCODE DMA command
* SET MAX SETPASSWORD/UNLOCK DMA commands
* WRITE BUFFER DMA command
* READ BUFFER DMA command
* Data Set Management TRIM supported (limit 8 blocks)
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
supported: enhanced erase
2min for SECURITY ERASE UNIT. 8min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50025388a05456c8
NAA : 5
IEEE OUI : 002538
Unique ID : 8a05456c8
Checksum: correct
[-- Attachment #3: smart.txt --]
[-- Type: text/plain, Size: 2650 bytes --]
smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.6] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
General Purpose Log 0x13 [SATA NCQ Send and Receive log], Page 0-0 (of 1)
0000000: 01 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 |................|
0000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
0000190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 15:28 ` Martin K. Petersen
2015-05-04 15:49 ` Sven Köhler
@ 2015-05-04 16:19 ` Günter Waller
2015-05-04 16:24 ` Martin K. Petersen
1 sibling, 1 reply; 11+ messages in thread
From: Günter Waller @ 2015-05-04 16:19 UTC (permalink / raw)
To: Martin K. Petersen; +Cc: Tejun Heo, linux-ide
[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]
Hi Martin,
here you are - the second command, however, did not seem to produce
anything useful.
But beware: this is the old firmware, EXT0BB6Q, i.e. the one with the
performance issues (not seen on my machine yet), not the one that causes
the problems with fstrim - named EXT0DB6Q.
As I mentioned I shy away from upgrading at this point, not knowing
whether my Linux system runs fstrim somewhere behind my back. I am not
that deeply involved in the inner workings of the kernel as you guys.
This is my bread and butter machine, I'd rather not expose it to risk if
I can help it.
Cheers
Günter
Am 04.05.2015 um 17:28 schrieb Martin K. Petersen:
>>>>>> "Günter" == Günter Waller <g.wal@web.de> writes:
>
> Günter> the subject Samsung SSD is having performance issues with "old
> Günter> data" which Samsung now addresses with a new firmware.
>
> I wonder if they enabled the same botched queued discard that they did
> in 850 Pro?
>
> If someone has an 840 Evo with the new firmware it would be helpful for
> us to receive the output of:
>
> # hdparm -I /dev/sdN
> # smartctl -l gplog,0x13 /dev/sdN
>
> I think I might have an 840 in the lab. Will head down and update it
> later this week unless somebody beats me to providing the information
> above.
>
[-- Attachment #2: 840evo.txt --]
[-- Type: text/plain, Size: 3616 bytes --]
hdparm -I /dev/sda
/dev/sda:
ATA device, with non-removable media
Model Number: Samsung SSD 840 EVO 250GB
Serial Number: S1DBNSBF767383T
Firmware Revision: EXT0BB6Q
Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x0039)
Supported: 9 8 7 6 5
Likely used: 9
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 488397168
Logical Sector size: 512 bytes
Physical Sector size: 512 bytes
Logical Sector-0 offset: 0 bytes
device size with M = 1024*1024: 238475 MBytes
device size with M = 1000*1000: 250059 MBytes (250 GB)
cache/buffer size = unknown
Nominal Media Rotation Rate: Solid State Device
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 1 Current = 1
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* WRITE_{DMA|MULTIPLE}_FUA_EXT
* 64-bit World wide name
Write-Read-Verify feature set
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Phy event counters
* READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
* DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Asynchronous notification (eg. media change)
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT Write Same (AC2)
* SCT Error Recovery Control (AC3)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
* reserved 69[4]
* DOWNLOAD MICROCODE DMA command
* SET MAX SETPASSWORD/UNLOCK DMA commands
* WRITE BUFFER DMA command
* READ BUFFER DMA command
* Data Set Management TRIM supported (limit 8 blocks)
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
supported: enhanced erase
2min for SECURITY ERASE UNIT. 8min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50025388a05d2afa
NAA : 5
IEEE OUI : 002538
Unique ID : 8a05d2afa
Checksum: correct
*************************************************************
smartctl -l gplog,0x13 /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-51-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
General Purpose Log 0x13 does not exist (override with '-T permissive' option)
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 15:49 ` Sven Köhler
@ 2015-05-04 16:21 ` Martin K. Petersen
2015-05-04 16:27 ` Sven Köhler
0 siblings, 1 reply; 11+ messages in thread
From: Martin K. Petersen @ 2015-05-04 16:21 UTC (permalink / raw)
To: Sven Köhler; +Cc: linux-ide
>>>>> "Sven" == Sven Köhler <sven.koehler@gmail.com> writes:
>> # hdparm -I /dev/sdN # smartctl -l gplog,0x13 /dev/sdN
Sven> I'm not sure the smartctl is helpful, since my machine has been
Sven> running with discard disabled for about half a day now.
It is helpful. It shows us that Samsung has indeed enabled queued TRIM
functionality in this firmware update.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 16:19 ` Günter Waller
@ 2015-05-04 16:24 ` Martin K. Petersen
2015-05-04 16:37 ` Günter Waller
0 siblings, 1 reply; 11+ messages in thread
From: Martin K. Petersen @ 2015-05-04 16:24 UTC (permalink / raw)
To: Günter Waller; +Cc: Martin K. Petersen, Tejun Heo, linux-ide
>>>>> "Günter" == Günter Waller <g.wal@web.de> writes:
Günter> the second command, however, did not seem to produce anything
Günter> useful. But beware: this is the old firmware, EXT0BB6Q,
Yes, this confirms that the old firmware did not support queued
TRIM. Whereas the new one does.
I'll try to reach out to my contacts at Samsung again so we can get this
resolved.
Until then, I have sent out a patch that blacklists queued TRIM for all
Samsung 800-series drives. It appears to be generic to their
implementation and not tied to a particular drive model.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 16:21 ` Martin K. Petersen
@ 2015-05-04 16:27 ` Sven Köhler
2015-05-04 16:35 ` Martin K. Petersen
0 siblings, 1 reply; 11+ messages in thread
From: Sven Köhler @ 2015-05-04 16:27 UTC (permalink / raw)
To: Martin K. Petersen; +Cc: linux-ide
Am 04.05.2015 um 19:21 schrieb Martin K. Petersen:
>>>>>> "Sven" == Sven Köhler <sven.koehler@gmail.com> writes:
>
>>> # hdparm -I /dev/sdN # smartctl -l gplog,0x13 /dev/sdN
>
> Sven> I'm not sure the smartctl is helpful, since my machine has been
> Sven> running with discard disabled for about half a day now.
>
> It is helpful. It shows us that Samsung has indeed enabled queued TRIM
> functionality in this firmware update.
Are you saying that previous firmwares didn't claim that the drive
supports queued TRIM. But the new firmware does - even though the
feature appears to be broken?
Regards,
Sven
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 16:27 ` Sven Köhler
@ 2015-05-04 16:35 ` Martin K. Petersen
0 siblings, 0 replies; 11+ messages in thread
From: Martin K. Petersen @ 2015-05-04 16:35 UTC (permalink / raw)
To: Sven Köhler; +Cc: Martin K. Petersen, linux-ide
>>>>> "Sven" == Sven Köhler <sven.koehler@gmail.com> writes:
>> It is helpful. It shows us that Samsung has indeed enabled queued
>> TRIM functionality in this firmware update.
Sven> Are you saying that previous firmwares didn't claim that the drive
Sven> supports queued TRIM. But the new firmware does - even though the
Sven> feature appears to be broken?
Correct. The old firmware releases for this drive did not advertise
support for queued TRIM whereas the new one does. We saw exactly the
same issue with a firmware update for 850 Pro.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 16:24 ` Martin K. Petersen
@ 2015-05-04 16:37 ` Günter Waller
2015-05-04 16:47 ` Martin K. Petersen
0 siblings, 1 reply; 11+ messages in thread
From: Günter Waller @ 2015-05-04 16:37 UTC (permalink / raw)
To: Martin K. Petersen; +Cc: Tejun Heo, linux-ide
Martin,
do you mind if I post the essence of your last two mails to my thread in
the Ubuntu MATE forum? So the maintainers of that system can see what is
going on.
Am 04.05.2015 um 18:24 schrieb Martin K. Petersen:
>>>>>> "Günter" == Günter Waller <g.wal@web.de> writes:
>
> Günter> the second command, however, did not seem to produce anything
> Günter> useful. But beware: this is the old firmware, EXT0BB6Q,
>
> Yes, this confirms that the old firmware did not support queued
> TRIM. Whereas the new one does.
>
> I'll try to reach out to my contacts at Samsung again so we can get this
> resolved.
>
> Until then, I have sent out a patch that blacklists queued TRIM for all
> Samsung 800-series drives. It appears to be generic to their
> implementation and not tied to a particular drive model.
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Samsung 840 EVO
2015-05-04 16:37 ` Günter Waller
@ 2015-05-04 16:47 ` Martin K. Petersen
0 siblings, 0 replies; 11+ messages in thread
From: Martin K. Petersen @ 2015-05-04 16:47 UTC (permalink / raw)
To: Günter Waller; +Cc: Martin K. Petersen, Tejun Heo, linux-ide
>>>>> "Günter" == Günter Waller <g.wal@web.de> writes:
Günter> Martin, do you mind if I post the essence of your last two mails
Günter> to my thread in the Ubuntu MATE forum?
Go right ahead.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-05-04 16:47 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-05-04 12:38 Samsung 840 EVO Günter Waller
2015-05-04 15:28 ` Martin K. Petersen
2015-05-04 15:49 ` Sven Köhler
2015-05-04 16:21 ` Martin K. Petersen
2015-05-04 16:27 ` Sven Köhler
2015-05-04 16:35 ` Martin K. Petersen
2015-05-04 16:19 ` Günter Waller
2015-05-04 16:24 ` Martin K. Petersen
2015-05-04 16:37 ` Günter Waller
2015-05-04 16:47 ` Martin K. Petersen
2015-05-04 15:45 ` Sven Köhler
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.