Mmap on SSD (directly mapping the device vs. mapping a file)

Mmap on SSD (directly mapping the device vs. mapping a file)

[Daniel Noack]

Hi, folks!

First of all I have to say sorry that I'm not a subscriber of the list now,
but I will subscribe in the near future (need an extra email account for it
;) ). It would be nice if you could put me on CC for an answer to my
question.

Like you can see on the subject I experimented a little bit with mmap in the
last time. I've written a little B+tree library which uses mmap to store
the tree to a file or the whole device (means it is also possible to map
the hwole device (i.e. /dev/sdb)). I used msync after every successfull
change on the tree. Next thing I did was to use this for a little benchmark
on performance of different storage devices (ramdisk, HDD, and a very fast
flashcard directly atteched to the PCIe bus). I recognized that in allmost
all cases when directly mapping teh device without a filesystem the
file-mapped version was a little bit slower. But when I tried it on an SSD
device the file-mapped version was an order of magnitude faster. I also
tried secure erase and did the benchmarks many times and in many
configurations, but I came to the same results. Can anyone explain me
what's happening here or have I something missed. Is this behavior correct?

Thanks,
Daniel

Mmap on SSD (directly mapping the device vs. mapping a file)

[Daniel Noack]

Hi, folks!

Like you can see on the subject I experimented a little with mmap in
the last time. I've written a little B+tree library which uses mmap to
store the tree to a file or the whole device (means it is also
possible to map the raw device (i.e. /dev/sdb)). I used msync after
every successfull change on the tree. Next thing I did was to use this
for a little benchmark on performance of different storage devices
(ramdisk, HDD, and a very fast flashcard directly atteched to the PCIe
bus). I recognized that in allmost all cases when directly mapping the
device without a filesystem the file-mapped version was a little bit
slower. But when I tried it on an SSD device the file-mapped version
was an order of magnitude faster. I also tried secure erase and did
the benchmarks many times and in many configurations, but I came to
the same results.
Last thing I tried were the different queue schedulers, without any changes.

In one of the posts from january I read that there is a performance
bug when directly reading from the raw SSD device, but I didn't find
any else comment which stated this as true. For the benchmarks I used
a current Ubuntu with a 3.2.16 kernel (from kernel.org). Is this
behavior normal, or did I miss something.