SSD and IO schedulers

SSD and IO schedulers

[Lorenzo Allegrucci]

Hi, I was wondering how IO schedulers such as as-iosched, deadline and
cfq behave on SSD
(that have virtually no seek time), from a theoretical point of view.
How do they affect
performance on these devices?
I heard that the noop scheduler is often chosen by owners of EeePcs
(with a SSD unit).
They report superior performance by using this (quite simple) scheduler.
Are there any scientific benchmarks around?

-- 
Lorenzo

Re: SSD and IO schedulers

[Jens Axboe]

On Fri, Jan 30 2009, Lorenzo Allegrucci wrote:
> Hi, I was wondering how IO schedulers such as as-iosched, deadline and
> cfq behave on SSD
> (that have virtually no seek time), from a theoretical point of view.
> How do they affect
> performance on these devices?
> I heard that the noop scheduler is often chosen by owners of EeePcs
> (with a SSD unit).
> They report superior performance by using this (quite simple) scheduler.
> Are there any scientific benchmarks around?

Just recently the io schedulers started checking for SSD devices, so
today it should not matter performance wise (throughput) whether you use
CFQ or NOOP on eg the eeepc. The io scheduler is still quite important
for providing fair access to the device, especially on the cheaper end
of the SSD segment.

-- 
Jens Axboe

Re: SSD and IO schedulers

[Alan Jenkins]

Jens Axboe wrote:
> On Fri, Jan 30 2009, Lorenzo Allegrucci wrote:
>   
>> Hi, I was wondering how IO schedulers such as as-iosched, deadline and
>> cfq behave on SSD
>> (that have virtually no seek time), from a theoretical point of view.
>> How do they affect
>> performance on these devices?
>> I heard that the noop scheduler is often chosen by owners of EeePcs
>> (with a SSD unit).
>> They report superior performance by using this (quite simple) scheduler.
>> Are there any scientific benchmarks around?
>>     
>
> Just recently the io schedulers started checking for SSD devices, so
> today it should not matter performance wise (throughput) whether you use
> CFQ or NOOP on eg the eeepc.

Although not all SSDs identify themselves, including the one on my
eeepc. So to get the benefit you have to tell the kernel manually. The
ability to do this was merged into mainline yesterday. (/me resolves to
test it).

http://git.kernel.org/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=1308835ffffe6d61ad1f48c5c381c9cc47f683ec

Anecdotally it was worth switching from CFQ to NOOP. CFQ caused
seconds-long hangs (with the SSD light solidly on); with NOOP this
happens far less often. I don't know or care if it halved throughput,
but I can't bear to use a system that hangs for seconds on end :-).

>  The io scheduler is still quite important
> for providing fair access to the device, especially on the cheaper end
> of the SSD segment.
>   

Re: SSD and IO schedulers

[J.A. Magallón]

On Fri, 30 Jan 2009 22:55:47 +0100, Lorenzo Allegrucci <l.allegrucci@gmail.com> wrote:

> Hi, I was wondering how IO schedulers such as as-iosched, deadline and
> cfq behave on SSD
> (that have virtually no seek time), from a theoretical point of view.
> How do they affect
> performance on these devices?
> I heard that the noop scheduler is often chosen by owners of EeePcs
> (with a SSD unit).

I'm beginning to think this is a _very_ bad idea. I have an AspireOne
A110 (512MB, 8GB SSD). This is 'not the best SSD in the world', to say
something. I had been booting with elevator=noop, because of all those
advices found along many blogs.

The fact is that the system behaved well, except when I was doing an
'urpmi --auto-update' (I use Mandriva). The bulk of rpm work rendered the
box unusable. 5-10 seconds stalls and so on.

After reading this, I killed the elevator option from grub, the system uses
the default cfq scheduler, and things are much smoother and the laptop
is perfectly usable while doing an update. I think even rpm itself works
faster.

Perhaps the reason is that, as the SSD is not so good, it behaves more
like a rotational drive ;).

> They report superior performance by using this (quite simple) scheduler.
> Are there any scientific benchmarks around?
> 

A couple questions:
- Apart from noop, which scheduler do you think is the best for SSDs ?
- I'm running 2.6.28.2. Does it have the 'SSD detector' in libata ?
  How can I verify it ?

TIA

-- 
J.A. Magallon <jamagallon()ono!com>     \               Software is like sex:
                                         \         It's better when it's free
Mandriva Linux release 2009.1 (Cooker) for x86_64
Linux 2.6.28.2-desktop-1mnb (gcc 4.3.2 (GCC) #1 Wed Jan

Re: SSD and IO schedulers

[Jan Knutar]

On Wednesday 04 February 2009, J.A. Magallón wrote:

> Perhaps the reason is that, as the SSD is not so good, it behaves
> more like a rotational drive ;).

Do any other SSDs except Intel's exist that DON'T behave more like a 
rotational drive? I am guessing using something like LogFS would give 
the biggest boost on cheap SSDs and all memory cards.

Re: SSD and IO schedulers

[Corrado Zoccolo]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

Hi,
I found that elevator=deadline performs much better than noop for
writes, and almost as well for reads, and started to wonder how a
combination of noop (for read) + deadline (for write) would work in
practice (since deadline still doesn't support the automatic ssd
detection).
I developed a simple hybrid scheduler (see attached), that implements
this idea, and it saved 1 s of my boot time compared to all other
schedulers (cfq, deadline and noop, each loses on some part of the
workload), that is a mixed read write workload, with writes that go
mainly to a very slow device (SDHC card, where I mount /var).

This proof of concept still doesn't support priorities, but I'm
willing to add them at least for read, in which the latency for
sync-reads could be improved.

Corrado

On Sat, Feb 7, 2009 at 6:58 PM, Jan Knutar <jk-lkml@sci.fi> wrote:
> On Wednesday 04 February 2009, J.A. Magallón wrote:
>
>> Perhaps the reason is that, as the SSD is not so good, it behaves
>> more like a rotational drive ;).
>
> Do any other SSDs except Intel's exist that DON'T behave more like a
> rotational drive? I am guessing using something like LogFS would give
> the biggest boost on cheap SSDs and all memory cards.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>



-- 
__________________________________________________________________________

dott. Corrado Zoccolo                          mailto:czoccolo@gmail.com
PhD - Department of Computer Science - University of Pisa, Italy
--------------------------------------------------------------------------
The self-confidence of a warrior is not the self-confidence of the average
man. The average man seeks certainty in the eyes of the onlooker and calls
that self-confidence. The warrior seeks impeccability in his own eyes and
calls that humbleness.
                               Tales of Power - C. Castaneda

[-- Attachment #2: ssd-iosched.c --]
[-- Type: text/x-csrc, Size: 10891 bytes --]

/* -*- mode:C; tab-width:8; c-basic-offset:8 -*-
 *  Ssd i/o scheduler.
 *
 *  Copyright (C) 2002 Jens Axboe <axboe@kernel.dk>
 */
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/blkdev.h>
#include <linux/elevator.h>
#include <linux/bio.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/init.h>
#include <linux/compiler.h>
#include <linux/rbtree.h>

/*
 * See Documentation/block/ssd-iosched.txt
 */
static const int write_expire = 5 * HZ; /* ditto for writes, these limits are SOFT! */
static const int writes_starved = 2;    /* max times reads can starve a write */
static const int fifo_batch = 16;       /* # of sequential requests treated as one
				     by the above parameters. For throughput. */

struct ssd_data {
	/*
	 * run time data
	 */

	/*
	 * requests (ssd_rq s) are present on both sort_list and fifo_list
	 */
	/* for read */
        struct list_head read_queue;

	/* for write */
	struct rb_root sort_list_write;	
	struct list_head fifo_list_write;

	/*
	 * next in sort order. read, write or both are NULL
	 */
	struct request *next_rq_write;

	/* stats */
	unsigned int batching;		/* number of sequential requests made */
	sector_t last_sector;		/* head position */
	unsigned int starved;		/* times reads have starved writes */

	/*
	 * settings that change how the i/o scheduler behaves
	 */
	int fifo_expire_write;
	int fifo_batch;
	int writes_starved;
};

static void ssd_move_write_request(struct ssd_data *, struct request *);

/*
 * get the request after `rq' in sector-sorted order
 */
static inline struct request *
ssd_latter_write_request(struct request *rq)
{
	struct rb_node *node = rb_next(&rq->rb_node);

	if (node)
		return rb_entry_rq(node);

	return NULL;
}

static void
ssd_add_write_rq_rb(struct ssd_data *dd, struct request *rq)
{
	struct rb_root *root = &dd->sort_list_write;
	struct request *__alias;

	while (unlikely(__alias = elv_rb_add(root, rq)))
		ssd_move_write_request(dd, __alias);
}

static inline void
ssd_del_write_rq_rb(struct ssd_data *dd, struct request *rq)
{
	if (dd->next_rq_write == rq)
		dd->next_rq_write = ssd_latter_write_request(rq);

	elv_rb_del(&dd->sort_list_write, rq);
}

/*
 * add rq to rbtree and fifo
 */
static void
ssd_add_request(struct request_queue *q, struct request *rq)
{
	struct ssd_data *dd = q->elevator->elevator_data;
	const int data_dir = rq_data_dir(rq);
	if(data_dir==READ) {
		list_add_tail(&rq->queuelist, &dd->read_queue);
	} else {
		ssd_add_write_rq_rb(dd, rq);

		/*
		 * set expire time and add to fifo list
		 */
		rq_set_fifo_time(rq, jiffies + dd->fifo_expire_write);
		list_add_tail(&rq->queuelist, &dd->fifo_list_write);
	}
}

/*
 * remove rq from rbtree and fifo.
 */
static void ssd_remove_write_request(struct request_queue *q, struct request *rq)
{
	struct ssd_data *dd = q->elevator->elevator_data;

	rq_fifo_clear(rq);
	ssd_del_write_rq_rb(dd, rq);
}

static void ssd_merged_request(struct request_queue *q,
				    struct request *req, int type)
{
	struct ssd_data *dd = q->elevator->elevator_data;

	/*
	 * if the merge was a front merge, we need to reposition request
	 */
	if (type == ELEVATOR_FRONT_MERGE && rq_data_dir(req)==WRITE) {
		elv_rb_del(&dd->sort_list_write, req);
		ssd_add_write_rq_rb(dd, req);
	}
}

static void
ssd_merged_requests(struct request_queue *q, struct request *req,
			 struct request *next)
{
	if(rq_data_dir(req)==READ) {
		list_del_init(&next->queuelist);
	} else {
		/*
		 * if next expires before rq, assign its expire time to rq
		 * and move into next position (next will be deleted) in fifo
		 */
		if (!list_empty(&req->queuelist) && !list_empty(&next->queuelist)) {
			if (time_before(rq_fifo_time(next), rq_fifo_time(req))) {
				list_move(&req->queuelist, &next->queuelist);
				rq_set_fifo_time(req, rq_fifo_time(next));
			}
		}

		/*
		 * kill knowledge of next, this one is a goner
		 */
		ssd_remove_write_request(q, next);
	}
}

/*
 * move request from sort list to dispatch queue.
 */
static inline void
ssd_move_write_to_dispatch(struct ssd_data *dd, struct request *rq)
{
	struct request_queue *q = rq->q;

	ssd_remove_write_request(q, rq);
	elv_dispatch_add_tail(q, rq);
}

/*
 * move an entry to dispatch queue
 */
static void
ssd_move_write_request(struct ssd_data *dd, struct request *rq)
{
	dd->next_rq_write = ssd_latter_write_request(rq);
	dd->last_sector = rq_end_sector(rq);

	/*
	 * take it off the sort and fifo list, move
	 * to dispatch queue
	 */
	ssd_move_write_to_dispatch(dd, rq);
}

/*
 * ssd_check_fifo_write returns 0 if there are no expired requests on the fifo,
 * 1 otherwise. Requires !list_empty(&dd->fifo_list[data_dir])
 */
static inline int ssd_check_fifo_write(struct ssd_data *dd)
{
	struct request *rq = rq_entry_fifo(dd->fifo_list_write.next);

	/*
	 * rq is expired!
	 */
	if (time_after(jiffies, rq_fifo_time(rq)))
		return 1;

	return 0;
}

/*
 * ssd_dispatch_requests selects the best request according to
 * read/write expire, fifo_batch, etc
 */
static int ssd_dispatch_requests(struct request_queue *q, int force)
{
	struct ssd_data *dd = q->elevator->elevator_data;
	const int reads = !list_empty(&dd->read_queue);
	const int writes = !list_empty(&dd->fifo_list_write);
	struct request *rq;

	if (reads) {
		if (writes && (dd->starved++ >= dd->writes_starved))
			goto dispatch_writes;

		rq = list_entry(dd->read_queue.next, struct request, queuelist);
		list_del_init(&rq->queuelist);
		elv_dispatch_sort(q, rq);
		return 1;
	}

	/*
	 * batches are currently reads XOR writes
	 */
	if (dd->next_rq_write)
		rq = dd->next_rq_write;

	if (rq && dd->batching < dd->fifo_batch)
		/* we have a next request are still entitled to batch */
		goto dispatch_request;

	/*
	 * there are either no reads or writes have been starved
	 */

	if (writes) {
dispatch_writes:
		BUG_ON(RB_EMPTY_ROOT(&dd->sort_list_write));
		dd->starved = 0;
		goto dispatch_find_request;
	}
	return 0;

dispatch_find_request:
	/*
	 * we are not running a batch, find best request for WRITE
	 */
	if (ssd_check_fifo_write(dd) || !dd->next_rq_write) {
		/*
		 * A deadline has expired, the last request was in the other
		 * direction, or we have run out of higher-sectored requests.
		 * Start again from the request with the earliest expiry time.
		 */
		rq = rq_entry_fifo(dd->fifo_list_write.next);
	} else {
		/*
		 * The last req was the same dir and we have a next request in
		 * sort order. No expired requests so continue on from here.
		 */
		rq = dd->next_rq_write;
	}

	dd->batching = 0;

dispatch_request:
	/*
	 * rq is the selected appropriate request.
	 */
	dd->batching++;
	ssd_move_write_request(dd, rq);

	return 1;
}

static int ssd_queue_empty(struct request_queue *q)
{
	struct ssd_data *dd = q->elevator->elevator_data;

	return list_empty(&dd->fifo_list_write)
		&& list_empty(&dd->read_queue);
}

static struct request * ssd_former_request(struct request_queue *q, struct request *rq)
{
	struct ssd_data *nd = q->elevator->elevator_data;
	const int data_dir = rq_data_dir(rq);
	if(data_dir==READ) {
		if (rq->queuelist.prev == &nd->read_queue)
			return NULL;
		return list_entry(rq->queuelist.prev, struct request, queuelist);
	} else {
		return elv_rb_former_request(q,rq);
	}
}

static struct request * ssd_latter_request(struct request_queue *q, struct request *rq)
{
	struct ssd_data *dd = q->elevator->elevator_data;
	const int data_dir = rq_data_dir(rq);
	if(data_dir==READ) {
		if (rq->queuelist.next == &dd->read_queue)
			return NULL;
		return list_entry(rq->queuelist.next, struct request, queuelist);
	} else {
		return elv_rb_latter_request(q,rq);
	}
}

static void ssd_exit_queue(struct elevator_queue *e)
{
	struct ssd_data *dd = e->elevator_data;

	BUG_ON(!list_empty(&dd->read_queue));
	BUG_ON(!list_empty(&dd->fifo_list_write));

	kfree(dd);
}

/*
 * initialize elevator private data (ssd_data).
 */
static void *ssd_init_queue(struct request_queue *q)
{
	struct ssd_data *dd;

	dd = kmalloc_node(sizeof(*dd), GFP_KERNEL | __GFP_ZERO, q->node);
	if (!dd)
		return NULL;

	INIT_LIST_HEAD(&dd->read_queue);
	INIT_LIST_HEAD(&dd->fifo_list_write);
	dd->sort_list_write = RB_ROOT;
	dd->fifo_expire_write = write_expire;
	dd->writes_starved = writes_starved;
	dd->fifo_batch = fifo_batch;
	return dd;
}

/*
 * sysfs parts below
 */

static ssize_t
ssd_var_show(int var, char *page)
{
	return sprintf(page, "%d\n", var);
}

static ssize_t
ssd_var_store(int *var, const char *page, size_t count)
{
	char *p = (char *) page;

	*var = simple_strtol(p, &p, 10);
	return count;
}

#define SHOW_FUNCTION(__FUNC, __VAR, __CONV)				\
static ssize_t __FUNC(struct elevator_queue *e, char *page)		\
{									\
	struct ssd_data *dd = e->elevator_data;			\
	int __data = __VAR;						\
	if (__CONV)							\
		__data = jiffies_to_msecs(__data);			\
	return ssd_var_show(__data, (page));			\
}
SHOW_FUNCTION(ssd_write_expire_show, dd->fifo_expire_write, 1);
SHOW_FUNCTION(ssd_writes_starved_show, dd->writes_starved, 0);
SHOW_FUNCTION(ssd_fifo_batch_show, dd->fifo_batch, 0);
#undef SHOW_FUNCTION

#define STORE_FUNCTION(__FUNC, __PTR, MIN, MAX, __CONV)			\
static ssize_t __FUNC(struct elevator_queue *e, const char *page, size_t count)	\
{									\
	struct ssd_data *dd = e->elevator_data;			\
	int __data;							\
	int ret = ssd_var_store(&__data, (page), count);		\
	if (__data < (MIN))						\
		__data = (MIN);						\
	else if (__data > (MAX))					\
		__data = (MAX);						\
	if (__CONV)							\
		*(__PTR) = msecs_to_jiffies(__data);			\
	else								\
		*(__PTR) = __data;					\
	return ret;							\
}
STORE_FUNCTION(ssd_write_expire_store, &dd->fifo_expire_write, 0, INT_MAX, 1);
STORE_FUNCTION(ssd_writes_starved_store, &dd->writes_starved, INT_MIN, INT_MAX, 0);
STORE_FUNCTION(ssd_fifo_batch_store, &dd->fifo_batch, 0, INT_MAX, 0);
#undef STORE_FUNCTION

#define DD_ATTR(name) \
	__ATTR(name, S_IRUGO|S_IWUSR, ssd_##name##_show, \
				      ssd_##name##_store)

static struct elv_fs_entry ssd_attrs[] = {
	DD_ATTR(write_expire),
	DD_ATTR(writes_starved),
	DD_ATTR(fifo_batch),
	__ATTR_NULL
};

static struct elevator_type iosched_ssd = {
	.ops = {
		.elevator_merged_fn =		ssd_merged_request,
		.elevator_merge_req_fn =	ssd_merged_requests,
		.elevator_dispatch_fn =		ssd_dispatch_requests,
		.elevator_add_req_fn =		ssd_add_request,
		.elevator_queue_empty_fn =	ssd_queue_empty,
		.elevator_former_req_fn	=       ssd_former_request,
		.elevator_latter_req_fn	=       ssd_latter_request,
		.elevator_init_fn =		ssd_init_queue,
		.elevator_exit_fn =		ssd_exit_queue,
	},

	.elevator_attrs = ssd_attrs,
	.elevator_name = "ssd",
	.elevator_owner = THIS_MODULE,
};

static int __init ssd_init(void)
{
	elv_register(&iosched_ssd);

	return 0;
}

static void __exit ssd_exit(void)
{
	elv_unregister(&iosched_ssd);
}

module_init(ssd_init);
module_exit(ssd_exit);

MODULE_AUTHOR("Corrado Zoccolo");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("ssd IO scheduler");

Re: SSD and IO schedulers

[Heinz Diehl]

On 08.04.2009, Corrado Zoccolo wrote: 

> I found that elevator=deadline performs much better than noop for
> writes, and almost as well for reads
[....]

The DL elevator has slightly more throughput than cfq and anticipatory,
but is almost unusuable under load.

Running Theodore Ts'os "fsync-tester" while doing Linus' torture test 
"while : ; do time sh -c "dd if=/dev/zero of=bigfile bs=8M count=256 ; sync; rm bigfile"; done"
shows it clearly:

mount: /dev/sda4 on /home type xfs (rw,noatime,logbsize=256k,logbufs=2,nobarrier)
Kernel 2.6.29.1 (vanilla)

with cfq:

htd@liesel:~/!> ./fsync-tester
fsync time: 0.7640
fsync time: 0.6166
fsync time: 1.2830
fsync time: 0.4273
fsync time: 1.1693
fsync time: 1.7466
fsync time: 1.2477
fsync time: 1.9411
fsync time: 1.9636
fsync time: 1.9065
fsync time: 1.1561
fsync time: 1.8267
fsync time: 0.2431
fsync time: 0.2898
fsync time: 0.2394
fsync time: 0.4309
fsync time: 1.5699
fsync time: 0.3742
fsync time: 1.3194
fsync time: 1.9442
fsync time: 1.0899
fsync time: 1.9443
fsync time: 1.0062

with dl:

fsync time: 10.5853
fsync time: 10.3339
fsync time: 5.3374
fsync time: 6.5707
fsync time: 10.6095
fsync time: 4.1154
fsync time: 4.9604
fsync time: 10.5325
fsync time: 10.4543
fsync time: 10.4970
fsync time: 10.5570
fsync time: 5.2717
fsync time: 10.5619
fsync time: 5.3058
fsync time: 3.1019
fsync time: 5.1504
fsync time: 5.7564
fsync time: 10.5998
fsync time: 4.0895


Regards, Heinz.

Re: SSD and IO schedulers

[Corrado Zoccolo]

Well, that's not an usual workload for netbooks, where most SSDs are
currently deployed.
For usual workloads, that are mostly read, cfq has lower performance
both in throughput and in latency than deadline.

Corrado

2009/4/8 Heinz Diehl <htd@fancy-poultry.org>:
> On 08.04.2009, Corrado Zoccolo wrote:
>
>> I found that elevator=deadline performs much better than noop for
>> writes, and almost as well for reads
> [....]
>
> The DL elevator has slightly more throughput than cfq and anticipatory,
> but is almost unusuable under load.
>
> Running Theodore Ts'os "fsync-tester" while doing Linus' torture test
> "while : ; do time sh -c "dd if=/dev/zero of=bigfile bs=8M count=256 ; sync; rm bigfile"; done"
> shows it clearly:
>
> mount: /dev/sda4 on /home type xfs (rw,noatime,logbsize=256k,logbufs=2,nobarrier)
> Kernel 2.6.29.1 (vanilla)
>
> with cfq:
>
> htd@liesel:~/!> ./fsync-tester
> fsync time: 0.7640
> fsync time: 0.6166
> fsync time: 1.2830
> fsync time: 0.4273
> fsync time: 1.1693
> fsync time: 1.7466
> fsync time: 1.2477
> fsync time: 1.9411
> fsync time: 1.9636
> fsync time: 1.9065
> fsync time: 1.1561
> fsync time: 1.8267
> fsync time: 0.2431
> fsync time: 0.2898
> fsync time: 0.2394
> fsync time: 0.4309
> fsync time: 1.5699
> fsync time: 0.3742
> fsync time: 1.3194
> fsync time: 1.9442
> fsync time: 1.0899
> fsync time: 1.9443
> fsync time: 1.0062
>
> with dl:
>
> fsync time: 10.5853
> fsync time: 10.3339
> fsync time: 5.3374
> fsync time: 6.5707
> fsync time: 10.6095
> fsync time: 4.1154
> fsync time: 4.9604
> fsync time: 10.5325
> fsync time: 10.4543
> fsync time: 10.4970
> fsync time: 10.5570
> fsync time: 5.2717
> fsync time: 10.5619
> fsync time: 5.3058
> fsync time: 3.1019
> fsync time: 5.1504
> fsync time: 5.7564
> fsync time: 10.5998
> fsync time: 4.0895
>
>
> Regards, Heinz.
>
>



-- 
__________________________________________________________________________

dott. Corrado Zoccolo                          mailto:czoccolo@gmail.com
PhD - Department of Computer Science - University of Pisa, Italy
--------------------------------------------------------------------------
The self-confidence of a warrior is not the self-confidence of the average
man. The average man seeks certainty in the eyes of the onlooker and calls
that self-confidence. The warrior seeks impeccability in his own eyes and
calls that humbleness.
                               Tales of Power - C. Castaneda

Re: SSD and IO schedulers

[Heinz Diehl]

On 08.04.2009, Corrado Zoccolo wrote: 

> Well, that's not an usual workload for netbooks, where most SSDs are
> currently deployed.

Yes, that's right.

> For usual workloads, that are mostly read, cfq has lower performance
> both in throughput and in latency than deadline.

I don't have a netbook myself, but a Notebook with a singlecore
Intel M-530 CPU and an SSD harddisk, hdparm says:

[....]
ATA device, with non-removable media
Model Number:       OCZ SOLID_SSD
Serial Number:      MK0708520E8AA000B
Firmware Revision:  02.10104
[....]

I did run a short test with fsync-tester, running 10 read-processes
on the disk at the same time. The results between CFQ and DL don't differ
visibly. Maybe I don't get the point, or my tests simply suck, 
but with these results in mind, and considering the fact that when 
the load gets gradually higher, DL will lead to hickups up to ca. 10 secs, 
I would say that DL sucks _bigtime_ , compared to CFQ. 
(Throughput doesn't differ that much either..).


CFQ:

fsync time: 0.0209
fsync time: 0.0204
fsync time: 0.2026
fsync time: 0.2053
fsync time: 0.2036
fsync time: 0.2348
fsync time: 0.2030
fsync time: 0.2051
fsync time: 0.2024
fsync time: 0.2108
fsync time: 0.2025
fsync time: 0.2025
fsync time: 0.2030
fsync time: 0.2006
fsync time: 0.2368
fsync time: 0.2070
fsync time: 0.2009
fsync time: 0.2033
fsync time: 0.2101
fsync time: 0.2054
fsync time: 0.2028
fsync time: 0.2031
fsync time: 0.2073
fsync time: 0.2100
fsync time: 0.2078
fsync time: 0.2093
fsync time: 0.0275
fsync time: 0.0217
fsync time: 0.0298
fsync time: 0.0206
fsync time: 0.0184
fsync time: 0.0201
fsync time: 0.0169
fsync time: 0.0202
fsync time: 0.0186
fsync time: 0.0224
fsync time: 0.0224
fsync time: 0.0214
fsync time: 0.0246

DL

fsync time: 0.0296
fsync time: 0.0223
fsync time: 0.0262
fsync time: 0.0232
fsync time: 0.0230
fsync time: 0.0235
fsync time: 0.0187
fsync time: 0.0284
fsync time: 0.0227
fsync time: 0.0314
fsync time: 0.0236
fsync time: 0.0251
fsync time: 0.0221
fsync time: 0.0279
fsync time: 0.0244
fsync time: 0.0217
fsync time: 0.0248
fsync time: 0.0241
fsync time: 0.0229
fsync time: 0.0212
fsync time: 0.0243
fsync time: 0.0227
fsync time: 0.0257
fsync time: 0.0206
fsync time: 0.0214
fsync time: 0.0255
fsync time: 0.0213
fsync time: 0.0212
fsync time: 0.0266
fsync time: 0.0221
fsync time: 0.0212
fsync time: 0.0246
fsync time: 0.0208
fsync time: 0.0267
fsync time: 0.0220
fsync time: 0.0213
fsync time: 0.0212
fsync time: 0.0264


htd@wildsau:~> bonnie++ -u htd:default -d /testing -s 4004m -m wildsau -n 16:100000:16:64

CFQ

Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
wildsau     16016M 79619  45 78058  14 28841   7 98629  61 138596  14  1292   3
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
    16:100000:16/64   594   7 +++++ +++  1309   6   556   6 +++++ +++   449   4


DL

Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine       Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec  %CP  /sec %CP
wildsau     16016M 80619  47 78123  14 27842   7 96317  59 135446  14  1383   4
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
    16:100000:16/64   601   8 +++++ +++  1288   6   546   6 +++++ +++   432   4

Re: SSD and IO schedulers

[Heinz Diehl]

On 09.04.2009, Heinz Diehl wrote: 

[....]

Quoting myself, I did copy in the wrong results from another machine,
sorry for that! The fsync-tester results were the right ones, though.

Here are the correct ones for the bonnie++ runs (the overall results doesn't differ 
significantly either):


CFQ

Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
wildsau       4004M 69133  48 67512  12 21810   8 94426  59 129977 13  1163   3
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
    16:100000:16/64   589   8 +++++ +++  1211   6   544   7 +++++ +++   421   4


DL

Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
wildsau       4004M 71112  46 66155  15 22001   8 95132  62 130416 14  1191   3
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
    16:100000:16/64   565   9 +++++ +++  1339   6   561   8 +++++ +++   430   4

Re: SSD and IO schedulers

[Bill Davidsen]

Heinz Diehl wrote:
> On 08.04.2009, Corrado Zoccolo wrote: 
> 
>> I found that elevator=deadline performs much better than noop for
>> writes, and almost as well for reads
> [....]
> 
> The DL elevator has slightly more throughput than cfq and anticipatory,
> but is almost unusuable under load.
> 
> Running Theodore Ts'os "fsync-tester" while doing Linus' torture test 
> "while : ; do time sh -c "dd if=/dev/zero of=bigfile bs=8M count=256 ; sync; rm bigfile"; done"
> shows it clearly:
> 
This is good information, and if I ever configure a netbook for run fsync-tester 
I shall avoid the DL scheduler. ;-(

However... this test, and several others designed to find the ultimate 
performance limits of disk io, don't mimic any typical use of most desktops and 
virtually all netbooks.

Is there a benchmark which would return so useful data for typical use, doing 
some mail, some browsing, and maybe some light presentation, spreadsheet, or 
word processing. None of those uses are likely to generate this level of io, 
this file size, etc. The number of users is one, it's not used as a server, and 
probably most of the tuning done (if any) is aimed at battery life rather than 
blinding speed with a three digit load average.

I don't think this is a useful benchmark for netbooks, and hopefully there is a 
test available which will give more insight into the performance in typical use.

-- 
Bill Davidsen <davidsen@tmr.com>
   "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

Re: SSD and IO schedulers

[Theodore Tso]

On Thu, Apr 09, 2009 at 07:56:32PM -0400, Bill Davidsen wrote:
> This is good information, and if I ever configure a netbook for run 
> fsync-tester I shall avoid the DL scheduler. ;-(
>
> However... this test, and several others designed to find the ultimate  
> performance limits of disk io, don't mimic any typical use of most 
> desktops and virtually all netbooks.
>
> Is there a benchmark which would return so useful data for typical use, 
> doing some mail, some browsing, and maybe some light presentation, 
> spreadsheet, or word processing. None of those uses are likely to 
> generate this level of io, this file size, etc. The number of users is 
> one, it's not used as a server, and probably most of the tuning done (if 
> any) is aimed at battery life rather than blinding speed with a three 
> digit load average.

As long as you don't believe a netbook user will ever try to type an
e-mail using a mail reader like alpine (which is what Linus uses),
while running "yum update" in the background, sure.  But if you don't
think that is a normal use case, I'll let you argue with Linus on that
score.  In any case, the big-file-write-and-flush plus fsync-tester
was designed to roughly replicate this scenario which Linus saw on his
desktop system.

    	     			  	   	       - Ted

Re: SSD and IO schedulers

[Bill Davidsen]

Theodore Tso wrote:
> On Thu, Apr 09, 2009 at 07:56:32PM -0400, Bill Davidsen wrote:
>   
>> This is good information, and if I ever configure a netbook for run 
>> fsync-tester I shall avoid the DL scheduler. ;-(
>>
>> However... this test, and several others designed to find the ultimate  
>> performance limits of disk io, don't mimic any typical use of most 
>> desktops and virtually all netbooks.
>>
>> Is there a benchmark which would return so useful data for typical use, 
>> doing some mail, some browsing, and maybe some light presentation, 
>> spreadsheet, or word processing. None of those uses are likely to 
>> generate this level of io, this file size, etc. The number of users is 
>> one, it's not used as a server, and probably most of the tuning done (if 
>> any) is aimed at battery life rather than blinding speed with a three 
>> digit load average.
>>     
>
> As long as you don't believe a netbook user will ever try to type an
> e-mail using a mail reader like alpine (which is what Linus uses),
> while running "yum update" in the background, sure.  But if you don't
> think that is a normal use case, I'll let you argue with Linus on that
> score.  In any case, the big-file-write-and-flush plus fsync-tester
> was designed to roughly replicate this scenario which Linus saw on his
> desktop system.
>   

I almost fell out of my chair reading this, because I was doing exactly 
what you describe, applying the latest Fedora 9  security stuff and 
reading mail. The difference is that I'm not on a netbook, with a puny 
CPU, small memory and 5400 rpm drive, but a VM running on a host with 
multi-core, 8GB RAM, and a raid array of fast TB drives. The way I would 
use a netbook would be as a glorified PDA, and I still feel that a 
useful benchmark should duplicate the typical use, rather than some case 
which occurs a few times a month, unless that case is the critical load 
and justified less optimal performance the majority of the time.

Thanks for the history lesson, but I want my netbook to handle best the 
stuff I do most. If the modified scheduler performed better when I am 
reading mail and need to open a spreadsheet attachment, I'd certainly 
prefer that. Whether I'd bother to build my own patched kernel would 
depend on how "faster" it ran, I'm reasonably happy with the way that 
runs on what I have.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
    - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.