Re: Bad SSD performance with recent kernels

[Herbert Poetzl <herbert@13thfloor.at>]

On Mon, Jan 30, 2012 at 10:28:37PM +0800, Wu Fengguang wrote:
> On Mon, Jan 30, 2012 at 06:31:34PM +0800, Li, Shaohua wrote:

>> Looks the 2.6.39 block plug introduces some latency here. deleting
>> blk_start_plug/blk_finish_plug in generic_file_aio_read seems
>> workaround
>> the issue. The plug seems not good for sequential IO, because readahead
>> code already has plug and has fine grained control.

> Why not remove the generic_file_aio_read() plug completely? It
> actually prevents unplugging immediately after the readahead IO is
> submitted and in turn stalls the IO pipeline as showed by Eric's
> blktrace data.

> Eric, will you test this patch? Thank you.

> --- linux.orig/mm/filemap.c	2012-01-30 22:27:11.000000000 +0800
> +++ linux/mm/filemap.c	2012-01-30 22:27:25.000000000 +0800
> @@ -1414,15 +1414,12 @@ generic_file_aio_read(struct kiocb *iocb
>  	unsigned long seg = 0;
>  	size_t count;
>  	loff_t *ppos = &iocb->ki_pos;
> -	struct blk_plug plug;

>  	count = 0;
>  	retval = generic_segment_checks(iov, &nr_segs, &count, VERIFY_WRITE);
>  	if (retval)
>  		return retval;

> -	blk_start_plug(&plug);
> -
>  	/* coalesce the iovecs and go direct-to-BIO for O_DIRECT */
>  	if (filp->f_flags & O_DIRECT) {
>  		loff_t size;
> @@ -1495,7 +1492,6 @@ generic_file_aio_read(struct kiocb *iocb
>  			break;
>  	}
>  out:
> -	blk_finish_plug(&plug);
>  	return retval;
>  }
>  EXPORT_SYMBOL(generic_file_aio_read);

I can confirm, this seems to fix the issue here as well:

kernel    ------------ read /dev/sda -------------
          --- noop ---  - deadline -  ---- cfs ---
          [MB/s]  %CPU  [MB/s]  %CPU  [MB/s]  %CPU
--------------------------------------------------
3.2.2      45.82   3.7   44.85   3.6   45.04   3.4
3.2.2i     45.59   2.3   51.78   2.6   46.03   2.2
3.2.2i128 250.24  20.9  252.68  21.3  250.00  21.6
3.2.2p    250.88  20.8  249.69  20.7  252.34  21.4
3.2.2p128 249.36  20.7  251.47  20.9  249.03  21.2

3.2.2 ... unmodified 3.2.2 kernel 1M
3.2.2i .. unmodified 3.2.2 kernel with idle=poll, 1M
3.2.2i128 unmodified 3.2.2 kernel with idle=poll, 128k 
3.2.2p .. unmodified 3.2.2 kernel with patch, 1M
3.2.2p128 unmodified 3.2.2 kernel with patch, 128k

thanks again,
Herbert