Re: Why is MEGASAS_SAS_QD set to 256?

From: Yu Kuai <yukuai1@huaweicloud.com>
To: Ming Lei <ming.lei@redhat.com>, Yu Kuai <yukuai1@huaweicloud.com>
Cc: John Garry <john.g.garry@oracle.com>,
	kashyap.desai@broadcom.com, sumit.saxena@broadcom.com,
	shivasharan.srikanteshwara@broadcom.com, jejb@linux.ibm.com,
	martin.petersen@oracle.com, megaraidlinux.pdl@broadcom.com,
	linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-block <linux-block@vger.kernel.org>,
	"zhangyi (F)" <yi.zhang@huawei.com>,
	"yukuai (C)" <yukuai3@huawei.com>
Subject: Re: Why is MEGASAS_SAS_QD set to 256?
Date: Mon, 28 Nov 2022 11:09:28 +0800	[thread overview]
Message-ID: <27aae921-d016-0416-17ff-a591cd2ae12d@huaweicloud.com> (raw)
In-Reply-To: <Y4MxEcF+DWCAgxGJ@T590>

Hi,

在 2022/11/27 17:42, Ming Lei 写道:
> On Sat, Nov 26, 2022 at 02:08:02PM +0800, Yu Kuai wrote:
>> Hi, Ming
>>
>> 在 2022/11/26 10:18, Ming Lei 写道:
>>>
>>> If you want aggressive merge on sequential IO workload, the queue depth need
>>> to be a bit less, then more requests can be staggered into scheduler queue,
>>> and merge chance is increased.
>>
>> But if nr_requests >= queue_depth, it seems to me elevator will have no
>> effect, no request can be merged or sorted by scheduler, right?
> 
> Yeah.
> 
> If nr_requests <= queue_depth, every request can be queued to
> driver/device, so requests won't be merged by scheduler.
> 
> But plug merge still works if IOs are submitted as batch.

Yes, io can still be merged by plug. I just find it a little werid to
set default elevator as deadline, and default queue_depth to 256. Which
means deadline here is useless.

> 
>>>
>>> If you want good perf on random IO perf, the queue depth needs to
>>> be deep enough to have enough parallelism for saturating SSD internal.
>>>
>>> But we don't recognize sequential/random IO pattern, and usually fixed
>>> queue depth is used.
>>
>> Is it possible to use none elevator and set large queue_depth if nvme is
>> used in this case?
> 
> Yeah, if the storage is SSD, usually none with bigger queue_depth should
> help, and usually 256 should be enough to saturate one single SSD for
> one well implemented driver.

Yes, I'm testing with multiple SSDs / NVMEs, and I can't get optimal
performance by default.

Thanks,
Kuai
> 
> 
> Thanks
> Ming
> 
> .
> 