From: Ming Lei <ming.lei@redhat.com> To: linux-scsi@vger.kernel.org, "Martin K . Petersen" <martin.petersen@oracle.com> Cc: James Bottomley <James.Bottomley@HansenPartnership.com>, Bart Van Assche <bvanassche@acm.org>, Hannes Reinecke <hare@suse.com>, Christoph Hellwig <hch@lst.de>, Jim Gill <jgill@vmware.com>, Cathy Avery <cavery@redhat.com>, "Ewan D . Milne" <emilne@redhat.com>, Brian King <brking@us.ibm.com>, James Smart <james.smart@broadcom.com>, "Juergen E . Fischer" <fischer@norbit.de>, Michael Schmitz <schmitzmic@gmail.com>, Finn Thain <fthain@telegraphics.com.au>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, devel@driverdev.osuosl.org, linux-usb@vger.kernel.org, Dan Carpenter <dan.carpenter@oracle.com>, Benjamin Block <bblock@linux.ibm.com>, Ming Lei <ming.lei@redhat.com> Subject: [PATCH V4 00/16] use sg helper to operate scatterlist Date: Mon, 17 Jun 2019 11:03:33 +0800 [thread overview] Message-ID: <20190617030349.26415-1-ming.lei@redhat.com> (raw) Hi, Scsi MQ makes a large static allocation for the first scatter gather list chunk for the driver to use. This is a performance headache we'd like to fix by reducing the size of the allocation to a 2 element array. Doing this will break the current guarantee that any driver using SG_ALL doesn't need to use the scatterlist iterators and can get away with directly dereferencing the array. Thus we need to update all drivers to use the scatterlist iterators and remove direct indexing of the scatterlist array before reducing the initial scatterlist allocation size in SCSI. So convert drivers to use scatterlist helper. There are two types of direct access on scatterlist in SCSI drivers: 1) operate on the scatterlist via scsi_sglist(scmd) directly, then one local variable of 'struct scatterlist *' is involved. 2) scsi_sglist(scmd) is stored to cmd->SCp.buffer and the scatterlist is used via cmd->SCp.buffer. The following coccinelle semantic patch is developed for finding the above two types of direct scatterlist uses: @@ struct scatterlist *p; @@ ( - ++p + p = sg_next(p) | - p++ + p = sg_next(p) | - p = p + 1 + p = sg_next(p) | - p += 1 + p = sg_next(p) | - --p + p = sg_non_exist_prev(p) | - p-- + p = sg_non_exist_prev(p) | - p = p - 1 + p = sg_non_exist_prev(p) | - p -= 1 + p = sg_non_exist_prev(p) ) @@ struct scatterlist *p; expression data != 0; @@ - p[data] + '!!!!!!use sg iterator helper!!!!!!' @@ struct scatterlist[] p; expression data != 0; @@ - p[data] + '!!!!!!use sg iterator helper!!!!!!' @@ struct scsi_cmnd *scmd; @@ ( - scmd->SCp.buffer++ + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - ++scmd->SCp.buffer + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - scmd->SCp.buffer += 1 + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - scmd->SCp.buffer = scmd->SCp.buffer + 1 + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - scmd->SCp.buffer-- + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) | - --scmd->SCp.buffer + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) | - scmd->SCp.buffer -= 1 + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) | - scmd->SCp.buffer = scmd->SCp.buffer - 1 + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) ) @@ struct scsi_cmnd *scmd; expression data != 0; @@ - scmd->SCp.buffer[data] + '!!!!!!use sg iterator helper!!!!!!' The 1st 10 patches are for handling type #1, and the other 6 patches for handling type #2, and all the 16 are found by the above coccinelle semantic patch. V4: - fix building failure on pmcraid's conversion - improve the coccinelle semantic patch to cover both two types of scatterlist direct use - driver 'staging: rtsx' is covered V3: - update commit log and cover letter, most of words are from James Bottomley V2: - use coccinelle semantic patch for finding direct sgl uses from scsi command(9 drivers found) - run 'git grep -E "SCp.buffer"' to find direct sgl uses from SCp.buffer(6 drivers are found) Finn Thain (1): NCR5380: Support chained sg lists Ming Lei (15): scsi: vmw_pscsi: use sg helper to operate scatterlist scsi: advansys: use sg helper to operate scatterlist scsi: lpfc: use sg helper to operate scatterlist scsi: mvumi: use sg helper to operate scatterlist scsi: ipr: use sg helper to operate scatterlist scsi: pmcraid: use sg helper to operate scatterlist usb: image: microtek: use sg helper to operate scatterlist staging: unisys: visorhba: use sg helper to operate scatterlist staging: rtsx: use sg helper to operate scatterlist s390: zfcp_fc: use sg helper to operate scatterlist scsi: aha152x: use sg helper to operate scatterlist scsi: imm: use sg helper to operate scatterlist scsi: pcmcia: nsp_cs: use sg helper to operate scatterlist scsi: ppa: use sg helper to operate scatterlist scsi: wd33c93: use sg helper to operate scatterlist drivers/s390/scsi/zfcp_fc.c | 4 +- drivers/scsi/NCR5380.c | 41 ++++++++---------- drivers/scsi/advansys.c | 2 +- drivers/scsi/aha152x.c | 42 +++++++++---------- drivers/scsi/imm.c | 2 +- drivers/scsi/ipr.c | 28 +++++++------ drivers/scsi/lpfc/lpfc_nvmet.c | 3 +- drivers/scsi/mvumi.c | 9 ++-- drivers/scsi/pcmcia/nsp_cs.c | 4 +- drivers/scsi/pmcraid.c | 14 +++---- drivers/scsi/ppa.c | 2 +- drivers/scsi/vmw_pvscsi.c | 2 +- drivers/scsi/wd33c93.c | 2 +- drivers/staging/rts5208/rtsx_transport.c | 4 +- .../staging/unisys/visorhba/visorhba_main.c | 9 ++-- drivers/usb/image/microtek.c | 20 ++++----- drivers/usb/image/microtek.h | 2 +- 17 files changed, 90 insertions(+), 100 deletions(-) -- 2.20.1
WARNING: multiple messages have this Message-ID (diff)
From: Ming Lei <ming.lei@redhat.com> To: linux-scsi@vger.kernel.org, "Martin K . Petersen" <martin.petersen@oracle.com> Cc: Michael Schmitz <schmitzmic@gmail.com>, devel@driverdev.osuosl.org, Hannes Reinecke <hare@suse.com>, Benjamin Block <bblock@linux.ibm.com>, Bart Van Assche <bvanassche@acm.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, linux-usb@vger.kernel.org, James Smart <james.smart@broadcom.com>, "Ewan D . Milne" <emilne@redhat.com>, Jim Gill <jgill@vmware.com>, James Bottomley <James.Bottomley@HansenPartnership.com>, Brian King <brking@us.ibm.com>, Finn Thain <fthain@telegraphics.com.au>, "Juergen E . Fischer" <fischer@norbit.de>, Ming Lei <ming.lei@redhat.com>, Christoph Hellwig <hch@lst.de>, Dan Carpenter <dan.carpenter@oracle.com> Subject: [PATCH V4 00/16] use sg helper to operate scatterlist Date: Mon, 17 Jun 2019 11:03:33 +0800 [thread overview] Message-ID: <20190617030349.26415-1-ming.lei@redhat.com> (raw) Hi, Scsi MQ makes a large static allocation for the first scatter gather list chunk for the driver to use. This is a performance headache we'd like to fix by reducing the size of the allocation to a 2 element array. Doing this will break the current guarantee that any driver using SG_ALL doesn't need to use the scatterlist iterators and can get away with directly dereferencing the array. Thus we need to update all drivers to use the scatterlist iterators and remove direct indexing of the scatterlist array before reducing the initial scatterlist allocation size in SCSI. So convert drivers to use scatterlist helper. There are two types of direct access on scatterlist in SCSI drivers: 1) operate on the scatterlist via scsi_sglist(scmd) directly, then one local variable of 'struct scatterlist *' is involved. 2) scsi_sglist(scmd) is stored to cmd->SCp.buffer and the scatterlist is used via cmd->SCp.buffer. The following coccinelle semantic patch is developed for finding the above two types of direct scatterlist uses: @@ struct scatterlist *p; @@ ( - ++p + p = sg_next(p) | - p++ + p = sg_next(p) | - p = p + 1 + p = sg_next(p) | - p += 1 + p = sg_next(p) | - --p + p = sg_non_exist_prev(p) | - p-- + p = sg_non_exist_prev(p) | - p = p - 1 + p = sg_non_exist_prev(p) | - p -= 1 + p = sg_non_exist_prev(p) ) @@ struct scatterlist *p; expression data != 0; @@ - p[data] + '!!!!!!use sg iterator helper!!!!!!' @@ struct scatterlist[] p; expression data != 0; @@ - p[data] + '!!!!!!use sg iterator helper!!!!!!' @@ struct scsi_cmnd *scmd; @@ ( - scmd->SCp.buffer++ + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - ++scmd->SCp.buffer + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - scmd->SCp.buffer += 1 + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - scmd->SCp.buffer = scmd->SCp.buffer + 1 + scmd->SCp.buffer = sg_next(scmd->SCp.buffer) | - scmd->SCp.buffer-- + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) | - --scmd->SCp.buffer + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) | - scmd->SCp.buffer -= 1 + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) | - scmd->SCp.buffer = scmd->SCp.buffer - 1 + scmd->SCp.buffer = sg_no_exit_prev(scmd->SCp.buffer) ) @@ struct scsi_cmnd *scmd; expression data != 0; @@ - scmd->SCp.buffer[data] + '!!!!!!use sg iterator helper!!!!!!' The 1st 10 patches are for handling type #1, and the other 6 patches for handling type #2, and all the 16 are found by the above coccinelle semantic patch. V4: - fix building failure on pmcraid's conversion - improve the coccinelle semantic patch to cover both two types of scatterlist direct use - driver 'staging: rtsx' is covered V3: - update commit log and cover letter, most of words are from James Bottomley V2: - use coccinelle semantic patch for finding direct sgl uses from scsi command(9 drivers found) - run 'git grep -E "SCp.buffer"' to find direct sgl uses from SCp.buffer(6 drivers are found) Finn Thain (1): NCR5380: Support chained sg lists Ming Lei (15): scsi: vmw_pscsi: use sg helper to operate scatterlist scsi: advansys: use sg helper to operate scatterlist scsi: lpfc: use sg helper to operate scatterlist scsi: mvumi: use sg helper to operate scatterlist scsi: ipr: use sg helper to operate scatterlist scsi: pmcraid: use sg helper to operate scatterlist usb: image: microtek: use sg helper to operate scatterlist staging: unisys: visorhba: use sg helper to operate scatterlist staging: rtsx: use sg helper to operate scatterlist s390: zfcp_fc: use sg helper to operate scatterlist scsi: aha152x: use sg helper to operate scatterlist scsi: imm: use sg helper to operate scatterlist scsi: pcmcia: nsp_cs: use sg helper to operate scatterlist scsi: ppa: use sg helper to operate scatterlist scsi: wd33c93: use sg helper to operate scatterlist drivers/s390/scsi/zfcp_fc.c | 4 +- drivers/scsi/NCR5380.c | 41 ++++++++---------- drivers/scsi/advansys.c | 2 +- drivers/scsi/aha152x.c | 42 +++++++++---------- drivers/scsi/imm.c | 2 +- drivers/scsi/ipr.c | 28 +++++++------ drivers/scsi/lpfc/lpfc_nvmet.c | 3 +- drivers/scsi/mvumi.c | 9 ++-- drivers/scsi/pcmcia/nsp_cs.c | 4 +- drivers/scsi/pmcraid.c | 14 +++---- drivers/scsi/ppa.c | 2 +- drivers/scsi/vmw_pvscsi.c | 2 +- drivers/scsi/wd33c93.c | 2 +- drivers/staging/rts5208/rtsx_transport.c | 4 +- .../staging/unisys/visorhba/visorhba_main.c | 9 ++-- drivers/usb/image/microtek.c | 20 ++++----- drivers/usb/image/microtek.h | 2 +- 17 files changed, 90 insertions(+), 100 deletions(-) -- 2.20.1
next reply other threads:[~2019-06-17 3:04 UTC|newest] Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-06-17 3:03 Ming Lei [this message] 2019-06-17 3:03 ` [PATCH V4 00/16] use sg helper to operate scatterlist Ming Lei 2019-06-17 3:03 ` [PATCH V4 01/16] scsi: vmw_pscsi: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 02/16] scsi: advansys: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 03/16] scsi: lpfc: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 04/16] scsi: mvumi: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 8:22 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 05/16] scsi: ipr: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:24 ` Christoph Hellwig 2019-06-17 8:24 ` Christoph Hellwig 2019-06-17 8:50 ` Ming Lei 2019-06-17 8:50 ` Ming Lei 2019-06-17 3:03 ` [PATCH V4 06/16] scsi: pmcraid: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:24 ` Christoph Hellwig 2019-06-17 8:24 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 07/16] usb: image: microtek: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:25 ` Christoph Hellwig 2019-06-17 8:25 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 08/16] staging: unisys: visorhba: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:25 ` Christoph Hellwig 2019-06-17 8:25 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 09/16] staging: rtsx: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:27 ` Christoph Hellwig 2019-06-17 8:27 ` Christoph Hellwig 2019-06-17 9:15 ` Ming Lei 2019-06-17 9:15 ` Ming Lei 2019-06-17 9:24 ` Christoph Hellwig 2019-06-17 9:24 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 10/16] s390: zfcp_fc: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:27 ` Christoph Hellwig 2019-06-17 8:27 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 11/16] scsi: aha152x: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 3:35 ` Finn Thain 2019-06-17 3:35 ` Finn Thain 2019-06-17 23:35 ` Finn Thain 2019-06-17 23:35 ` Finn Thain 2019-06-18 0:15 ` Ming Lei 2019-06-18 0:15 ` Ming Lei 2019-06-17 8:28 ` Christoph Hellwig 2019-06-17 8:28 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 12/16] scsi: imm: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:28 ` Christoph Hellwig 2019-06-17 8:28 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 13/16] scsi: pcmcia: nsp_cs: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:29 ` Christoph Hellwig 2019-06-17 8:29 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 14/16] scsi: ppa: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:29 ` Christoph Hellwig 2019-06-17 8:29 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 15/16] scsi: wd33c93: " Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:29 ` Christoph Hellwig 2019-06-17 8:29 ` Christoph Hellwig 2019-06-17 3:03 ` [PATCH V4 16/16] NCR5380: Support chained sg lists Ming Lei 2019-06-17 3:03 ` Ming Lei 2019-06-17 8:30 ` Christoph Hellwig 2019-06-17 8:30 ` Christoph Hellwig 2019-06-17 23:52 ` [PATCH V4 00/16] use sg helper to operate scatterlist Bart Van Assche 2019-06-17 23:52 ` Bart Van Assche
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190617030349.26415-1-ming.lei@redhat.com \ --to=ming.lei@redhat.com \ --cc=James.Bottomley@HansenPartnership.com \ --cc=bblock@linux.ibm.com \ --cc=brking@us.ibm.com \ --cc=bvanassche@acm.org \ --cc=cavery@redhat.com \ --cc=dan.carpenter@oracle.com \ --cc=devel@driverdev.osuosl.org \ --cc=emilne@redhat.com \ --cc=fischer@norbit.de \ --cc=fthain@telegraphics.com.au \ --cc=gregkh@linuxfoundation.org \ --cc=hare@suse.com \ --cc=hch@lst.de \ --cc=james.smart@broadcom.com \ --cc=jgill@vmware.com \ --cc=linux-scsi@vger.kernel.org \ --cc=linux-usb@vger.kernel.org \ --cc=martin.petersen@oracle.com \ --cc=schmitzmic@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.