iommu.lists.linux-foundation.org archive mirror
 help / color / mirror / Atom feed
From: Marek Szyprowski <m.szyprowski@samsung.com>
To: dri-devel@lists.freedesktop.org,
	iommu@lists.linux-foundation.org, linaro-mm-sig@lists.linaro.org,
	linux-kernel@vger.kernel.org
Cc: devel@driverdev.osuosl.org, "Todd Kjos" <tkjos@android.com>,
	"Bartlomiej Zolnierkiewicz" <b.zolnierkie@samsung.com>,
	"David Airlie" <airlied@linux.ie>,
	"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
	"Sumit Semwal" <sumit.semwal@linaro.org>,
	"Arve Hjønnevåg" <arve@android.com>,
	"Martijn Coenen" <maco@android.com>,
	"Christian Brauner" <christian@brauner.io>,
	"Daniel Vetter" <daniel@ffwll.ch>,
	"Joel Fernandes" <joel@joelfernandes.org>,
	"Laura Abbott" <labbott@redhat.com>,
	"Robin Murphy" <robin.murphy@arm.com>,
	"Christoph Hellwig" <hch@lst.de>,
	linux-arm-kernel@lists.infradead.org
Subject: [PATCH v5 32/38] staging: ion: fix common struct sg_table related issues
Date: Wed, 13 May 2020 15:32:39 +0200	[thread overview]
Message-ID: <20200513133245.6408-32-m.szyprowski@samsung.com> (raw)
In-Reply-To: <20200513133245.6408-1-m.szyprowski@samsung.com>

The Documentation/DMA-API-HOWTO.txt states that the dma_map_sg() function
returns the number of the created entries in the DMA address space.
However the subsequent calls to the dma_sync_sg_for_{device,cpu}() and
dma_unmap_sg must be called with the original number of the entries
passed to the dma_map_sg().

struct sg_table is a common structure used for describing a non-contiguous
memory buffer, used commonly in the DRM and graphics subsystems. It
consists of a scatterlist with memory pages and DMA addresses (sgl entry),
as well as the number of scatterlist entries: CPU pages (orig_nents entry)
and DMA mapped pages (nents entry).

It turned out that it was a common mistake to misuse nents and orig_nents
entries, calling DMA-mapping functions with a wrong number of entries or
ignoring the number of mapped entries returned by the dma_map_sg()
function.

To avoid such issues, lets use a common dma-mapping wrappers operating
directly on the struct sg_table objects and use scatterlist page
iterators where possible. This, almost always, hides references to the
nents and orig_nents entries, making the code robust, easier to follow
and copy/paste safe.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
For more information, see '[PATCH v5 00/38] DRM: fix struct sg_table nents
vs. orig_nents misuse' thread:
https://lore.kernel.org/linux-iommu/20200513132114.6046-1-m.szyprowski@samsung.com/T/
---
 drivers/staging/android/ion/ion.c             | 25 +++++++--------
 drivers/staging/android/ion/ion_heap.c        | 44 ++++++++-------------------
 drivers/staging/android/ion/ion_system_heap.c |  2 +-
 3 files changed, 25 insertions(+), 46 deletions(-)

diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c
index 38b51ea..3c9f095 100644
--- a/drivers/staging/android/ion/ion.c
+++ b/drivers/staging/android/ion/ion.c
@@ -147,14 +147,14 @@ static struct sg_table *dup_sg_table(struct sg_table *table)
 	if (!new_table)
 		return ERR_PTR(-ENOMEM);
 
-	ret = sg_alloc_table(new_table, table->nents, GFP_KERNEL);
+	ret = sg_alloc_table(new_table, table->orig_nents, GFP_KERNEL);
 	if (ret) {
 		kfree(new_table);
 		return ERR_PTR(-ENOMEM);
 	}
 
 	new_sg = new_table->sgl;
-	for_each_sg(table->sgl, sg, table->nents, i) {
+	for_each_sgtable_sg(table, sg, i) {
 		memcpy(new_sg, sg, sizeof(*sg));
 		new_sg->dma_address = 0;
 		new_sg = sg_next(new_sg);
@@ -224,12 +224,13 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment,
 {
 	struct ion_dma_buf_attachment *a = attachment->priv;
 	struct sg_table *table;
+	int ret;
 
 	table = a->table;
 
-	if (!dma_map_sg(attachment->dev, table->sgl, table->nents,
-			direction))
-		return ERR_PTR(-ENOMEM);
+	ret = dma_map_sgtable(attachment->dev, table, direction, 0);
+	if (ret)
+		return ERR_PTR(ret);
 
 	return table;
 }
@@ -238,7 +239,7 @@ static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment,
 			      struct sg_table *table,
 			      enum dma_data_direction direction)
 {
-	dma_unmap_sg(attachment->dev, table->sgl, table->nents, direction);
+	dma_unmap_sgtable(attachment->dev, table, direction, 0);
 }
 
 static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
@@ -296,10 +297,8 @@ static int ion_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
 	}
 
 	mutex_lock(&buffer->lock);
-	list_for_each_entry(a, &buffer->attachments, list) {
-		dma_sync_sg_for_cpu(a->dev, a->table->sgl, a->table->nents,
-				    direction);
-	}
+	list_for_each_entry(a, &buffer->attachments, list)
+		dma_sync_sgtable_for_cpu(a->dev, a->table, direction);
 
 unlock:
 	mutex_unlock(&buffer->lock);
@@ -319,10 +318,8 @@ static int ion_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
 	}
 
 	mutex_lock(&buffer->lock);
-	list_for_each_entry(a, &buffer->attachments, list) {
-		dma_sync_sg_for_device(a->dev, a->table->sgl, a->table->nents,
-				       direction);
-	}
+	list_for_each_entry(a, &buffer->attachments, list)
+		dma_sync_sgtable_for_device(a->dev, a->table, direction);
 	mutex_unlock(&buffer->lock);
 
 	return 0;
diff --git a/drivers/staging/android/ion/ion_heap.c b/drivers/staging/android/ion/ion_heap.c
index 9c23b23..79f2794 100644
--- a/drivers/staging/android/ion/ion_heap.c
+++ b/drivers/staging/android/ion/ion_heap.c
@@ -20,8 +20,7 @@
 void *ion_heap_map_kernel(struct ion_heap *heap,
 			  struct ion_buffer *buffer)
 {
-	struct scatterlist *sg;
-	int i, j;
+	struct sg_page_iter piter;
 	void *vaddr;
 	pgprot_t pgprot;
 	struct sg_table *table = buffer->sg_table;
@@ -38,14 +37,11 @@ void *ion_heap_map_kernel(struct ion_heap *heap,
 	else
 		pgprot = pgprot_writecombine(PAGE_KERNEL);
 
-	for_each_sg(table->sgl, sg, table->nents, i) {
-		int npages_this_entry = PAGE_ALIGN(sg->length) / PAGE_SIZE;
-		struct page *page = sg_page(sg);
-
-		BUG_ON(i >= npages);
-		for (j = 0; j < npages_this_entry; j++)
-			*(tmp++) = page++;
+	for_each_sgtable_page(table, &piter, 0) {
+		BUG_ON(tmp - pages >= npages);
+		*tmp++ = sg_page_iter_page(&piter);
 	}
+
 	vaddr = vmap(pages, npages, VM_MAP, pgprot);
 	vfree(pages);
 
@@ -64,32 +60,19 @@ void ion_heap_unmap_kernel(struct ion_heap *heap,
 int ion_heap_map_user(struct ion_heap *heap, struct ion_buffer *buffer,
 		      struct vm_area_struct *vma)
 {
+	struct sg_page_iter piter;
 	struct sg_table *table = buffer->sg_table;
 	unsigned long addr = vma->vm_start;
-	unsigned long offset = vma->vm_pgoff * PAGE_SIZE;
-	struct scatterlist *sg;
-	int i;
 	int ret;
 
-	for_each_sg(table->sgl, sg, table->nents, i) {
-		struct page *page = sg_page(sg);
-		unsigned long remainder = vma->vm_end - addr;
-		unsigned long len = sg->length;
+	for_each_sgtable_page(table, &piter, vma->vm_pgoff) {
+		struct page *page = sg_page_iter_page(&piter);
 
-		if (offset >= sg->length) {
-			offset -= sg->length;
-			continue;
-		} else if (offset) {
-			page += offset / PAGE_SIZE;
-			len = sg->length - offset;
-			offset = 0;
-		}
-		len = min(len, remainder);
-		ret = remap_pfn_range(vma, addr, page_to_pfn(page), len,
+		ret = remap_pfn_range(vma, addr, page_to_pfn(page), PAGE_SIZE,
 				      vma->vm_page_prot);
 		if (ret)
 			return ret;
-		addr += len;
+		addr += PAGE_SIZE;
 		if (addr >= vma->vm_end)
 			return 0;
 	}
@@ -109,15 +92,14 @@ static int ion_heap_clear_pages(struct page **pages, int num, pgprot_t pgprot)
 	return 0;
 }
 
-static int ion_heap_sglist_zero(struct scatterlist *sgl, unsigned int nents,
-				pgprot_t pgprot)
+static int ion_heap_sglist_zero(struct sg_table *sgt, pgprot_t pgprot)
 {
 	int p = 0;
 	int ret = 0;
 	struct sg_page_iter piter;
 	struct page *pages[32];
 
-	for_each_sg_page(sgl, &piter, nents, 0) {
+	for_each_sgtable_page(sgt, &piter, 0) {
 		pages[p++] = sg_page_iter_page(&piter);
 		if (p == ARRAY_SIZE(pages)) {
 			ret = ion_heap_clear_pages(pages, p, pgprot);
@@ -142,7 +124,7 @@ int ion_heap_buffer_zero(struct ion_buffer *buffer)
 	else
 		pgprot = pgprot_writecombine(PAGE_KERNEL);
 
-	return ion_heap_sglist_zero(table->sgl, table->nents, pgprot);
+	return ion_heap_sglist_zero(table, pgprot);
 }
 
 void ion_heap_freelist_add(struct ion_heap *heap, struct ion_buffer *buffer)
diff --git a/drivers/staging/android/ion/ion_system_heap.c b/drivers/staging/android/ion/ion_system_heap.c
index b83a1d1..eac0632 100644
--- a/drivers/staging/android/ion/ion_system_heap.c
+++ b/drivers/staging/android/ion/ion_system_heap.c
@@ -162,7 +162,7 @@ static void ion_system_heap_free(struct ion_buffer *buffer)
 	if (!(buffer->private_flags & ION_PRIV_FLAG_SHRINKER_FREE))
 		ion_heap_buffer_zero(buffer);
 
-	for_each_sg(table->sgl, sg, table->nents, i)
+	for_each_sgtable_sg(table, sg, i)
 		free_buffer_page(sys_heap, buffer, sg_page(sg));
 	sg_free_table(table);
 	kfree(table);
-- 
1.9.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  parent reply	other threads:[~2020-05-13 13:33 UTC|newest]

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20200513132127eucas1p23f6be10bbd627e69e36d2451068b3204@eucas1p2.samsung.com>
2020-05-13 13:21 ` [PATCH v5 00/38] DRM: fix struct sg_table nents vs. orig_nents misuse Marek Szyprowski
     [not found]   ` <CGME20200513133256eucas1p273110d0c8f67e52fc7385acef776efaa@eucas1p2.samsung.com>
2020-05-13 13:32     ` [PATCH v5 01/38] dma-mapping: add generic helpers for mapping sgtable objects Marek Szyprowski
     [not found]       ` <CGME20200513133257eucas1p1613a28a71a6c328c406397f7094534d1@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 02/38] scatterlist: add generic wrappers for iterating over " Marek Szyprowski
     [not found]       ` <CGME20200513133258eucas1p2e8ffb90e5553c12419a5af5261b48f5e@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 03/38] iommu: add generic helper for mapping " Marek Szyprowski
     [not found]       ` <CGME20200513133258eucas1p1eba6e0ced0c749655f028206d5f84b7a@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 04/38] drm: prime: add common helper to check scatterlist contiguity Marek Szyprowski
     [not found]       ` <CGME20200513133259eucas1p273f0e05005b7b1158d884295d35745fd@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 05/38] drm: prime: use sgtable iterators in drm_prime_sg_to_page_addr_arrays() Marek Szyprowski
2020-09-21 23:15           ` Alex Goins
2020-09-22  6:27             ` Marek Szyprowski
2020-09-22 21:12               ` Alex Goins
2020-09-25 21:23               ` [Linaro-mm-sig] " Alex Deucher
2020-09-30  7:15                 ` Marek Szyprowski
     [not found]       ` <CGME20200513133300eucas1p2fced04876641789a0f5623e70c910690@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 06/38] drm: core: fix common struct sg_table related issues Marek Szyprowski
     [not found]       ` <CGME20200513133301eucas1p244d094c774ad0efd5ea9125747d5cb1b@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 07/38] drm: amdgpu: " Marek Szyprowski
     [not found]       ` <CGME20200513133301eucas1p28c059d25e488ed5ec10feb8f414a1af2@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 08/38] drm: armada: " Marek Szyprowski
     [not found]       ` <CGME20200513133302eucas1p1edbd033e0137d0c973f82c409d3a4e5d@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 09/38] drm: etnaviv: " Marek Szyprowski
     [not found]       ` <CGME20200513133303eucas1p1e3ede8aa38f730d2be317dff27c03019@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 10/38] drm: exynos: use common helper for a scatterlist contiguity check Marek Szyprowski
     [not found]       ` <CGME20200513133304eucas1p1114334ce7250cf8189553a22080b69f4@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 11/38] drm: exynos: fix common struct sg_table related issues Marek Szyprowski
     [not found]       ` <CGME20200513133304eucas1p2d24ddf9d7d60aad254486010db86e6bc@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 12/38] drm: i915: " Marek Szyprowski
     [not found]       ` <CGME20200513133305eucas1p15187ed8fa1e4af181cafa5c65ddab4cd@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 13/38] drm: lima: " Marek Szyprowski
2020-05-18  1:27           ` Qiang Yu
     [not found]       ` <CGME20200513133306eucas1p1ab15386cc4bbcf09052a8175d3660997@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 14/38] drm: mediatek: use common helper for a scatterlist contiguity check Marek Szyprowski
     [not found]       ` <CGME20200513133307eucas1p1da70ed30775467792f4898ccce829d02@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 15/38] drm: mediatek: use common helper for extracting pages array Marek Szyprowski
     [not found]       ` <CGME20200513133308eucas1p205607f34ec3d6df747e21c9b27204ca3@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 16/38] drm: msm: fix common struct sg_table related issues Marek Szyprowski
     [not found]       ` <CGME20200513133308eucas1p248892466634483dfa3b23f11e4da0e68@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 17/38] drm: omapdrm: use common helper for extracting pages array Marek Szyprowski
     [not found]       ` <CGME20200513133309eucas1p1b5b2505556566570e6fb3433397c6168@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 18/38] drm: omapdrm: fix common struct sg_table related issues Marek Szyprowski
     [not found]       ` <CGME20200513133310eucas1p176c920345e7f05f670e26e330b358d7f@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 19/38] drm: panfrost: " Marek Szyprowski
2020-05-14 13:51           ` Rob Herring
     [not found]       ` <CGME20200513133311eucas1p261d84f947b03e4fc733137048aab55fb@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 20/38] drm: radeon: " Marek Szyprowski
     [not found]       ` <CGME20200513133312eucas1p15e5e3bb3ba8e74b923fdccc7decab375@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 21/38] drm: rockchip: use common helper for a scatterlist contiguity check Marek Szyprowski
     [not found]       ` <CGME20200513133312eucas1p2fc1186c0aeee2367898fc0c146f1ed52@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 22/38] drm: rockchip: fix common struct sg_table related issues Marek Szyprowski
     [not found]       ` <CGME20200513133313eucas1p1fa06e23680b6d983578f598166e64cd0@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 23/38] drm: tegra: " Marek Szyprowski
     [not found]       ` <CGME20200513133314eucas1p1a607b556e32887e7f5ca60eb09a476c6@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 24/38] drm: v3d: " Marek Szyprowski
2020-05-13 21:51           ` Eric Anholt
     [not found]       ` <CGME20200513133314eucas1p2f04e32d65e71c613a2a9aacb29064a7d@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 25/38] drm: virtio: " Marek Szyprowski
2020-05-15 10:12           ` Gerd Hoffmann
     [not found]       ` <CGME20200513133315eucas1p17499a099b78c332b30911345c3963368@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 26/38] drm: vmwgfx: " Marek Szyprowski
     [not found]       ` <CGME20200513133316eucas1p2ad01d27ea4388cb50424bcf112d710ef@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 27/38] xen: gntdev: " Marek Szyprowski
2020-05-18  5:17           ` Jürgen Groß
     [not found]       ` <CGME20200513133317eucas1p25446bcdce5220030c16477e1992385ff@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 28/38] drm: host1x: " Marek Szyprowski
     [not found]       ` <CGME20200513133317eucas1p27aead4025db2da13e5b7c3e14a7cd79d@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 29/38] drm: rcar-du: " Marek Szyprowski
2020-05-13 14:23           ` Laurent Pinchart
     [not found]       ` <CGME20200513133318eucas1p1dd6c1ac6a777874c115070d8b5197f34@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 30/38] dmabuf: " Marek Szyprowski
2020-05-15 10:13           ` Gerd Hoffmann
     [not found]       ` <CGME20200513133319eucas1p2eaae7e606163ec1b211a07a80a52655d@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 31/38] staging: ion: remove dead code Marek Szyprowski
     [not found]       ` <CGME20200513133320eucas1p14cc8a7155b3d9ac2e0e30db2128c1477@eucas1p1.samsung.com>
2020-05-13 13:32         ` Marek Szyprowski [this message]
     [not found]       ` <CGME20200513133321eucas1p13acea3aa6219ce5f7076c7677ef9eae3@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 33/38] staging: tegra-vde: fix common struct sg_table related issues Marek Szyprowski
2020-05-14 20:07           ` Dmitry Osipenko
     [not found]       ` <CGME20200513133322eucas1p19b55463a3926b7c1459034f2b02969f8@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 34/38] misc: fastrpc: " Marek Szyprowski
     [not found]       ` <CGME20200513133323eucas1p1344fe42c8710f0ed84cf3c9fdb5ca515@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 35/38] rapidio: " Marek Szyprowski
     [not found]       ` <CGME20200513133323eucas1p1519f5901d2a4ee85b781fcc36e9601f7@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 36/38] samples: vfio-mdev/mbochs: " Marek Szyprowski
     [not found]       ` <CGME20200513133324eucas1p25dc380046c516ca8f07677ed14f93e63@eucas1p2.samsung.com>
2020-05-13 13:32         ` [PATCH v5 37/38] media: pci: fix common ALSA DMA-mapping related codes Marek Szyprowski
     [not found]       ` <CGME20200513133325eucas1p1c8faf557b23b5d171b7c328eca5f6482@eucas1p1.samsung.com>
2020-05-13 13:32         ` [PATCH v5 38/38] videobuf2: use sgtable-based scatterlist wrappers Marek Szyprowski
2020-05-13 13:47   ` [PATCH v5 00/38] DRM: fix struct sg_table nents vs. orig_nents misuse Christoph Hellwig
2020-05-26  7:01     ` Marek Szyprowski
2020-05-26 11:31       ` Daniel Vetter
     [not found]   ` <CGME20200522125708eucas1p233b80b0741f087a84d47f24b6d91985f@eucas1p2.samsung.com>
2020-05-22 12:56     ` [PATCH v5 39/38] drm: xen: fix common struct sg_table related issues Marek Szyprowski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200513133245.6408-32-m.szyprowski@samsung.com \
    --to=m.szyprowski@samsung.com \
    --cc=airlied@linux.ie \
    --cc=arve@android.com \
    --cc=b.zolnierkie@samsung.com \
    --cc=christian@brauner.io \
    --cc=daniel@ffwll.ch \
    --cc=devel@driverdev.osuosl.org \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hch@lst.de \
    --cc=iommu@lists.linux-foundation.org \
    --cc=joel@joelfernandes.org \
    --cc=labbott@redhat.com \
    --cc=linaro-mm-sig@lists.linaro.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maco@android.com \
    --cc=robin.murphy@arm.com \
    --cc=sumit.semwal@linaro.org \
    --cc=tkjos@android.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).