* [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency @ 2020-04-02 9:27 leonid.ravich 2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich 2020-04-02 15:04 ` [PATCH " Dave Jiang 0 siblings, 2 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-02 9:27 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman, Alexios Zavras, Dave Jiang, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> prepare for changing alloc size. Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.c | 14 ++++++++------ drivers/dma/ioat/dma.h | 10 ++++++---- drivers/dma/ioat/init.c | 2 +- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 18c011e..1e0e6c1 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) u8 *pos; off_t offs; - chunk = idx / IOAT_DESCS_PER_2M; - idx &= (IOAT_DESCS_PER_2M - 1); + chunk = idx / IOAT_DESCS_PER_CHUNK; + idx &= (IOAT_DESCS_PER_CHUNK - 1); offs = idx * IOAT_DESC_SZ; pos = (u8 *)ioat_chan->descs[chunk].virt + offs; phys = ioat_chan->descs[chunk].hw + offs; @@ -370,7 +370,8 @@ struct ioat_ring_ent ** if (!ring) return NULL; - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; + ioat_chan->desc_chunks = chunks; for (i = 0; i < chunks; i++) { struct ioat_descs *descs = &ioat_chan->descs[i]; @@ -382,8 +383,9 @@ struct ioat_ring_ent ** for (idx = 0; idx < i; idx++) { descs = &ioat_chan->descs[idx]; - dma_free_coherent(to_dev(ioat_chan), SZ_2M, - descs->virt, descs->hw); + dma_free_coherent(to_dev(ioat_chan), + IOAT_CHUNK_SIZE, + descs->virt, descs->hw); descs->virt = NULL; descs->hw = 0; } @@ -404,7 +406,7 @@ struct ioat_ring_ent ** for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { dma_free_coherent(to_dev(ioat_chan), - SZ_2M, + IOAT_CHUNK_SIZE, ioat_chan->descs[idx].virt, ioat_chan->descs[idx].hw); ioat_chan->descs[idx].virt = NULL; diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index b8e8e0b..535aba9 100644 --- a/drivers/dma/ioat/dma.h +++ b/drivers/dma/ioat/dma.h @@ -81,6 +81,11 @@ struct ioatdma_device { u32 msixpba; }; +#define IOAT_MAX_ORDER 16 +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) +#define IOAT_CHUNK_SIZE (SZ_2M) +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ) + struct ioat_descs { void *virt; dma_addr_t hw; @@ -128,7 +133,7 @@ struct ioatdma_chan { u16 produce; struct ioat_ring_ent **ring; spinlock_t prep_lock; - struct ioat_descs descs[2]; + struct ioat_descs descs[IOAT_MAX_DESCS/IOAT_DESCS_PER_CHUNK]; int desc_chunks; int intr_coalesce; int prev_intr_coalesce; @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) return !!err; } -#define IOAT_MAX_ORDER 16 -#define IOAT_MAX_DESCS 65536 -#define IOAT_DESCS_PER_2M 32768 static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) { diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c index 60e9afb..58d1356 100644 --- a/drivers/dma/ioat/init.c +++ b/drivers/dma/ioat/init.c @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) } for (i = 0; i < ioat_chan->desc_chunks; i++) { - dma_free_coherent(to_dev(ioat_chan), SZ_2M, + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, ioat_chan->descs[i].virt, ioat_chan->descs[i].hw); ioat_chan->descs[i].virt = NULL; -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K 2020-04-02 9:27 [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich @ 2020-04-02 9:27 ` leonid.ravich 2020-04-02 15:06 ` Dave Jiang 2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-02 15:04 ` [PATCH " Dave Jiang 1 sibling, 2 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-02 9:27 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman, Alexios Zavras, Dave Jiang, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> current IOAT driver using big (2MB) allocations chunk for its descriptors therefore each ioat dma engine need 2 such chunks (64k entres in ring each entry 64B = 4MB) requiring 2 * 2M * dmaengine contiguies memory chunk might fail due to memory fragmention. so we decresging chunk size and using more chunks. Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index 535aba9..e9757bc 100644 --- a/drivers/dma/ioat/dma.h +++ b/drivers/dma/ioat/dma.h @@ -83,7 +83,7 @@ struct ioatdma_device { #define IOAT_MAX_ORDER 16 #define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) -#define IOAT_CHUNK_SIZE (SZ_2M) +#define IOAT_CHUNK_SIZE (SZ_512K) #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ) struct ioat_descs { -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K 2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich @ 2020-04-02 15:06 ` Dave Jiang 2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 1 sibling, 0 replies; 22+ messages in thread From: Dave Jiang @ 2020-04-02 15:06 UTC (permalink / raw) To: leonid.ravich, dmaengine Cc: lravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel On 4/2/2020 2:27 AM, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > current IOAT driver using big (2MB) allocations chunk for its descriptors > therefore each ioat dma engine need 2 such chunks > (64k entres in ring each entry 64B = 4MB) > requiring 2 * 2M * dmaengine contiguies memory chunk > might fail due to memory fragmention. > > so we decresging chunk size and using more chunks. decreasing > > Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> Acked-by: Dave Jiang <dave.jiang@intel.com> if the two patches have been tested on hw. > --- > drivers/dma/ioat/dma.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h > index 535aba9..e9757bc 100644 > --- a/drivers/dma/ioat/dma.h > +++ b/drivers/dma/ioat/dma.h > @@ -83,7 +83,7 @@ struct ioatdma_device { > > #define IOAT_MAX_ORDER 16 > #define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) > -#define IOAT_CHUNK_SIZE (SZ_2M) > +#define IOAT_CHUNK_SIZE (SZ_512K) > #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ) > > struct ioat_descs { > ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich 2020-04-02 15:06 ` Dave Jiang @ 2020-04-02 16:33 ` leonid.ravich 2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich ` (2 more replies) 1 sibling, 3 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-02 16:33 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> prepare for changing alloc size. Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.c | 14 ++++++++------ drivers/dma/ioat/dma.h | 10 ++++++---- drivers/dma/ioat/init.c | 2 +- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 18c011e..1e0e6c1 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) u8 *pos; off_t offs; - chunk = idx / IOAT_DESCS_PER_2M; - idx &= (IOAT_DESCS_PER_2M - 1); + chunk = idx / IOAT_DESCS_PER_CHUNK; + idx &= (IOAT_DESCS_PER_CHUNK - 1); offs = idx * IOAT_DESC_SZ; pos = (u8 *)ioat_chan->descs[chunk].virt + offs; phys = ioat_chan->descs[chunk].hw + offs; @@ -370,7 +370,8 @@ struct ioat_ring_ent ** if (!ring) return NULL; - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; + ioat_chan->desc_chunks = chunks; for (i = 0; i < chunks; i++) { struct ioat_descs *descs = &ioat_chan->descs[i]; @@ -382,8 +383,9 @@ struct ioat_ring_ent ** for (idx = 0; idx < i; idx++) { descs = &ioat_chan->descs[idx]; - dma_free_coherent(to_dev(ioat_chan), SZ_2M, - descs->virt, descs->hw); + dma_free_coherent(to_dev(ioat_chan), + IOAT_CHUNK_SIZE, + descs->virt, descs->hw); descs->virt = NULL; descs->hw = 0; } @@ -404,7 +406,7 @@ struct ioat_ring_ent ** for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { dma_free_coherent(to_dev(ioat_chan), - SZ_2M, + IOAT_CHUNK_SIZE, ioat_chan->descs[idx].virt, ioat_chan->descs[idx].hw); ioat_chan->descs[idx].virt = NULL; diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index b8e8e0b..5216c6b 100644 --- a/drivers/dma/ioat/dma.h +++ b/drivers/dma/ioat/dma.h @@ -81,6 +81,11 @@ struct ioatdma_device { u32 msixpba; }; +#define IOAT_MAX_ORDER 16 +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) +#define IOAT_CHUNK_SIZE (SZ_2M) +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) + struct ioat_descs { void *virt; dma_addr_t hw; @@ -128,7 +133,7 @@ struct ioatdma_chan { u16 produce; struct ioat_ring_ent **ring; spinlock_t prep_lock; - struct ioat_descs descs[2]; + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK]; int desc_chunks; int intr_coalesce; int prev_intr_coalesce; @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) return !!err; } -#define IOAT_MAX_ORDER 16 -#define IOAT_MAX_DESCS 65536 -#define IOAT_DESCS_PER_2M 32768 static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) { diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c index 60e9afb..58d1356 100644 --- a/drivers/dma/ioat/init.c +++ b/drivers/dma/ioat/init.c @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) } for (i = 0; i < ioat_chan->desc_chunks; i++) { - dma_free_coherent(to_dev(ioat_chan), SZ_2M, + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, ioat_chan->descs[i].virt, ioat_chan->descs[i].hw); ioat_chan->descs[i].virt = NULL; -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K 2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich @ 2020-04-02 16:33 ` leonid.ravich 2020-04-15 15:53 ` Vinod Koul 2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-02 16:42 ` [PATCH v2 " Dave Jiang 2020-04-15 15:52 ` Vinod Koul 2 siblings, 2 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-02 16:33 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> current IOAT driver using big (2MB) allocations chunk for its descriptors therefore each ioat dma engine need 2 such chunks (64k entres in ring each entry 64B = 4MB) requiring 2 * 2M * dmaengine contiguies memory chunk might fail due to memory fragmention. so we decreasing chunk size and using more chunks. Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index 5216c6b..e6b622e 100644 --- a/drivers/dma/ioat/dma.h +++ b/drivers/dma/ioat/dma.h @@ -83,7 +83,7 @@ struct ioatdma_device { #define IOAT_MAX_ORDER 16 #define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) -#define IOAT_CHUNK_SIZE (SZ_2M) +#define IOAT_CHUNK_SIZE (SZ_512K) #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) struct ioat_descs { -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K 2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich @ 2020-04-15 15:53 ` Vinod Koul 2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 1 sibling, 0 replies; 22+ messages in thread From: Vinod Koul @ 2020-04-15 15:53 UTC (permalink / raw) To: leonid.ravich Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel On 02-04-20, 19:33, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > current IOAT driver using big (2MB) allocations chunk for its descriptors > therefore each ioat dma engine need 2 such chunks > (64k entres in ring each entry 64B = 4MB) > requiring 2 * 2M * dmaengine contiguies memory chunk > might fail due to memory fragmention. This is quite decent explanation :) pls use upto 72 chars to make it a better read. > > so we decreasing chunk size and using more chunks. > > Acked-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> > --- > drivers/dma/ioat/dma.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h > index 5216c6b..e6b622e 100644 > --- a/drivers/dma/ioat/dma.h > +++ b/drivers/dma/ioat/dma.h > @@ -83,7 +83,7 @@ struct ioatdma_device { > > #define IOAT_MAX_ORDER 16 > #define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) > -#define IOAT_CHUNK_SIZE (SZ_2M) > +#define IOAT_CHUNK_SIZE (SZ_512K) > #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) > > struct ioat_descs { > -- > 1.9.3 -- ~Vinod ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich 2020-04-15 15:53 ` Vinod Koul @ 2020-04-16 17:06 ` leonid.ravich 2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich 2020-04-17 11:58 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency Vinod Koul 1 sibling, 2 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-16 17:06 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> changing macros which assumption is chunk size of 2M, which can be other size prepare for changing allocation chunk size. Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- Changing in v3: - Make the commit message more clearer. drivers/dma/ioat/dma.c | 14 ++++++++------ drivers/dma/ioat/dma.h | 10 ++++++---- drivers/dma/ioat/init.c | 2 +- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 18c011e..1e0e6c1 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) u8 *pos; off_t offs; - chunk = idx / IOAT_DESCS_PER_2M; - idx &= (IOAT_DESCS_PER_2M - 1); + chunk = idx / IOAT_DESCS_PER_CHUNK; + idx &= (IOAT_DESCS_PER_CHUNK - 1); offs = idx * IOAT_DESC_SZ; pos = (u8 *)ioat_chan->descs[chunk].virt + offs; phys = ioat_chan->descs[chunk].hw + offs; @@ -370,7 +370,8 @@ struct ioat_ring_ent ** if (!ring) return NULL; - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; + ioat_chan->desc_chunks = chunks; for (i = 0; i < chunks; i++) { struct ioat_descs *descs = &ioat_chan->descs[i]; @@ -382,8 +383,9 @@ struct ioat_ring_ent ** for (idx = 0; idx < i; idx++) { descs = &ioat_chan->descs[idx]; - dma_free_coherent(to_dev(ioat_chan), SZ_2M, - descs->virt, descs->hw); + dma_free_coherent(to_dev(ioat_chan), + IOAT_CHUNK_SIZE, + descs->virt, descs->hw); descs->virt = NULL; descs->hw = 0; } @@ -404,7 +406,7 @@ struct ioat_ring_ent ** for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { dma_free_coherent(to_dev(ioat_chan), - SZ_2M, + IOAT_CHUNK_SIZE, ioat_chan->descs[idx].virt, ioat_chan->descs[idx].hw); ioat_chan->descs[idx].virt = NULL; diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index b8e8e0b..5216c6b 100644 --- a/drivers/dma/ioat/dma.h +++ b/drivers/dma/ioat/dma.h @@ -81,6 +81,11 @@ struct ioatdma_device { u32 msixpba; }; +#define IOAT_MAX_ORDER 16 +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) +#define IOAT_CHUNK_SIZE (SZ_2M) +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) + struct ioat_descs { void *virt; dma_addr_t hw; @@ -128,7 +133,7 @@ struct ioatdma_chan { u16 produce; struct ioat_ring_ent **ring; spinlock_t prep_lock; - struct ioat_descs descs[2]; + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK]; int desc_chunks; int intr_coalesce; int prev_intr_coalesce; @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) return !!err; } -#define IOAT_MAX_ORDER 16 -#define IOAT_MAX_DESCS 65536 -#define IOAT_DESCS_PER_2M 32768 static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) { diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c index 60e9afb..58d1356 100644 --- a/drivers/dma/ioat/init.c +++ b/drivers/dma/ioat/init.c @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) } for (i = 0; i < ioat_chan->desc_chunks; i++) { - dma_free_coherent(to_dev(ioat_chan), SZ_2M, + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, ioat_chan->descs[i].virt, ioat_chan->descs[i].hw); ioat_chan->descs[i].virt = NULL; -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K 2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich @ 2020-04-16 17:06 ` leonid.ravich 2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich 2020-04-17 11:58 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency Vinod Koul 1 sibling, 1 reply; 22+ messages in thread From: leonid.ravich @ 2020-04-16 17:06 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> requreing kmalloc of 2M high chance to fail in fragmented memory. IOAT ring requires 64k * 64B memory which will be achived by 512k * 8 allocation instead of 2M * 2. Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- Changing in v3: - Make the commit message more clearer. drivers/dma/ioat/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index 5216c6b..e6b622e 100644 --- a/drivers/dma/ioat/dma.h +++ b/drivers/dma/ioat/dma.h @@ -83,7 +83,7 @@ struct ioatdma_device { #define IOAT_MAX_ORDER 16 #define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) -#define IOAT_CHUNK_SIZE (SZ_2M) +#define IOAT_CHUNK_SIZE (SZ_512K) #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) struct ioat_descs { -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler 2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich @ 2020-04-22 19:25 ` leonid.ravich 2020-04-22 19:25 ` [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich 2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich 0 siblings, 2 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-22 19:25 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang, Alexander.Barabash, Allison Randal, Thomas Gleixner, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> moving duplicate code from timeout error handling to common function. Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.c | 45 +++++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 26 deletions(-) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 1e0e6c1..da59b28 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -869,6 +869,23 @@ static void check_active(struct ioatdma_chan *ioat_chan) mod_timer(&ioat_chan->timer, jiffies + IDLE_TIMEOUT); } +static void ioat_reboot_chan(struct ioatdma_chan *ioat_chan) +{ + spin_lock_bh(&ioat_chan->prep_lock); + set_bit(IOAT_CHAN_DOWN, &ioat_chan->state); + spin_unlock_bh(&ioat_chan->prep_lock); + + ioat_abort_descs(ioat_chan); + dev_warn(to_dev(ioat_chan), "Reset channel...\n"); + ioat_reset_hw(ioat_chan); + dev_warn(to_dev(ioat_chan), "Restart channel...\n"); + ioat_restart_channel(ioat_chan); + + spin_lock_bh(&ioat_chan->prep_lock); + clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state); + spin_unlock_bh(&ioat_chan->prep_lock); +} + void ioat_timer_event(struct timer_list *t) { struct ioatdma_chan *ioat_chan = from_timer(ioat_chan, t, timer); @@ -891,19 +908,7 @@ void ioat_timer_event(struct timer_list *t) if (test_bit(IOAT_RUN, &ioat_chan->state)) { spin_lock_bh(&ioat_chan->cleanup_lock); - spin_lock_bh(&ioat_chan->prep_lock); - set_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); - - ioat_abort_descs(ioat_chan); - dev_warn(to_dev(ioat_chan), "Reset channel...\n"); - ioat_reset_hw(ioat_chan); - dev_warn(to_dev(ioat_chan), "Restart channel...\n"); - ioat_restart_channel(ioat_chan); - - spin_lock_bh(&ioat_chan->prep_lock); - clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); + ioat_reboot_chan(ioat_chan); spin_unlock_bh(&ioat_chan->cleanup_lock); } @@ -939,19 +944,7 @@ void ioat_timer_event(struct timer_list *t) dev_dbg(to_dev(ioat_chan), "Active descriptors: %d\n", ioat_ring_active(ioat_chan)); - spin_lock_bh(&ioat_chan->prep_lock); - set_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); - - ioat_abort_descs(ioat_chan); - dev_warn(to_dev(ioat_chan), "Resetting channel...\n"); - ioat_reset_hw(ioat_chan); - dev_warn(to_dev(ioat_chan), "Restarting channel...\n"); - ioat_restart_channel(ioat_chan); - - spin_lock_bh(&ioat_chan->prep_lock); - clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); + ioat_reboot_chan(ioat_chan); spin_unlock_bh(&ioat_chan->cleanup_lock); return; } else -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification. 2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich @ 2020-04-22 19:25 ` leonid.ravich 2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich 1 sibling, 0 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-22 19:25 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang, Allison Randal, Thomas Gleixner, Greg Kroah-Hartman, Alexander.Barabash, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> removing unnecessary mod_timer from timeout handler incase of ioat_cleanup_preamble() is true for cleaner code Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index da59b28..55a8cf1 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -922,17 +922,23 @@ void ioat_timer_event(struct timer_list *t) spin_lock_bh(&ioat_chan->prep_lock); check_active(ioat_chan); spin_unlock_bh(&ioat_chan->prep_lock); - spin_unlock_bh(&ioat_chan->cleanup_lock); - return; + goto unlock_out; + } + + /* handle the missed cleanup case */ + if (ioat_cleanup_preamble(ioat_chan, &phys_complete)) { + /* timer restarted in ioat_cleanup_preamble + * and IOAT_COMPLETION_ACK cleared + */ + __cleanup(ioat_chan, phys_complete); + goto unlock_out; } /* if we haven't made progress and we have already * acknowledged a pending completion once, then be more * forceful with a restart */ - if (ioat_cleanup_preamble(ioat_chan, &phys_complete)) - __cleanup(ioat_chan, phys_complete); - else if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) { + if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) { u32 chanerr; chanerr = readl(ioat_chan->reg_base + IOAT_CHANERR_OFFSET); @@ -945,12 +951,13 @@ void ioat_timer_event(struct timer_list *t) ioat_ring_active(ioat_chan)); ioat_reboot_chan(ioat_chan); - spin_unlock_bh(&ioat_chan->cleanup_lock); - return; - } else - set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state); + goto unlock_out; + } + + set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state); mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT); +unlock_out: spin_unlock_bh(&ioat_chan->cleanup_lock); } -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler 2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich 2020-04-22 19:25 ` [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich @ 2020-04-22 19:25 ` leonid.ravich 2020-04-22 19:58 ` Dave Jiang 2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich 1 sibling, 2 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-22 19:25 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Thomas Gleixner, Alexander.Barabash, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> completion timeout might trigger unnesesery DMA engine hw reboot in case of missed issue_pending() . Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- Changing in v2 - add log in case of such scenario drivers/dma/ioat/dma.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 55a8cf1..a958aaf 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -955,6 +955,14 @@ void ioat_timer_event(struct timer_list *t) goto unlock_out; } + /* handle missed issue pending case */ + if (ioat_ring_pending(ioat_chan)) { + dev_dbg(to_dev(ioat_chan), "Complition timeout while pending\n") + spin_lock_bh(&ioat_chan->prep_lock); + __ioat_issue_pending(ioat_chan); + spin_unlock_bh(&ioat_chan->prep_lock); + } + set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state); mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT); unlock_out: -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler 2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich @ 2020-04-22 19:58 ` Dave Jiang 2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich 1 sibling, 0 replies; 22+ messages in thread From: Dave Jiang @ 2020-04-22 19:58 UTC (permalink / raw) To: leonid.ravich, dmaengine Cc: lravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman, Thomas Gleixner, Alexander.Barabash, linux-kernel On 4/22/2020 12:25 PM, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > completion timeout might trigger unnesesery DMA engine hw reboot > in case of missed issue_pending() . > > Acked-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> > --- > Changing in v2 > - add log in case of such scenario > drivers/dma/ioat/dma.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c > index 55a8cf1..a958aaf 100644 > --- a/drivers/dma/ioat/dma.c > +++ b/drivers/dma/ioat/dma.c > @@ -955,6 +955,14 @@ void ioat_timer_event(struct timer_list *t) > goto unlock_out; > } > > + /* handle missed issue pending case */ > + if (ioat_ring_pending(ioat_chan)) { > + dev_dbg(to_dev(ioat_chan), "Complition timeout while pending\n") Completion timeout with pending descriptors. Also, maybe dev_warn() you think? > + spin_lock_bh(&ioat_chan->prep_lock); > + __ioat_issue_pending(ioat_chan); > + spin_unlock_bh(&ioat_chan->prep_lock); > + } > + > set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state); > mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT); > unlock_out: > ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from timeout handler 2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich 2020-04-22 19:58 ` Dave Jiang @ 2020-04-22 21:09 ` leonid.ravich 2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich ` (2 more replies) 1 sibling, 3 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-22 21:09 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang, Allison Randal, Alexander.Barabash, Alexios Zavras, Thomas Gleixner, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> moving duplicate code from timeout error handling to common function. Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.c | 45 +++++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 26 deletions(-) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 1e0e6c1..da59b28 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -869,6 +869,23 @@ static void check_active(struct ioatdma_chan *ioat_chan) mod_timer(&ioat_chan->timer, jiffies + IDLE_TIMEOUT); } +static void ioat_reboot_chan(struct ioatdma_chan *ioat_chan) +{ + spin_lock_bh(&ioat_chan->prep_lock); + set_bit(IOAT_CHAN_DOWN, &ioat_chan->state); + spin_unlock_bh(&ioat_chan->prep_lock); + + ioat_abort_descs(ioat_chan); + dev_warn(to_dev(ioat_chan), "Reset channel...\n"); + ioat_reset_hw(ioat_chan); + dev_warn(to_dev(ioat_chan), "Restart channel...\n"); + ioat_restart_channel(ioat_chan); + + spin_lock_bh(&ioat_chan->prep_lock); + clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state); + spin_unlock_bh(&ioat_chan->prep_lock); +} + void ioat_timer_event(struct timer_list *t) { struct ioatdma_chan *ioat_chan = from_timer(ioat_chan, t, timer); @@ -891,19 +908,7 @@ void ioat_timer_event(struct timer_list *t) if (test_bit(IOAT_RUN, &ioat_chan->state)) { spin_lock_bh(&ioat_chan->cleanup_lock); - spin_lock_bh(&ioat_chan->prep_lock); - set_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); - - ioat_abort_descs(ioat_chan); - dev_warn(to_dev(ioat_chan), "Reset channel...\n"); - ioat_reset_hw(ioat_chan); - dev_warn(to_dev(ioat_chan), "Restart channel...\n"); - ioat_restart_channel(ioat_chan); - - spin_lock_bh(&ioat_chan->prep_lock); - clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); + ioat_reboot_chan(ioat_chan); spin_unlock_bh(&ioat_chan->cleanup_lock); } @@ -939,19 +944,7 @@ void ioat_timer_event(struct timer_list *t) dev_dbg(to_dev(ioat_chan), "Active descriptors: %d\n", ioat_ring_active(ioat_chan)); - spin_lock_bh(&ioat_chan->prep_lock); - set_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); - - ioat_abort_descs(ioat_chan); - dev_warn(to_dev(ioat_chan), "Resetting channel...\n"); - ioat_reset_hw(ioat_chan); - dev_warn(to_dev(ioat_chan), "Restarting channel...\n"); - ioat_restart_channel(ioat_chan); - - spin_lock_bh(&ioat_chan->prep_lock); - clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state); - spin_unlock_bh(&ioat_chan->prep_lock); + ioat_reboot_chan(ioat_chan); spin_unlock_bh(&ioat_chan->cleanup_lock); return; } else -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification. 2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich @ 2020-04-22 21:09 ` leonid.ravich 2020-04-22 21:09 ` [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich 2020-04-23 7:21 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " Vinod Koul 2 siblings, 0 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-22 21:09 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang, Greg Kroah-Hartman, Alexander.Barabash, Alexios Zavras, Thomas Gleixner, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> removing unnecessary mod_timer from timeout handler incase of ioat_cleanup_preamble() is true for cleaner code Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- drivers/dma/ioat/dma.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index da59b28..55a8cf1 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -922,17 +922,23 @@ void ioat_timer_event(struct timer_list *t) spin_lock_bh(&ioat_chan->prep_lock); check_active(ioat_chan); spin_unlock_bh(&ioat_chan->prep_lock); - spin_unlock_bh(&ioat_chan->cleanup_lock); - return; + goto unlock_out; + } + + /* handle the missed cleanup case */ + if (ioat_cleanup_preamble(ioat_chan, &phys_complete)) { + /* timer restarted in ioat_cleanup_preamble + * and IOAT_COMPLETION_ACK cleared + */ + __cleanup(ioat_chan, phys_complete); + goto unlock_out; } /* if we haven't made progress and we have already * acknowledged a pending completion once, then be more * forceful with a restart */ - if (ioat_cleanup_preamble(ioat_chan, &phys_complete)) - __cleanup(ioat_chan, phys_complete); - else if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) { + if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) { u32 chanerr; chanerr = readl(ioat_chan->reg_base + IOAT_CHANERR_OFFSET); @@ -945,12 +951,13 @@ void ioat_timer_event(struct timer_list *t) ioat_ring_active(ioat_chan)); ioat_reboot_chan(ioat_chan); - spin_unlock_bh(&ioat_chan->cleanup_lock); - return; - } else - set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state); + goto unlock_out; + } + + set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state); mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT); +unlock_out: spin_unlock_bh(&ioat_chan->cleanup_lock); } -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler 2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich 2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich @ 2020-04-22 21:09 ` leonid.ravich 2020-04-23 7:21 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " Vinod Koul 2 siblings, 0 replies; 22+ messages in thread From: leonid.ravich @ 2020-04-22 21:09 UTC (permalink / raw) To: dmaengine Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang, Alexander.Barabash, Alexios Zavras, Thomas Gleixner, linux-kernel From: Leonid Ravich <Leonid.Ravich@emc.com> completion timeout might trigger unnesesery DMA engine hw reboot in case of missed issue_pending() . Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> --- Changing in v2: - fixing log spelling and level drivers/dma/ioat/dma.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index 55a8cf1..8ad0ad8 100644 --- a/drivers/dma/ioat/dma.c +++ b/drivers/dma/ioat/dma.c @@ -955,6 +955,15 @@ void ioat_timer_event(struct timer_list *t) goto unlock_out; } + /* handle missed issue pending case */ + if (ioat_ring_pending(ioat_chan)) { + dev_warn(to_dev(ioat_chan), + "Completion timeout with pending descriptors\n"); + spin_lock_bh(&ioat_chan->prep_lock); + __ioat_issue_pending(ioat_chan); + spin_unlock_bh(&ioat_chan->prep_lock); + } + set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state); mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT); unlock_out: -- 1.9.3 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from timeout handler 2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich 2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich 2020-04-22 21:09 ` [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich @ 2020-04-23 7:21 ` Vinod Koul 2 siblings, 0 replies; 22+ messages in thread From: Vinod Koul @ 2020-04-23 7:21 UTC (permalink / raw) To: leonid.ravich Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Allison Randal, Alexander.Barabash, Alexios Zavras, Thomas Gleixner, linux-kernel On 23-04-20, 00:09, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > moving duplicate code from timeout error handling to common > function. Applied all, thanks -- ~Vinod ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich @ 2020-04-17 11:58 ` Vinod Koul 1 sibling, 0 replies; 22+ messages in thread From: Vinod Koul @ 2020-04-17 11:58 UTC (permalink / raw) To: leonid.ravich Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel On 16-04-20, 20:06, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > changing macros which assumption is chunk size of 2M, > which can be other size > prepare for changing allocation chunk size. Applied both, thanks -- ~Vinod ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich @ 2020-04-02 16:42 ` Dave Jiang 2020-04-02 16:46 ` Ravich, Leonid 2020-04-15 15:52 ` Vinod Koul 2 siblings, 1 reply; 22+ messages in thread From: Dave Jiang @ 2020-04-02 16:42 UTC (permalink / raw) To: leonid.ravich, dmaengine Cc: lravich, Vinod Koul, Williams, Dan J, Greg Kroah-Hartman, Zavras, Alexios, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel On 4/2/2020 9:33 AM, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > prepare for changing alloc size. > > Acked-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> Hi Leonid, I haven't actually acked this patch yet, pending your answer on if this has been tested on hardware. Thanks. > --- > drivers/dma/ioat/dma.c | 14 ++++++++------ > drivers/dma/ioat/dma.h | 10 ++++++---- > drivers/dma/ioat/init.c | 2 +- > 3 files changed, 15 insertions(+), 11 deletions(-) > > diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c > index 18c011e..1e0e6c1 100644 > --- a/drivers/dma/ioat/dma.c > +++ b/drivers/dma/ioat/dma.c > @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) > u8 *pos; > off_t offs; > > - chunk = idx / IOAT_DESCS_PER_2M; > - idx &= (IOAT_DESCS_PER_2M - 1); > + chunk = idx / IOAT_DESCS_PER_CHUNK; > + idx &= (IOAT_DESCS_PER_CHUNK - 1); > offs = idx * IOAT_DESC_SZ; > pos = (u8 *)ioat_chan->descs[chunk].virt + offs; > phys = ioat_chan->descs[chunk].hw + offs; > @@ -370,7 +370,8 @@ struct ioat_ring_ent ** > if (!ring) > return NULL; > > - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; > + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; > + ioat_chan->desc_chunks = chunks; > > for (i = 0; i < chunks; i++) { > struct ioat_descs *descs = &ioat_chan->descs[i]; > @@ -382,8 +383,9 @@ struct ioat_ring_ent ** > > for (idx = 0; idx < i; idx++) { > descs = &ioat_chan->descs[idx]; > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > - descs->virt, descs->hw); > + dma_free_coherent(to_dev(ioat_chan), > + IOAT_CHUNK_SIZE, > + descs->virt, descs->hw); > descs->virt = NULL; > descs->hw = 0; > } > @@ -404,7 +406,7 @@ struct ioat_ring_ent ** > > for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { > dma_free_coherent(to_dev(ioat_chan), > - SZ_2M, > + IOAT_CHUNK_SIZE, > ioat_chan->descs[idx].virt, > ioat_chan->descs[idx].hw); > ioat_chan->descs[idx].virt = NULL; > diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h > index b8e8e0b..5216c6b 100644 > --- a/drivers/dma/ioat/dma.h > +++ b/drivers/dma/ioat/dma.h > @@ -81,6 +81,11 @@ struct ioatdma_device { > u32 msixpba; > }; > > +#define IOAT_MAX_ORDER 16 > +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) > +#define IOAT_CHUNK_SIZE (SZ_2M) > +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) > + > struct ioat_descs { > void *virt; > dma_addr_t hw; > @@ -128,7 +133,7 @@ struct ioatdma_chan { > u16 produce; > struct ioat_ring_ent **ring; > spinlock_t prep_lock; > - struct ioat_descs descs[2]; > + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK]; > int desc_chunks; > int intr_coalesce; > int prev_intr_coalesce; > @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) > return !!err; > } > > -#define IOAT_MAX_ORDER 16 > -#define IOAT_MAX_DESCS 65536 > -#define IOAT_DESCS_PER_2M 32768 > > static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) > { > diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c > index 60e9afb..58d1356 100644 > --- a/drivers/dma/ioat/init.c > +++ b/drivers/dma/ioat/init.c > @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) > } > > for (i = 0; i < ioat_chan->desc_chunks; i++) { > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, > ioat_chan->descs[i].virt, > ioat_chan->descs[i].hw); > ioat_chan->descs[i].virt = NULL; > ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-02 16:42 ` [PATCH v2 " Dave Jiang @ 2020-04-02 16:46 ` Ravich, Leonid 2020-04-02 16:49 ` Dave Jiang 0 siblings, 1 reply; 22+ messages in thread From: Ravich, Leonid @ 2020-04-02 16:46 UTC (permalink / raw) To: Dave Jiang, dmaengine Cc: lravich, Vinod Koul, Williams, Dan J, Greg Kroah-Hartman, Zavras, Alexios, Barabash, Alexander, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel Sorry Dave , sure was tested on Intel Sky Lake-E CBDMA -----Original Message----- From: Dave Jiang <dave.jiang@intel.com> Sent: Thursday, April 2, 2020 7:43 PM To: Ravich, Leonid; dmaengine@vger.kernel.org Cc: lravich@gmail.com; Vinod Koul; Williams, Dan J; Greg Kroah-Hartman; Zavras, Alexios; Barabash, Alexander; Thomas Gleixner; Kate Stewart; Jilayne Lovejoy; Logan Gunthorpe; linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency [EXTERNAL EMAIL] On 4/2/2020 9:33 AM, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > prepare for changing alloc size. > > Acked-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> Hi Leonid, I haven't actually acked this patch yet, pending your answer on if this has been tested on hardware. Thanks. > --- > drivers/dma/ioat/dma.c | 14 ++++++++------ > drivers/dma/ioat/dma.h | 10 ++++++---- > drivers/dma/ioat/init.c | 2 +- > 3 files changed, 15 insertions(+), 11 deletions(-) > > diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index > 18c011e..1e0e6c1 100644 > --- a/drivers/dma/ioat/dma.c > +++ b/drivers/dma/ioat/dma.c > @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) > u8 *pos; > off_t offs; > > - chunk = idx / IOAT_DESCS_PER_2M; > - idx &= (IOAT_DESCS_PER_2M - 1); > + chunk = idx / IOAT_DESCS_PER_CHUNK; > + idx &= (IOAT_DESCS_PER_CHUNK - 1); > offs = idx * IOAT_DESC_SZ; > pos = (u8 *)ioat_chan->descs[chunk].virt + offs; > phys = ioat_chan->descs[chunk].hw + offs; @@ -370,7 +370,8 @@ > struct ioat_ring_ent ** > if (!ring) > return NULL; > > - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; > + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; > + ioat_chan->desc_chunks = chunks; > > for (i = 0; i < chunks; i++) { > struct ioat_descs *descs = &ioat_chan->descs[i]; @@ -382,8 +383,9 > @@ struct ioat_ring_ent ** > > for (idx = 0; idx < i; idx++) { > descs = &ioat_chan->descs[idx]; > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > - descs->virt, descs->hw); > + dma_free_coherent(to_dev(ioat_chan), > + IOAT_CHUNK_SIZE, > + descs->virt, descs->hw); > descs->virt = NULL; > descs->hw = 0; > } > @@ -404,7 +406,7 @@ struct ioat_ring_ent ** > > for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { > dma_free_coherent(to_dev(ioat_chan), > - SZ_2M, > + IOAT_CHUNK_SIZE, > ioat_chan->descs[idx].virt, > ioat_chan->descs[idx].hw); > ioat_chan->descs[idx].virt = NULL; diff --git > a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index > b8e8e0b..5216c6b 100644 > --- a/drivers/dma/ioat/dma.h > +++ b/drivers/dma/ioat/dma.h > @@ -81,6 +81,11 @@ struct ioatdma_device { > u32 msixpba; > }; > > +#define IOAT_MAX_ORDER 16 > +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) #define IOAT_CHUNK_SIZE > +(SZ_2M) #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) > + > struct ioat_descs { > void *virt; > dma_addr_t hw; > @@ -128,7 +133,7 @@ struct ioatdma_chan { > u16 produce; > struct ioat_ring_ent **ring; > spinlock_t prep_lock; > - struct ioat_descs descs[2]; > + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK]; > int desc_chunks; > int intr_coalesce; > int prev_intr_coalesce; > @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) > return !!err; > } > > -#define IOAT_MAX_ORDER 16 > -#define IOAT_MAX_DESCS 65536 > -#define IOAT_DESCS_PER_2M 32768 > > static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) > { > diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c index > 60e9afb..58d1356 100644 > --- a/drivers/dma/ioat/init.c > +++ b/drivers/dma/ioat/init.c > @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) > } > > for (i = 0; i < ioat_chan->desc_chunks; i++) { > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, > ioat_chan->descs[i].virt, > ioat_chan->descs[i].hw); > ioat_chan->descs[i].virt = NULL; > ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-02 16:46 ` Ravich, Leonid @ 2020-04-02 16:49 ` Dave Jiang 0 siblings, 0 replies; 22+ messages in thread From: Dave Jiang @ 2020-04-02 16:49 UTC (permalink / raw) To: Ravich, Leonid, dmaengine Cc: lravich, Vinod Koul, Williams, Dan J, Greg Kroah-Hartman, Zavras, Alexios, Barabash, Alexander, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel On 4/2/2020 9:46 AM, Ravich, Leonid wrote: > Sorry Dave , > sure was tested on Intel Sky Lake-E CBDMA Thanks Leonid. Acked. > > -----Original Message----- > From: Dave Jiang <dave.jiang@intel.com> > Sent: Thursday, April 2, 2020 7:43 PM > To: Ravich, Leonid; dmaengine@vger.kernel.org > Cc: lravich@gmail.com; Vinod Koul; Williams, Dan J; Greg Kroah-Hartman; Zavras, Alexios; Barabash, Alexander; Thomas Gleixner; Kate Stewart; Jilayne Lovejoy; Logan Gunthorpe; linux-kernel@vger.kernel.org > Subject: Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency > > > [EXTERNAL EMAIL] > > > > On 4/2/2020 9:33 AM, leonid.ravich@dell.com wrote: >> From: Leonid Ravich <Leonid.Ravich@emc.com> >> >> prepare for changing alloc size. >> >> Acked-by: Dave Jiang <dave.jiang@intel.com> >> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> > > Hi Leonid, I haven't actually acked this patch yet, pending your answer on if this has been tested on hardware. Thanks. > >> --- >> drivers/dma/ioat/dma.c | 14 ++++++++------ >> drivers/dma/ioat/dma.h | 10 ++++++---- >> drivers/dma/ioat/init.c | 2 +- >> 3 files changed, 15 insertions(+), 11 deletions(-) >> >> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index >> 18c011e..1e0e6c1 100644 >> --- a/drivers/dma/ioat/dma.c >> +++ b/drivers/dma/ioat/dma.c >> @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) >> u8 *pos; >> off_t offs; >> >> - chunk = idx / IOAT_DESCS_PER_2M; >> - idx &= (IOAT_DESCS_PER_2M - 1); >> + chunk = idx / IOAT_DESCS_PER_CHUNK; >> + idx &= (IOAT_DESCS_PER_CHUNK - 1); >> offs = idx * IOAT_DESC_SZ; >> pos = (u8 *)ioat_chan->descs[chunk].virt + offs; >> phys = ioat_chan->descs[chunk].hw + offs; @@ -370,7 +370,8 @@ >> struct ioat_ring_ent ** >> if (!ring) >> return NULL; >> >> - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; >> + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; >> + ioat_chan->desc_chunks = chunks; >> >> for (i = 0; i < chunks; i++) { >> struct ioat_descs *descs = &ioat_chan->descs[i]; @@ -382,8 +383,9 >> @@ struct ioat_ring_ent ** >> >> for (idx = 0; idx < i; idx++) { >> descs = &ioat_chan->descs[idx]; >> - dma_free_coherent(to_dev(ioat_chan), SZ_2M, >> - descs->virt, descs->hw); >> + dma_free_coherent(to_dev(ioat_chan), >> + IOAT_CHUNK_SIZE, >> + descs->virt, descs->hw); >> descs->virt = NULL; >> descs->hw = 0; >> } >> @@ -404,7 +406,7 @@ struct ioat_ring_ent ** >> >> for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { >> dma_free_coherent(to_dev(ioat_chan), >> - SZ_2M, >> + IOAT_CHUNK_SIZE, >> ioat_chan->descs[idx].virt, >> ioat_chan->descs[idx].hw); >> ioat_chan->descs[idx].virt = NULL; diff --git >> a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index >> b8e8e0b..5216c6b 100644 >> --- a/drivers/dma/ioat/dma.h >> +++ b/drivers/dma/ioat/dma.h >> @@ -81,6 +81,11 @@ struct ioatdma_device { >> u32 msixpba; >> }; >> >> +#define IOAT_MAX_ORDER 16 >> +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) #define IOAT_CHUNK_SIZE >> +(SZ_2M) #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) >> + >> struct ioat_descs { >> void *virt; >> dma_addr_t hw; >> @@ -128,7 +133,7 @@ struct ioatdma_chan { >> u16 produce; >> struct ioat_ring_ent **ring; >> spinlock_t prep_lock; >> - struct ioat_descs descs[2]; >> + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK]; >> int desc_chunks; >> int intr_coalesce; >> int prev_intr_coalesce; >> @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) >> return !!err; >> } >> >> -#define IOAT_MAX_ORDER 16 >> -#define IOAT_MAX_DESCS 65536 >> -#define IOAT_DESCS_PER_2M 32768 >> >> static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) >> { >> diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c index >> 60e9afb..58d1356 100644 >> --- a/drivers/dma/ioat/init.c >> +++ b/drivers/dma/ioat/init.c >> @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) >> } >> >> for (i = 0; i < ioat_chan->desc_chunks; i++) { >> - dma_free_coherent(to_dev(ioat_chan), SZ_2M, >> + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, >> ioat_chan->descs[i].virt, >> ioat_chan->descs[i].hw); >> ioat_chan->descs[i].virt = NULL; >> ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich 2020-04-02 16:42 ` [PATCH v2 " Dave Jiang @ 2020-04-15 15:52 ` Vinod Koul 2 siblings, 0 replies; 22+ messages in thread From: Vinod Koul @ 2020-04-15 15:52 UTC (permalink / raw) To: leonid.ravich Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel On 02-04-20, 19:33, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > prepare for changing alloc size. This does not tell what the change is doing. A patch should describe the change... pls explain the change is size here > > Acked-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> > --- > drivers/dma/ioat/dma.c | 14 ++++++++------ > drivers/dma/ioat/dma.h | 10 ++++++---- > drivers/dma/ioat/init.c | 2 +- > 3 files changed, 15 insertions(+), 11 deletions(-) > > diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c > index 18c011e..1e0e6c1 100644 > --- a/drivers/dma/ioat/dma.c > +++ b/drivers/dma/ioat/dma.c > @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) > u8 *pos; > off_t offs; > > - chunk = idx / IOAT_DESCS_PER_2M; > - idx &= (IOAT_DESCS_PER_2M - 1); > + chunk = idx / IOAT_DESCS_PER_CHUNK; > + idx &= (IOAT_DESCS_PER_CHUNK - 1); > offs = idx * IOAT_DESC_SZ; > pos = (u8 *)ioat_chan->descs[chunk].virt + offs; > phys = ioat_chan->descs[chunk].hw + offs; > @@ -370,7 +370,8 @@ struct ioat_ring_ent ** > if (!ring) > return NULL; > > - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; > + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; > + ioat_chan->desc_chunks = chunks; > > for (i = 0; i < chunks; i++) { > struct ioat_descs *descs = &ioat_chan->descs[i]; > @@ -382,8 +383,9 @@ struct ioat_ring_ent ** > > for (idx = 0; idx < i; idx++) { > descs = &ioat_chan->descs[idx]; > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > - descs->virt, descs->hw); > + dma_free_coherent(to_dev(ioat_chan), > + IOAT_CHUNK_SIZE, > + descs->virt, descs->hw); > descs->virt = NULL; > descs->hw = 0; > } > @@ -404,7 +406,7 @@ struct ioat_ring_ent ** > > for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { > dma_free_coherent(to_dev(ioat_chan), > - SZ_2M, > + IOAT_CHUNK_SIZE, > ioat_chan->descs[idx].virt, > ioat_chan->descs[idx].hw); > ioat_chan->descs[idx].virt = NULL; > diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h > index b8e8e0b..5216c6b 100644 > --- a/drivers/dma/ioat/dma.h > +++ b/drivers/dma/ioat/dma.h > @@ -81,6 +81,11 @@ struct ioatdma_device { > u32 msixpba; > }; > > +#define IOAT_MAX_ORDER 16 > +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) > +#define IOAT_CHUNK_SIZE (SZ_2M) > +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) > + > struct ioat_descs { > void *virt; > dma_addr_t hw; > @@ -128,7 +133,7 @@ struct ioatdma_chan { > u16 produce; > struct ioat_ring_ent **ring; > spinlock_t prep_lock; > - struct ioat_descs descs[2]; > + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK]; > int desc_chunks; > int intr_coalesce; > int prev_intr_coalesce; > @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) > return !!err; > } > > -#define IOAT_MAX_ORDER 16 > -#define IOAT_MAX_DESCS 65536 > -#define IOAT_DESCS_PER_2M 32768 > > static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) > { > diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c > index 60e9afb..58d1356 100644 > --- a/drivers/dma/ioat/init.c > +++ b/drivers/dma/ioat/init.c > @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) > } > > for (i = 0; i < ioat_chan->desc_chunks; i++) { > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, > ioat_chan->descs[i].virt, > ioat_chan->descs[i].hw); > ioat_chan->descs[i].virt = NULL; > -- > 1.9.3 -- ~Vinod ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency 2020-04-02 9:27 [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich @ 2020-04-02 15:04 ` Dave Jiang 1 sibling, 0 replies; 22+ messages in thread From: Dave Jiang @ 2020-04-02 15:04 UTC (permalink / raw) To: leonid.ravich, dmaengine Cc: lravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash, Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel On 4/2/2020 2:27 AM, leonid.ravich@dell.com wrote: > From: Leonid Ravich <Leonid.Ravich@emc.com> > > prepare for changing alloc size. > > Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com> I'm ok with the changes in the series. Were you able to test this on hardware? A few formating nits below > --- > drivers/dma/ioat/dma.c | 14 ++++++++------ > drivers/dma/ioat/dma.h | 10 ++++++---- > drivers/dma/ioat/init.c | 2 +- > 3 files changed, 15 insertions(+), 11 deletions(-) > > diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c > index 18c011e..1e0e6c1 100644 > --- a/drivers/dma/ioat/dma.c > +++ b/drivers/dma/ioat/dma.c > @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx) > u8 *pos; > off_t offs; > > - chunk = idx / IOAT_DESCS_PER_2M; > - idx &= (IOAT_DESCS_PER_2M - 1); > + chunk = idx / IOAT_DESCS_PER_CHUNK; > + idx &= (IOAT_DESCS_PER_CHUNK - 1); > offs = idx * IOAT_DESC_SZ; > pos = (u8 *)ioat_chan->descs[chunk].virt + offs; > phys = ioat_chan->descs[chunk].hw + offs; > @@ -370,7 +370,8 @@ struct ioat_ring_ent ** > if (!ring) > return NULL; > > - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M; > + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE; > + ioat_chan->desc_chunks = chunks; > > for (i = 0; i < chunks; i++) { > struct ioat_descs *descs = &ioat_chan->descs[i]; > @@ -382,8 +383,9 @@ struct ioat_ring_ent ** > > for (idx = 0; idx < i; idx++) { > descs = &ioat_chan->descs[idx]; > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > - descs->virt, descs->hw); > + dma_free_coherent(to_dev(ioat_chan), > + IOAT_CHUNK_SIZE, > + descs->virt, descs->hw); > descs->virt = NULL; > descs->hw = 0; > } > @@ -404,7 +406,7 @@ struct ioat_ring_ent ** > > for (idx = 0; idx < ioat_chan->desc_chunks; idx++) { > dma_free_coherent(to_dev(ioat_chan), > - SZ_2M, > + IOAT_CHUNK_SIZE, > ioat_chan->descs[idx].virt, > ioat_chan->descs[idx].hw); > ioat_chan->descs[idx].virt = NULL; > diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h > index b8e8e0b..535aba9 100644 > --- a/drivers/dma/ioat/dma.h > +++ b/drivers/dma/ioat/dma.h > @@ -81,6 +81,11 @@ struct ioatdma_device { > u32 msixpba; > }; > > +#define IOAT_MAX_ORDER 16 > +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) > +#define IOAT_CHUNK_SIZE (SZ_2M) > +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ) (IOAT_CHUNK_SIZE / IOAT_DESC_SZ) > + > struct ioat_descs { > void *virt; > dma_addr_t hw; > @@ -128,7 +133,7 @@ struct ioatdma_chan { > u16 produce; > struct ioat_ring_ent **ring; > spinlock_t prep_lock; > - struct ioat_descs descs[2]; > + struct ioat_descs descs[IOAT_MAX_DESCS/IOAT_DESCS_PER_CHUNK]; IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK > int desc_chunks; > int intr_coalesce; > int prev_intr_coalesce; > @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err) > return !!err; > } > > -#define IOAT_MAX_ORDER 16 > -#define IOAT_MAX_DESCS 65536 > -#define IOAT_DESCS_PER_2M 32768 > > static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan) > { > diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c > index 60e9afb..58d1356 100644 > --- a/drivers/dma/ioat/init.c > +++ b/drivers/dma/ioat/init.c > @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c) > } > > for (i = 0; i < ioat_chan->desc_chunks; i++) { > - dma_free_coherent(to_dev(ioat_chan), SZ_2M, > + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE, > ioat_chan->descs[i].virt, > ioat_chan->descs[i].hw); > ioat_chan->descs[i].virt = NULL; > ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2020-04-23 7:22 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-04-02 9:27 [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich 2020-04-02 15:06 ` Dave Jiang 2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich 2020-04-15 15:53 ` Vinod Koul 2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich 2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich 2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich 2020-04-22 19:25 ` [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich 2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich 2020-04-22 19:58 ` Dave Jiang 2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich 2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich 2020-04-22 21:09 ` [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich 2020-04-23 7:21 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " Vinod Koul 2020-04-17 11:58 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency Vinod Koul 2020-04-02 16:42 ` [PATCH v2 " Dave Jiang 2020-04-02 16:46 ` Ravich, Leonid 2020-04-02 16:49 ` Dave Jiang 2020-04-15 15:52 ` Vinod Koul 2020-04-02 15:04 ` [PATCH " Dave Jiang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.