* [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
@ 2020-04-02 9:27 leonid.ravich
2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
2020-04-02 15:04 ` [PATCH " Dave Jiang
0 siblings, 2 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-02 9:27 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams,
Greg Kroah-Hartman, Alexios Zavras, Dave Jiang,
Alexander.Barabash, Thomas Gleixner, Kate Stewart,
Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
prepare for changing alloc size.
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.c | 14 ++++++++------
drivers/dma/ioat/dma.h | 10 ++++++----
drivers/dma/ioat/init.c | 2 +-
3 files changed, 15 insertions(+), 11 deletions(-)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index 18c011e..1e0e6c1 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
u8 *pos;
off_t offs;
- chunk = idx / IOAT_DESCS_PER_2M;
- idx &= (IOAT_DESCS_PER_2M - 1);
+ chunk = idx / IOAT_DESCS_PER_CHUNK;
+ idx &= (IOAT_DESCS_PER_CHUNK - 1);
offs = idx * IOAT_DESC_SZ;
pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
phys = ioat_chan->descs[chunk].hw + offs;
@@ -370,7 +370,8 @@ struct ioat_ring_ent **
if (!ring)
return NULL;
- ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
+ chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
+ ioat_chan->desc_chunks = chunks;
for (i = 0; i < chunks; i++) {
struct ioat_descs *descs = &ioat_chan->descs[i];
@@ -382,8 +383,9 @@ struct ioat_ring_ent **
for (idx = 0; idx < i; idx++) {
descs = &ioat_chan->descs[idx];
- dma_free_coherent(to_dev(ioat_chan), SZ_2M,
- descs->virt, descs->hw);
+ dma_free_coherent(to_dev(ioat_chan),
+ IOAT_CHUNK_SIZE,
+ descs->virt, descs->hw);
descs->virt = NULL;
descs->hw = 0;
}
@@ -404,7 +406,7 @@ struct ioat_ring_ent **
for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
dma_free_coherent(to_dev(ioat_chan),
- SZ_2M,
+ IOAT_CHUNK_SIZE,
ioat_chan->descs[idx].virt,
ioat_chan->descs[idx].hw);
ioat_chan->descs[idx].virt = NULL;
diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
index b8e8e0b..535aba9 100644
--- a/drivers/dma/ioat/dma.h
+++ b/drivers/dma/ioat/dma.h
@@ -81,6 +81,11 @@ struct ioatdma_device {
u32 msixpba;
};
+#define IOAT_MAX_ORDER 16
+#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
+#define IOAT_CHUNK_SIZE (SZ_2M)
+#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ)
+
struct ioat_descs {
void *virt;
dma_addr_t hw;
@@ -128,7 +133,7 @@ struct ioatdma_chan {
u16 produce;
struct ioat_ring_ent **ring;
spinlock_t prep_lock;
- struct ioat_descs descs[2];
+ struct ioat_descs descs[IOAT_MAX_DESCS/IOAT_DESCS_PER_CHUNK];
int desc_chunks;
int intr_coalesce;
int prev_intr_coalesce;
@@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
return !!err;
}
-#define IOAT_MAX_ORDER 16
-#define IOAT_MAX_DESCS 65536
-#define IOAT_DESCS_PER_2M 32768
static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
{
diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
index 60e9afb..58d1356 100644
--- a/drivers/dma/ioat/init.c
+++ b/drivers/dma/ioat/init.c
@@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
}
for (i = 0; i < ioat_chan->desc_chunks; i++) {
- dma_free_coherent(to_dev(ioat_chan), SZ_2M,
+ dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
ioat_chan->descs[i].virt,
ioat_chan->descs[i].hw);
ioat_chan->descs[i].virt = NULL;
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K
2020-04-02 9:27 [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
@ 2020-04-02 9:27 ` leonid.ravich
2020-04-02 15:06 ` Dave Jiang
2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-02 15:04 ` [PATCH " Dave Jiang
1 sibling, 2 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-02 9:27 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams,
Greg Kroah-Hartman, Alexios Zavras, Dave Jiang,
Alexander.Barabash, Thomas Gleixner, Kate Stewart,
Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
current IOAT driver using big (2MB) allocations chunk for its descriptors
therefore each ioat dma engine need 2 such chunks
(64k entres in ring each entry 64B = 4MB)
requiring 2 * 2M * dmaengine contiguies memory chunk
might fail due to memory fragmention.
so we decresging chunk size and using more chunks.
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
index 535aba9..e9757bc 100644
--- a/drivers/dma/ioat/dma.h
+++ b/drivers/dma/ioat/dma.h
@@ -83,7 +83,7 @@ struct ioatdma_device {
#define IOAT_MAX_ORDER 16
#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
-#define IOAT_CHUNK_SIZE (SZ_2M)
+#define IOAT_CHUNK_SIZE (SZ_512K)
#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ)
struct ioat_descs {
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-02 9:27 [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
@ 2020-04-02 15:04 ` Dave Jiang
1 sibling, 0 replies; 22+ messages in thread
From: Dave Jiang @ 2020-04-02 15:04 UTC (permalink / raw)
To: leonid.ravich, dmaengine
Cc: lravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman,
Alexios Zavras, Alexander.Barabash, Thomas Gleixner,
Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
On 4/2/2020 2:27 AM, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> prepare for changing alloc size.
>
> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
I'm ok with the changes in the series. Were you able to test this on
hardware? A few formating nits below
> ---
> drivers/dma/ioat/dma.c | 14 ++++++++------
> drivers/dma/ioat/dma.h | 10 ++++++----
> drivers/dma/ioat/init.c | 2 +-
> 3 files changed, 15 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
> index 18c011e..1e0e6c1 100644
> --- a/drivers/dma/ioat/dma.c
> +++ b/drivers/dma/ioat/dma.c
> @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
> u8 *pos;
> off_t offs;
>
> - chunk = idx / IOAT_DESCS_PER_2M;
> - idx &= (IOAT_DESCS_PER_2M - 1);
> + chunk = idx / IOAT_DESCS_PER_CHUNK;
> + idx &= (IOAT_DESCS_PER_CHUNK - 1);
> offs = idx * IOAT_DESC_SZ;
> pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
> phys = ioat_chan->descs[chunk].hw + offs;
> @@ -370,7 +370,8 @@ struct ioat_ring_ent **
> if (!ring)
> return NULL;
>
> - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
> + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
> + ioat_chan->desc_chunks = chunks;
>
> for (i = 0; i < chunks; i++) {
> struct ioat_descs *descs = &ioat_chan->descs[i];
> @@ -382,8 +383,9 @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < i; idx++) {
> descs = &ioat_chan->descs[idx];
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> - descs->virt, descs->hw);
> + dma_free_coherent(to_dev(ioat_chan),
> + IOAT_CHUNK_SIZE,
> + descs->virt, descs->hw);
> descs->virt = NULL;
> descs->hw = 0;
> }
> @@ -404,7 +406,7 @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
> dma_free_coherent(to_dev(ioat_chan),
> - SZ_2M,
> + IOAT_CHUNK_SIZE,
> ioat_chan->descs[idx].virt,
> ioat_chan->descs[idx].hw);
> ioat_chan->descs[idx].virt = NULL;
> diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
> index b8e8e0b..535aba9 100644
> --- a/drivers/dma/ioat/dma.h
> +++ b/drivers/dma/ioat/dma.h
> @@ -81,6 +81,11 @@ struct ioatdma_device {
> u32 msixpba;
> };
>
> +#define IOAT_MAX_ORDER 16
> +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
> +#define IOAT_CHUNK_SIZE (SZ_2M)
> +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ)
(IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
> +
> struct ioat_descs {
> void *virt;
> dma_addr_t hw;
> @@ -128,7 +133,7 @@ struct ioatdma_chan {
> u16 produce;
> struct ioat_ring_ent **ring;
> spinlock_t prep_lock;
> - struct ioat_descs descs[2];
> + struct ioat_descs descs[IOAT_MAX_DESCS/IOAT_DESCS_PER_CHUNK];
IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK
> int desc_chunks;
> int intr_coalesce;
> int prev_intr_coalesce;
> @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
> return !!err;
> }
>
> -#define IOAT_MAX_ORDER 16
> -#define IOAT_MAX_DESCS 65536
> -#define IOAT_DESCS_PER_2M 32768
>
> static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
> {
> diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
> index 60e9afb..58d1356 100644
> --- a/drivers/dma/ioat/init.c
> +++ b/drivers/dma/ioat/init.c
> @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
> }
>
> for (i = 0; i < ioat_chan->desc_chunks; i++) {
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
> ioat_chan->descs[i].virt,
> ioat_chan->descs[i].hw);
> ioat_chan->descs[i].virt = NULL;
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K
2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
@ 2020-04-02 15:06 ` Dave Jiang
2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
1 sibling, 0 replies; 22+ messages in thread
From: Dave Jiang @ 2020-04-02 15:06 UTC (permalink / raw)
To: leonid.ravich, dmaengine
Cc: lravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman,
Alexios Zavras, Alexander.Barabash, Thomas Gleixner,
Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
On 4/2/2020 2:27 AM, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> current IOAT driver using big (2MB) allocations chunk for its descriptors
> therefore each ioat dma engine need 2 such chunks
> (64k entres in ring each entry 64B = 4MB)
> requiring 2 * 2M * dmaengine contiguies memory chunk
> might fail due to memory fragmention.
>
> so we decresging chunk size and using more chunks.
decreasing
>
> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
Acked-by: Dave Jiang <dave.jiang@intel.com> if the two patches have been
tested on hw.
> ---
> drivers/dma/ioat/dma.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
> index 535aba9..e9757bc 100644
> --- a/drivers/dma/ioat/dma.h
> +++ b/drivers/dma/ioat/dma.h
> @@ -83,7 +83,7 @@ struct ioatdma_device {
>
> #define IOAT_MAX_ORDER 16
> #define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
> -#define IOAT_CHUNK_SIZE (SZ_2M)
> +#define IOAT_CHUNK_SIZE (SZ_512K)
> #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE/IOAT_DESC_SZ)
>
> struct ioat_descs {
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
2020-04-02 15:06 ` Dave Jiang
@ 2020-04-02 16:33 ` leonid.ravich
2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
` (2 more replies)
1 sibling, 3 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-02 16:33 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang,
Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash,
Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe,
linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
prepare for changing alloc size.
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.c | 14 ++++++++------
drivers/dma/ioat/dma.h | 10 ++++++----
drivers/dma/ioat/init.c | 2 +-
3 files changed, 15 insertions(+), 11 deletions(-)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index 18c011e..1e0e6c1 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
u8 *pos;
off_t offs;
- chunk = idx / IOAT_DESCS_PER_2M;
- idx &= (IOAT_DESCS_PER_2M - 1);
+ chunk = idx / IOAT_DESCS_PER_CHUNK;
+ idx &= (IOAT_DESCS_PER_CHUNK - 1);
offs = idx * IOAT_DESC_SZ;
pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
phys = ioat_chan->descs[chunk].hw + offs;
@@ -370,7 +370,8 @@ struct ioat_ring_ent **
if (!ring)
return NULL;
- ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
+ chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
+ ioat_chan->desc_chunks = chunks;
for (i = 0; i < chunks; i++) {
struct ioat_descs *descs = &ioat_chan->descs[i];
@@ -382,8 +383,9 @@ struct ioat_ring_ent **
for (idx = 0; idx < i; idx++) {
descs = &ioat_chan->descs[idx];
- dma_free_coherent(to_dev(ioat_chan), SZ_2M,
- descs->virt, descs->hw);
+ dma_free_coherent(to_dev(ioat_chan),
+ IOAT_CHUNK_SIZE,
+ descs->virt, descs->hw);
descs->virt = NULL;
descs->hw = 0;
}
@@ -404,7 +406,7 @@ struct ioat_ring_ent **
for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
dma_free_coherent(to_dev(ioat_chan),
- SZ_2M,
+ IOAT_CHUNK_SIZE,
ioat_chan->descs[idx].virt,
ioat_chan->descs[idx].hw);
ioat_chan->descs[idx].virt = NULL;
diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
index b8e8e0b..5216c6b 100644
--- a/drivers/dma/ioat/dma.h
+++ b/drivers/dma/ioat/dma.h
@@ -81,6 +81,11 @@ struct ioatdma_device {
u32 msixpba;
};
+#define IOAT_MAX_ORDER 16
+#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
+#define IOAT_CHUNK_SIZE (SZ_2M)
+#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
+
struct ioat_descs {
void *virt;
dma_addr_t hw;
@@ -128,7 +133,7 @@ struct ioatdma_chan {
u16 produce;
struct ioat_ring_ent **ring;
spinlock_t prep_lock;
- struct ioat_descs descs[2];
+ struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK];
int desc_chunks;
int intr_coalesce;
int prev_intr_coalesce;
@@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
return !!err;
}
-#define IOAT_MAX_ORDER 16
-#define IOAT_MAX_DESCS 65536
-#define IOAT_DESCS_PER_2M 32768
static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
{
diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
index 60e9afb..58d1356 100644
--- a/drivers/dma/ioat/init.c
+++ b/drivers/dma/ioat/init.c
@@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
}
for (i = 0; i < ioat_chan->desc_chunks; i++) {
- dma_free_coherent(to_dev(ioat_chan), SZ_2M,
+ dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
ioat_chan->descs[i].virt,
ioat_chan->descs[i].hw);
ioat_chan->descs[i].virt = NULL;
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K
2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
@ 2020-04-02 16:33 ` leonid.ravich
2020-04-15 15:53 ` Vinod Koul
2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-02 16:42 ` [PATCH v2 " Dave Jiang
2020-04-15 15:52 ` Vinod Koul
2 siblings, 2 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-02 16:33 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang,
Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash,
Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe,
linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
current IOAT driver using big (2MB) allocations chunk for its descriptors
therefore each ioat dma engine need 2 such chunks
(64k entres in ring each entry 64B = 4MB)
requiring 2 * 2M * dmaengine contiguies memory chunk
might fail due to memory fragmention.
so we decreasing chunk size and using more chunks.
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
index 5216c6b..e6b622e 100644
--- a/drivers/dma/ioat/dma.h
+++ b/drivers/dma/ioat/dma.h
@@ -83,7 +83,7 @@ struct ioatdma_device {
#define IOAT_MAX_ORDER 16
#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
-#define IOAT_CHUNK_SIZE (SZ_2M)
+#define IOAT_CHUNK_SIZE (SZ_512K)
#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
struct ioat_descs {
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
@ 2020-04-02 16:42 ` Dave Jiang
2020-04-02 16:46 ` Ravich, Leonid
2020-04-15 15:52 ` Vinod Koul
2 siblings, 1 reply; 22+ messages in thread
From: Dave Jiang @ 2020-04-02 16:42 UTC (permalink / raw)
To: leonid.ravich, dmaengine
Cc: lravich, Vinod Koul, Williams, Dan J, Greg Kroah-Hartman, Zavras,
Alexios, Alexander.Barabash, Thomas Gleixner, Kate Stewart,
Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
On 4/2/2020 9:33 AM, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> prepare for changing alloc size.
>
> Acked-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
Hi Leonid, I haven't actually acked this patch yet, pending your answer
on if this has been tested on hardware. Thanks.
> ---
> drivers/dma/ioat/dma.c | 14 ++++++++------
> drivers/dma/ioat/dma.h | 10 ++++++----
> drivers/dma/ioat/init.c | 2 +-
> 3 files changed, 15 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
> index 18c011e..1e0e6c1 100644
> --- a/drivers/dma/ioat/dma.c
> +++ b/drivers/dma/ioat/dma.c
> @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
> u8 *pos;
> off_t offs;
>
> - chunk = idx / IOAT_DESCS_PER_2M;
> - idx &= (IOAT_DESCS_PER_2M - 1);
> + chunk = idx / IOAT_DESCS_PER_CHUNK;
> + idx &= (IOAT_DESCS_PER_CHUNK - 1);
> offs = idx * IOAT_DESC_SZ;
> pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
> phys = ioat_chan->descs[chunk].hw + offs;
> @@ -370,7 +370,8 @@ struct ioat_ring_ent **
> if (!ring)
> return NULL;
>
> - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
> + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
> + ioat_chan->desc_chunks = chunks;
>
> for (i = 0; i < chunks; i++) {
> struct ioat_descs *descs = &ioat_chan->descs[i];
> @@ -382,8 +383,9 @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < i; idx++) {
> descs = &ioat_chan->descs[idx];
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> - descs->virt, descs->hw);
> + dma_free_coherent(to_dev(ioat_chan),
> + IOAT_CHUNK_SIZE,
> + descs->virt, descs->hw);
> descs->virt = NULL;
> descs->hw = 0;
> }
> @@ -404,7 +406,7 @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
> dma_free_coherent(to_dev(ioat_chan),
> - SZ_2M,
> + IOAT_CHUNK_SIZE,
> ioat_chan->descs[idx].virt,
> ioat_chan->descs[idx].hw);
> ioat_chan->descs[idx].virt = NULL;
> diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
> index b8e8e0b..5216c6b 100644
> --- a/drivers/dma/ioat/dma.h
> +++ b/drivers/dma/ioat/dma.h
> @@ -81,6 +81,11 @@ struct ioatdma_device {
> u32 msixpba;
> };
>
> +#define IOAT_MAX_ORDER 16
> +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
> +#define IOAT_CHUNK_SIZE (SZ_2M)
> +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
> +
> struct ioat_descs {
> void *virt;
> dma_addr_t hw;
> @@ -128,7 +133,7 @@ struct ioatdma_chan {
> u16 produce;
> struct ioat_ring_ent **ring;
> spinlock_t prep_lock;
> - struct ioat_descs descs[2];
> + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK];
> int desc_chunks;
> int intr_coalesce;
> int prev_intr_coalesce;
> @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
> return !!err;
> }
>
> -#define IOAT_MAX_ORDER 16
> -#define IOAT_MAX_DESCS 65536
> -#define IOAT_DESCS_PER_2M 32768
>
> static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
> {
> diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
> index 60e9afb..58d1356 100644
> --- a/drivers/dma/ioat/init.c
> +++ b/drivers/dma/ioat/init.c
> @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
> }
>
> for (i = 0; i < ioat_chan->desc_chunks; i++) {
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
> ioat_chan->descs[i].virt,
> ioat_chan->descs[i].hw);
> ioat_chan->descs[i].virt = NULL;
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-02 16:42 ` [PATCH v2 " Dave Jiang
@ 2020-04-02 16:46 ` Ravich, Leonid
2020-04-02 16:49 ` Dave Jiang
0 siblings, 1 reply; 22+ messages in thread
From: Ravich, Leonid @ 2020-04-02 16:46 UTC (permalink / raw)
To: Dave Jiang, dmaengine
Cc: lravich, Vinod Koul, Williams, Dan J, Greg Kroah-Hartman, Zavras,
Alexios, Barabash, Alexander, Thomas Gleixner, Kate Stewart,
Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
Sorry Dave ,
sure was tested on Intel Sky Lake-E CBDMA
-----Original Message-----
From: Dave Jiang <dave.jiang@intel.com>
Sent: Thursday, April 2, 2020 7:43 PM
To: Ravich, Leonid; dmaengine@vger.kernel.org
Cc: lravich@gmail.com; Vinod Koul; Williams, Dan J; Greg Kroah-Hartman; Zavras, Alexios; Barabash, Alexander; Thomas Gleixner; Kate Stewart; Jilayne Lovejoy; Logan Gunthorpe; linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
[EXTERNAL EMAIL]
On 4/2/2020 9:33 AM, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> prepare for changing alloc size.
>
> Acked-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
Hi Leonid, I haven't actually acked this patch yet, pending your answer on if this has been tested on hardware. Thanks.
> ---
> drivers/dma/ioat/dma.c | 14 ++++++++------
> drivers/dma/ioat/dma.h | 10 ++++++----
> drivers/dma/ioat/init.c | 2 +-
> 3 files changed, 15 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index
> 18c011e..1e0e6c1 100644
> --- a/drivers/dma/ioat/dma.c
> +++ b/drivers/dma/ioat/dma.c
> @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
> u8 *pos;
> off_t offs;
>
> - chunk = idx / IOAT_DESCS_PER_2M;
> - idx &= (IOAT_DESCS_PER_2M - 1);
> + chunk = idx / IOAT_DESCS_PER_CHUNK;
> + idx &= (IOAT_DESCS_PER_CHUNK - 1);
> offs = idx * IOAT_DESC_SZ;
> pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
> phys = ioat_chan->descs[chunk].hw + offs; @@ -370,7 +370,8 @@
> struct ioat_ring_ent **
> if (!ring)
> return NULL;
>
> - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
> + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
> + ioat_chan->desc_chunks = chunks;
>
> for (i = 0; i < chunks; i++) {
> struct ioat_descs *descs = &ioat_chan->descs[i]; @@ -382,8 +383,9
> @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < i; idx++) {
> descs = &ioat_chan->descs[idx];
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> - descs->virt, descs->hw);
> + dma_free_coherent(to_dev(ioat_chan),
> + IOAT_CHUNK_SIZE,
> + descs->virt, descs->hw);
> descs->virt = NULL;
> descs->hw = 0;
> }
> @@ -404,7 +406,7 @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
> dma_free_coherent(to_dev(ioat_chan),
> - SZ_2M,
> + IOAT_CHUNK_SIZE,
> ioat_chan->descs[idx].virt,
> ioat_chan->descs[idx].hw);
> ioat_chan->descs[idx].virt = NULL; diff --git
> a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index
> b8e8e0b..5216c6b 100644
> --- a/drivers/dma/ioat/dma.h
> +++ b/drivers/dma/ioat/dma.h
> @@ -81,6 +81,11 @@ struct ioatdma_device {
> u32 msixpba;
> };
>
> +#define IOAT_MAX_ORDER 16
> +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) #define IOAT_CHUNK_SIZE
> +(SZ_2M) #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
> +
> struct ioat_descs {
> void *virt;
> dma_addr_t hw;
> @@ -128,7 +133,7 @@ struct ioatdma_chan {
> u16 produce;
> struct ioat_ring_ent **ring;
> spinlock_t prep_lock;
> - struct ioat_descs descs[2];
> + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK];
> int desc_chunks;
> int intr_coalesce;
> int prev_intr_coalesce;
> @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
> return !!err;
> }
>
> -#define IOAT_MAX_ORDER 16
> -#define IOAT_MAX_DESCS 65536
> -#define IOAT_DESCS_PER_2M 32768
>
> static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
> {
> diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c index
> 60e9afb..58d1356 100644
> --- a/drivers/dma/ioat/init.c
> +++ b/drivers/dma/ioat/init.c
> @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
> }
>
> for (i = 0; i < ioat_chan->desc_chunks; i++) {
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
> ioat_chan->descs[i].virt,
> ioat_chan->descs[i].hw);
> ioat_chan->descs[i].virt = NULL;
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-02 16:46 ` Ravich, Leonid
@ 2020-04-02 16:49 ` Dave Jiang
0 siblings, 0 replies; 22+ messages in thread
From: Dave Jiang @ 2020-04-02 16:49 UTC (permalink / raw)
To: Ravich, Leonid, dmaengine
Cc: lravich, Vinod Koul, Williams, Dan J, Greg Kroah-Hartman, Zavras,
Alexios, Barabash, Alexander, Thomas Gleixner, Kate Stewart,
Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
On 4/2/2020 9:46 AM, Ravich, Leonid wrote:
> Sorry Dave ,
> sure was tested on Intel Sky Lake-E CBDMA
Thanks Leonid. Acked.
>
> -----Original Message-----
> From: Dave Jiang <dave.jiang@intel.com>
> Sent: Thursday, April 2, 2020 7:43 PM
> To: Ravich, Leonid; dmaengine@vger.kernel.org
> Cc: lravich@gmail.com; Vinod Koul; Williams, Dan J; Greg Kroah-Hartman; Zavras, Alexios; Barabash, Alexander; Thomas Gleixner; Kate Stewart; Jilayne Lovejoy; Logan Gunthorpe; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
>
>
> [EXTERNAL EMAIL]
>
>
>
> On 4/2/2020 9:33 AM, leonid.ravich@dell.com wrote:
>> From: Leonid Ravich <Leonid.Ravich@emc.com>
>>
>> prepare for changing alloc size.
>>
>> Acked-by: Dave Jiang <dave.jiang@intel.com>
>> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
>
> Hi Leonid, I haven't actually acked this patch yet, pending your answer on if this has been tested on hardware. Thanks.
>
>> ---
>> drivers/dma/ioat/dma.c | 14 ++++++++------
>> drivers/dma/ioat/dma.h | 10 ++++++----
>> drivers/dma/ioat/init.c | 2 +-
>> 3 files changed, 15 insertions(+), 11 deletions(-)
>>
>> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c index
>> 18c011e..1e0e6c1 100644
>> --- a/drivers/dma/ioat/dma.c
>> +++ b/drivers/dma/ioat/dma.c
>> @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
>> u8 *pos;
>> off_t offs;
>>
>> - chunk = idx / IOAT_DESCS_PER_2M;
>> - idx &= (IOAT_DESCS_PER_2M - 1);
>> + chunk = idx / IOAT_DESCS_PER_CHUNK;
>> + idx &= (IOAT_DESCS_PER_CHUNK - 1);
>> offs = idx * IOAT_DESC_SZ;
>> pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
>> phys = ioat_chan->descs[chunk].hw + offs; @@ -370,7 +370,8 @@
>> struct ioat_ring_ent **
>> if (!ring)
>> return NULL;
>>
>> - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
>> + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
>> + ioat_chan->desc_chunks = chunks;
>>
>> for (i = 0; i < chunks; i++) {
>> struct ioat_descs *descs = &ioat_chan->descs[i]; @@ -382,8 +383,9
>> @@ struct ioat_ring_ent **
>>
>> for (idx = 0; idx < i; idx++) {
>> descs = &ioat_chan->descs[idx];
>> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
>> - descs->virt, descs->hw);
>> + dma_free_coherent(to_dev(ioat_chan),
>> + IOAT_CHUNK_SIZE,
>> + descs->virt, descs->hw);
>> descs->virt = NULL;
>> descs->hw = 0;
>> }
>> @@ -404,7 +406,7 @@ struct ioat_ring_ent **
>>
>> for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
>> dma_free_coherent(to_dev(ioat_chan),
>> - SZ_2M,
>> + IOAT_CHUNK_SIZE,
>> ioat_chan->descs[idx].virt,
>> ioat_chan->descs[idx].hw);
>> ioat_chan->descs[idx].virt = NULL; diff --git
>> a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h index
>> b8e8e0b..5216c6b 100644
>> --- a/drivers/dma/ioat/dma.h
>> +++ b/drivers/dma/ioat/dma.h
>> @@ -81,6 +81,11 @@ struct ioatdma_device {
>> u32 msixpba;
>> };
>>
>> +#define IOAT_MAX_ORDER 16
>> +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER) #define IOAT_CHUNK_SIZE
>> +(SZ_2M) #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
>> +
>> struct ioat_descs {
>> void *virt;
>> dma_addr_t hw;
>> @@ -128,7 +133,7 @@ struct ioatdma_chan {
>> u16 produce;
>> struct ioat_ring_ent **ring;
>> spinlock_t prep_lock;
>> - struct ioat_descs descs[2];
>> + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK];
>> int desc_chunks;
>> int intr_coalesce;
>> int prev_intr_coalesce;
>> @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
>> return !!err;
>> }
>>
>> -#define IOAT_MAX_ORDER 16
>> -#define IOAT_MAX_DESCS 65536
>> -#define IOAT_DESCS_PER_2M 32768
>>
>> static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
>> {
>> diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c index
>> 60e9afb..58d1356 100644
>> --- a/drivers/dma/ioat/init.c
>> +++ b/drivers/dma/ioat/init.c
>> @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
>> }
>>
>> for (i = 0; i < ioat_chan->desc_chunks; i++) {
>> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
>> + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
>> ioat_chan->descs[i].virt,
>> ioat_chan->descs[i].hw);
>> ioat_chan->descs[i].virt = NULL;
>>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
2020-04-02 16:42 ` [PATCH v2 " Dave Jiang
@ 2020-04-15 15:52 ` Vinod Koul
2 siblings, 0 replies; 22+ messages in thread
From: Vinod Koul @ 2020-04-15 15:52 UTC (permalink / raw)
To: leonid.ravich
Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Greg Kroah-Hartman,
Alexios Zavras, Alexander.Barabash, Thomas Gleixner,
Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
On 02-04-20, 19:33, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> prepare for changing alloc size.
This does not tell what the change is doing. A patch should describe the
change... pls explain the change is size here
>
> Acked-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
> ---
> drivers/dma/ioat/dma.c | 14 ++++++++------
> drivers/dma/ioat/dma.h | 10 ++++++----
> drivers/dma/ioat/init.c | 2 +-
> 3 files changed, 15 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
> index 18c011e..1e0e6c1 100644
> --- a/drivers/dma/ioat/dma.c
> +++ b/drivers/dma/ioat/dma.c
> @@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
> u8 *pos;
> off_t offs;
>
> - chunk = idx / IOAT_DESCS_PER_2M;
> - idx &= (IOAT_DESCS_PER_2M - 1);
> + chunk = idx / IOAT_DESCS_PER_CHUNK;
> + idx &= (IOAT_DESCS_PER_CHUNK - 1);
> offs = idx * IOAT_DESC_SZ;
> pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
> phys = ioat_chan->descs[chunk].hw + offs;
> @@ -370,7 +370,8 @@ struct ioat_ring_ent **
> if (!ring)
> return NULL;
>
> - ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
> + chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
> + ioat_chan->desc_chunks = chunks;
>
> for (i = 0; i < chunks; i++) {
> struct ioat_descs *descs = &ioat_chan->descs[i];
> @@ -382,8 +383,9 @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < i; idx++) {
> descs = &ioat_chan->descs[idx];
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> - descs->virt, descs->hw);
> + dma_free_coherent(to_dev(ioat_chan),
> + IOAT_CHUNK_SIZE,
> + descs->virt, descs->hw);
> descs->virt = NULL;
> descs->hw = 0;
> }
> @@ -404,7 +406,7 @@ struct ioat_ring_ent **
>
> for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
> dma_free_coherent(to_dev(ioat_chan),
> - SZ_2M,
> + IOAT_CHUNK_SIZE,
> ioat_chan->descs[idx].virt,
> ioat_chan->descs[idx].hw);
> ioat_chan->descs[idx].virt = NULL;
> diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
> index b8e8e0b..5216c6b 100644
> --- a/drivers/dma/ioat/dma.h
> +++ b/drivers/dma/ioat/dma.h
> @@ -81,6 +81,11 @@ struct ioatdma_device {
> u32 msixpba;
> };
>
> +#define IOAT_MAX_ORDER 16
> +#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
> +#define IOAT_CHUNK_SIZE (SZ_2M)
> +#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
> +
> struct ioat_descs {
> void *virt;
> dma_addr_t hw;
> @@ -128,7 +133,7 @@ struct ioatdma_chan {
> u16 produce;
> struct ioat_ring_ent **ring;
> spinlock_t prep_lock;
> - struct ioat_descs descs[2];
> + struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK];
> int desc_chunks;
> int intr_coalesce;
> int prev_intr_coalesce;
> @@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
> return !!err;
> }
>
> -#define IOAT_MAX_ORDER 16
> -#define IOAT_MAX_DESCS 65536
> -#define IOAT_DESCS_PER_2M 32768
>
> static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
> {
> diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
> index 60e9afb..58d1356 100644
> --- a/drivers/dma/ioat/init.c
> +++ b/drivers/dma/ioat/init.c
> @@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
> }
>
> for (i = 0; i < ioat_chan->desc_chunks; i++) {
> - dma_free_coherent(to_dev(ioat_chan), SZ_2M,
> + dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
> ioat_chan->descs[i].virt,
> ioat_chan->descs[i].hw);
> ioat_chan->descs[i].virt = NULL;
> --
> 1.9.3
--
~Vinod
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K
2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
@ 2020-04-15 15:53 ` Vinod Koul
2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
1 sibling, 0 replies; 22+ messages in thread
From: Vinod Koul @ 2020-04-15 15:53 UTC (permalink / raw)
To: leonid.ravich
Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Greg Kroah-Hartman,
Alexios Zavras, Alexander.Barabash, Thomas Gleixner,
Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
On 02-04-20, 19:33, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> current IOAT driver using big (2MB) allocations chunk for its descriptors
> therefore each ioat dma engine need 2 such chunks
> (64k entres in ring each entry 64B = 4MB)
> requiring 2 * 2M * dmaengine contiguies memory chunk
> might fail due to memory fragmention.
This is quite decent explanation :) pls use upto 72 chars to make it a
better read.
>
> so we decreasing chunk size and using more chunks.
>
> Acked-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
> ---
> drivers/dma/ioat/dma.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
> index 5216c6b..e6b622e 100644
> --- a/drivers/dma/ioat/dma.h
> +++ b/drivers/dma/ioat/dma.h
> @@ -83,7 +83,7 @@ struct ioatdma_device {
>
> #define IOAT_MAX_ORDER 16
> #define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
> -#define IOAT_CHUNK_SIZE (SZ_2M)
> +#define IOAT_CHUNK_SIZE (SZ_512K)
> #define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
>
> struct ioat_descs {
> --
> 1.9.3
--
~Vinod
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
2020-04-15 15:53 ` Vinod Koul
@ 2020-04-16 17:06 ` leonid.ravich
2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich
2020-04-17 11:58 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency Vinod Koul
1 sibling, 2 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-16 17:06 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang,
Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash,
Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe,
linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
changing macros which assumption is chunk size of 2M,
which can be other size
prepare for changing allocation chunk size.
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
Changing in v3:
- Make the commit message more clearer.
drivers/dma/ioat/dma.c | 14 ++++++++------
drivers/dma/ioat/dma.h | 10 ++++++----
drivers/dma/ioat/init.c | 2 +-
3 files changed, 15 insertions(+), 11 deletions(-)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index 18c011e..1e0e6c1 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -332,8 +332,8 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
u8 *pos;
off_t offs;
- chunk = idx / IOAT_DESCS_PER_2M;
- idx &= (IOAT_DESCS_PER_2M - 1);
+ chunk = idx / IOAT_DESCS_PER_CHUNK;
+ idx &= (IOAT_DESCS_PER_CHUNK - 1);
offs = idx * IOAT_DESC_SZ;
pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
phys = ioat_chan->descs[chunk].hw + offs;
@@ -370,7 +370,8 @@ struct ioat_ring_ent **
if (!ring)
return NULL;
- ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
+ chunks = (total_descs * IOAT_DESC_SZ) / IOAT_CHUNK_SIZE;
+ ioat_chan->desc_chunks = chunks;
for (i = 0; i < chunks; i++) {
struct ioat_descs *descs = &ioat_chan->descs[i];
@@ -382,8 +383,9 @@ struct ioat_ring_ent **
for (idx = 0; idx < i; idx++) {
descs = &ioat_chan->descs[idx];
- dma_free_coherent(to_dev(ioat_chan), SZ_2M,
- descs->virt, descs->hw);
+ dma_free_coherent(to_dev(ioat_chan),
+ IOAT_CHUNK_SIZE,
+ descs->virt, descs->hw);
descs->virt = NULL;
descs->hw = 0;
}
@@ -404,7 +406,7 @@ struct ioat_ring_ent **
for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
dma_free_coherent(to_dev(ioat_chan),
- SZ_2M,
+ IOAT_CHUNK_SIZE,
ioat_chan->descs[idx].virt,
ioat_chan->descs[idx].hw);
ioat_chan->descs[idx].virt = NULL;
diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
index b8e8e0b..5216c6b 100644
--- a/drivers/dma/ioat/dma.h
+++ b/drivers/dma/ioat/dma.h
@@ -81,6 +81,11 @@ struct ioatdma_device {
u32 msixpba;
};
+#define IOAT_MAX_ORDER 16
+#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
+#define IOAT_CHUNK_SIZE (SZ_2M)
+#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
+
struct ioat_descs {
void *virt;
dma_addr_t hw;
@@ -128,7 +133,7 @@ struct ioatdma_chan {
u16 produce;
struct ioat_ring_ent **ring;
spinlock_t prep_lock;
- struct ioat_descs descs[2];
+ struct ioat_descs descs[IOAT_MAX_DESCS / IOAT_DESCS_PER_CHUNK];
int desc_chunks;
int intr_coalesce;
int prev_intr_coalesce;
@@ -301,9 +306,6 @@ static inline bool is_ioat_bug(unsigned long err)
return !!err;
}
-#define IOAT_MAX_ORDER 16
-#define IOAT_MAX_DESCS 65536
-#define IOAT_DESCS_PER_2M 32768
static inline u32 ioat_ring_size(struct ioatdma_chan *ioat_chan)
{
diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
index 60e9afb..58d1356 100644
--- a/drivers/dma/ioat/init.c
+++ b/drivers/dma/ioat/init.c
@@ -651,7 +651,7 @@ static void ioat_free_chan_resources(struct dma_chan *c)
}
for (i = 0; i < ioat_chan->desc_chunks; i++) {
- dma_free_coherent(to_dev(ioat_chan), SZ_2M,
+ dma_free_coherent(to_dev(ioat_chan), IOAT_CHUNK_SIZE,
ioat_chan->descs[i].virt,
ioat_chan->descs[i].hw);
ioat_chan->descs[i].virt = NULL;
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K
2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
@ 2020-04-16 17:06 ` leonid.ravich
2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich
2020-04-17 11:58 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency Vinod Koul
1 sibling, 1 reply; 22+ messages in thread
From: leonid.ravich @ 2020-04-16 17:06 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang,
Greg Kroah-Hartman, Alexios Zavras, Alexander.Barabash,
Thomas Gleixner, Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe,
linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
requreing kmalloc of 2M high chance to fail in
fragmented memory.
IOAT ring requires 64k * 64B memory
which will be achived by 512k * 8 allocation
instead of 2M * 2.
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
Changing in v3:
- Make the commit message more clearer.
drivers/dma/ioat/dma.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
index 5216c6b..e6b622e 100644
--- a/drivers/dma/ioat/dma.h
+++ b/drivers/dma/ioat/dma.h
@@ -83,7 +83,7 @@ struct ioatdma_device {
#define IOAT_MAX_ORDER 16
#define IOAT_MAX_DESCS (1 << IOAT_MAX_ORDER)
-#define IOAT_CHUNK_SIZE (SZ_2M)
+#define IOAT_CHUNK_SIZE (SZ_512K)
#define IOAT_DESCS_PER_CHUNK (IOAT_CHUNK_SIZE / IOAT_DESC_SZ)
struct ioat_descs {
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency
2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich
@ 2020-04-17 11:58 ` Vinod Koul
1 sibling, 0 replies; 22+ messages in thread
From: Vinod Koul @ 2020-04-17 11:58 UTC (permalink / raw)
To: leonid.ravich
Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Greg Kroah-Hartman,
Alexios Zavras, Alexander.Barabash, Thomas Gleixner,
Kate Stewart, Jilayne Lovejoy, Logan Gunthorpe, linux-kernel
On 16-04-20, 20:06, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> changing macros which assumption is chunk size of 2M,
> which can be other size
> prepare for changing allocation chunk size.
Applied both, thanks
--
~Vinod
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler
2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich
@ 2020-04-22 19:25 ` leonid.ravich
2020-04-22 19:25 ` [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich
2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
0 siblings, 2 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-22 19:25 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang,
Alexander.Barabash, Allison Randal, Thomas Gleixner,
linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
moving duplicate code from timeout error handling to common
function.
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.c | 45 +++++++++++++++++++--------------------------
1 file changed, 19 insertions(+), 26 deletions(-)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index 1e0e6c1..da59b28 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -869,6 +869,23 @@ static void check_active(struct ioatdma_chan *ioat_chan)
mod_timer(&ioat_chan->timer, jiffies + IDLE_TIMEOUT);
}
+static void ioat_reboot_chan(struct ioatdma_chan *ioat_chan)
+{
+ spin_lock_bh(&ioat_chan->prep_lock);
+ set_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
+ spin_unlock_bh(&ioat_chan->prep_lock);
+
+ ioat_abort_descs(ioat_chan);
+ dev_warn(to_dev(ioat_chan), "Reset channel...\n");
+ ioat_reset_hw(ioat_chan);
+ dev_warn(to_dev(ioat_chan), "Restart channel...\n");
+ ioat_restart_channel(ioat_chan);
+
+ spin_lock_bh(&ioat_chan->prep_lock);
+ clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
+ spin_unlock_bh(&ioat_chan->prep_lock);
+}
+
void ioat_timer_event(struct timer_list *t)
{
struct ioatdma_chan *ioat_chan = from_timer(ioat_chan, t, timer);
@@ -891,19 +908,7 @@ void ioat_timer_event(struct timer_list *t)
if (test_bit(IOAT_RUN, &ioat_chan->state)) {
spin_lock_bh(&ioat_chan->cleanup_lock);
- spin_lock_bh(&ioat_chan->prep_lock);
- set_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
-
- ioat_abort_descs(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Reset channel...\n");
- ioat_reset_hw(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Restart channel...\n");
- ioat_restart_channel(ioat_chan);
-
- spin_lock_bh(&ioat_chan->prep_lock);
- clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
+ ioat_reboot_chan(ioat_chan);
spin_unlock_bh(&ioat_chan->cleanup_lock);
}
@@ -939,19 +944,7 @@ void ioat_timer_event(struct timer_list *t)
dev_dbg(to_dev(ioat_chan), "Active descriptors: %d\n",
ioat_ring_active(ioat_chan));
- spin_lock_bh(&ioat_chan->prep_lock);
- set_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
-
- ioat_abort_descs(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Resetting channel...\n");
- ioat_reset_hw(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Restarting channel...\n");
- ioat_restart_channel(ioat_chan);
-
- spin_lock_bh(&ioat_chan->prep_lock);
- clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
+ ioat_reboot_chan(ioat_chan);
spin_unlock_bh(&ioat_chan->cleanup_lock);
return;
} else
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification.
2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich
@ 2020-04-22 19:25 ` leonid.ravich
2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
1 sibling, 0 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-22 19:25 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang,
Allison Randal, Thomas Gleixner, Greg Kroah-Hartman,
Alexander.Barabash, linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
removing unnecessary mod_timer from timeout handler
incase of ioat_cleanup_preamble() is true for cleaner code
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.c | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index da59b28..55a8cf1 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -922,17 +922,23 @@ void ioat_timer_event(struct timer_list *t)
spin_lock_bh(&ioat_chan->prep_lock);
check_active(ioat_chan);
spin_unlock_bh(&ioat_chan->prep_lock);
- spin_unlock_bh(&ioat_chan->cleanup_lock);
- return;
+ goto unlock_out;
+ }
+
+ /* handle the missed cleanup case */
+ if (ioat_cleanup_preamble(ioat_chan, &phys_complete)) {
+ /* timer restarted in ioat_cleanup_preamble
+ * and IOAT_COMPLETION_ACK cleared
+ */
+ __cleanup(ioat_chan, phys_complete);
+ goto unlock_out;
}
/* if we haven't made progress and we have already
* acknowledged a pending completion once, then be more
* forceful with a restart
*/
- if (ioat_cleanup_preamble(ioat_chan, &phys_complete))
- __cleanup(ioat_chan, phys_complete);
- else if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) {
+ if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) {
u32 chanerr;
chanerr = readl(ioat_chan->reg_base + IOAT_CHANERR_OFFSET);
@@ -945,12 +951,13 @@ void ioat_timer_event(struct timer_list *t)
ioat_ring_active(ioat_chan));
ioat_reboot_chan(ioat_chan);
- spin_unlock_bh(&ioat_chan->cleanup_lock);
- return;
- } else
- set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state);
+ goto unlock_out;
+ }
+
+ set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state);
mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT);
+unlock_out:
spin_unlock_bh(&ioat_chan->cleanup_lock);
}
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler
2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich
2020-04-22 19:25 ` [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich
@ 2020-04-22 19:25 ` leonid.ravich
2020-04-22 19:58 ` Dave Jiang
2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich
1 sibling, 2 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-22 19:25 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Vinod Koul, Dan Williams, Dave Jiang,
Greg Kroah-Hartman, Thomas Gleixner, Alexander.Barabash,
linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
completion timeout might trigger unnesesery DMA engine hw reboot
in case of missed issue_pending() .
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
Changing in v2
- add log in case of such scenario
drivers/dma/ioat/dma.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index 55a8cf1..a958aaf 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -955,6 +955,14 @@ void ioat_timer_event(struct timer_list *t)
goto unlock_out;
}
+ /* handle missed issue pending case */
+ if (ioat_ring_pending(ioat_chan)) {
+ dev_dbg(to_dev(ioat_chan), "Complition timeout while pending\n")
+ spin_lock_bh(&ioat_chan->prep_lock);
+ __ioat_issue_pending(ioat_chan);
+ spin_unlock_bh(&ioat_chan->prep_lock);
+ }
+
set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state);
mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT);
unlock_out:
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler
2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
@ 2020-04-22 19:58 ` Dave Jiang
2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich
1 sibling, 0 replies; 22+ messages in thread
From: Dave Jiang @ 2020-04-22 19:58 UTC (permalink / raw)
To: leonid.ravich, dmaengine
Cc: lravich, Vinod Koul, Dan Williams, Greg Kroah-Hartman,
Thomas Gleixner, Alexander.Barabash, linux-kernel
On 4/22/2020 12:25 PM, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> completion timeout might trigger unnesesery DMA engine hw reboot
> in case of missed issue_pending() .
>
> Acked-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
> ---
> Changing in v2
> - add log in case of such scenario
> drivers/dma/ioat/dma.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
> index 55a8cf1..a958aaf 100644
> --- a/drivers/dma/ioat/dma.c
> +++ b/drivers/dma/ioat/dma.c
> @@ -955,6 +955,14 @@ void ioat_timer_event(struct timer_list *t)
> goto unlock_out;
> }
>
> + /* handle missed issue pending case */
> + if (ioat_ring_pending(ioat_chan)) {
> + dev_dbg(to_dev(ioat_chan), "Complition timeout while pending\n")
Completion timeout with pending descriptors.
Also, maybe dev_warn() you think?
> + spin_lock_bh(&ioat_chan->prep_lock);
> + __ioat_issue_pending(ioat_chan);
> + spin_unlock_bh(&ioat_chan->prep_lock);
> + }
> +
> set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state);
> mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT);
> unlock_out:
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from timeout handler
2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
2020-04-22 19:58 ` Dave Jiang
@ 2020-04-22 21:09 ` leonid.ravich
2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich
` (2 more replies)
1 sibling, 3 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-22 21:09 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang,
Allison Randal, Alexander.Barabash, Alexios Zavras,
Thomas Gleixner, linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
moving duplicate code from timeout error handling to common
function.
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.c | 45 +++++++++++++++++++--------------------------
1 file changed, 19 insertions(+), 26 deletions(-)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index 1e0e6c1..da59b28 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -869,6 +869,23 @@ static void check_active(struct ioatdma_chan *ioat_chan)
mod_timer(&ioat_chan->timer, jiffies + IDLE_TIMEOUT);
}
+static void ioat_reboot_chan(struct ioatdma_chan *ioat_chan)
+{
+ spin_lock_bh(&ioat_chan->prep_lock);
+ set_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
+ spin_unlock_bh(&ioat_chan->prep_lock);
+
+ ioat_abort_descs(ioat_chan);
+ dev_warn(to_dev(ioat_chan), "Reset channel...\n");
+ ioat_reset_hw(ioat_chan);
+ dev_warn(to_dev(ioat_chan), "Restart channel...\n");
+ ioat_restart_channel(ioat_chan);
+
+ spin_lock_bh(&ioat_chan->prep_lock);
+ clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
+ spin_unlock_bh(&ioat_chan->prep_lock);
+}
+
void ioat_timer_event(struct timer_list *t)
{
struct ioatdma_chan *ioat_chan = from_timer(ioat_chan, t, timer);
@@ -891,19 +908,7 @@ void ioat_timer_event(struct timer_list *t)
if (test_bit(IOAT_RUN, &ioat_chan->state)) {
spin_lock_bh(&ioat_chan->cleanup_lock);
- spin_lock_bh(&ioat_chan->prep_lock);
- set_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
-
- ioat_abort_descs(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Reset channel...\n");
- ioat_reset_hw(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Restart channel...\n");
- ioat_restart_channel(ioat_chan);
-
- spin_lock_bh(&ioat_chan->prep_lock);
- clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
+ ioat_reboot_chan(ioat_chan);
spin_unlock_bh(&ioat_chan->cleanup_lock);
}
@@ -939,19 +944,7 @@ void ioat_timer_event(struct timer_list *t)
dev_dbg(to_dev(ioat_chan), "Active descriptors: %d\n",
ioat_ring_active(ioat_chan));
- spin_lock_bh(&ioat_chan->prep_lock);
- set_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
-
- ioat_abort_descs(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Resetting channel...\n");
- ioat_reset_hw(ioat_chan);
- dev_warn(to_dev(ioat_chan), "Restarting channel...\n");
- ioat_restart_channel(ioat_chan);
-
- spin_lock_bh(&ioat_chan->prep_lock);
- clear_bit(IOAT_CHAN_DOWN, &ioat_chan->state);
- spin_unlock_bh(&ioat_chan->prep_lock);
+ ioat_reboot_chan(ioat_chan);
spin_unlock_bh(&ioat_chan->cleanup_lock);
return;
} else
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification.
2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich
@ 2020-04-22 21:09 ` leonid.ravich
2020-04-22 21:09 ` [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
2020-04-23 7:21 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " Vinod Koul
2 siblings, 0 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-22 21:09 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang,
Greg Kroah-Hartman, Alexander.Barabash, Alexios Zavras,
Thomas Gleixner, linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
removing unnecessary mod_timer from timeout handler
incase of ioat_cleanup_preamble() is true for cleaner code
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
drivers/dma/ioat/dma.c | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index da59b28..55a8cf1 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -922,17 +922,23 @@ void ioat_timer_event(struct timer_list *t)
spin_lock_bh(&ioat_chan->prep_lock);
check_active(ioat_chan);
spin_unlock_bh(&ioat_chan->prep_lock);
- spin_unlock_bh(&ioat_chan->cleanup_lock);
- return;
+ goto unlock_out;
+ }
+
+ /* handle the missed cleanup case */
+ if (ioat_cleanup_preamble(ioat_chan, &phys_complete)) {
+ /* timer restarted in ioat_cleanup_preamble
+ * and IOAT_COMPLETION_ACK cleared
+ */
+ __cleanup(ioat_chan, phys_complete);
+ goto unlock_out;
}
/* if we haven't made progress and we have already
* acknowledged a pending completion once, then be more
* forceful with a restart
*/
- if (ioat_cleanup_preamble(ioat_chan, &phys_complete))
- __cleanup(ioat_chan, phys_complete);
- else if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) {
+ if (test_bit(IOAT_COMPLETION_ACK, &ioat_chan->state)) {
u32 chanerr;
chanerr = readl(ioat_chan->reg_base + IOAT_CHANERR_OFFSET);
@@ -945,12 +951,13 @@ void ioat_timer_event(struct timer_list *t)
ioat_ring_active(ioat_chan));
ioat_reboot_chan(ioat_chan);
- spin_unlock_bh(&ioat_chan->cleanup_lock);
- return;
- } else
- set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state);
+ goto unlock_out;
+ }
+
+ set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state);
mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT);
+unlock_out:
spin_unlock_bh(&ioat_chan->cleanup_lock);
}
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler
2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich
2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich
@ 2020-04-22 21:09 ` leonid.ravich
2020-04-23 7:21 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " Vinod Koul
2 siblings, 0 replies; 22+ messages in thread
From: leonid.ravich @ 2020-04-22 21:09 UTC (permalink / raw)
To: dmaengine
Cc: lravich, Leonid Ravich, Dan Williams, Vinod Koul, Dave Jiang,
Alexander.Barabash, Alexios Zavras, Thomas Gleixner,
linux-kernel
From: Leonid Ravich <Leonid.Ravich@emc.com>
completion timeout might trigger unnesesery DMA engine hw reboot
in case of missed issue_pending() .
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Leonid Ravich <Leonid.Ravich@emc.com>
---
Changing in v2:
- fixing log spelling and level
drivers/dma/ioat/dma.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index 55a8cf1..8ad0ad8 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -955,6 +955,15 @@ void ioat_timer_event(struct timer_list *t)
goto unlock_out;
}
+ /* handle missed issue pending case */
+ if (ioat_ring_pending(ioat_chan)) {
+ dev_warn(to_dev(ioat_chan),
+ "Completion timeout with pending descriptors\n");
+ spin_lock_bh(&ioat_chan->prep_lock);
+ __ioat_issue_pending(ioat_chan);
+ spin_unlock_bh(&ioat_chan->prep_lock);
+ }
+
set_bit(IOAT_COMPLETION_ACK, &ioat_chan->state);
mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT);
unlock_out:
--
1.9.3
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from timeout handler
2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich
2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich
2020-04-22 21:09 ` [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
@ 2020-04-23 7:21 ` Vinod Koul
2 siblings, 0 replies; 22+ messages in thread
From: Vinod Koul @ 2020-04-23 7:21 UTC (permalink / raw)
To: leonid.ravich
Cc: dmaengine, lravich, Dan Williams, Dave Jiang, Allison Randal,
Alexander.Barabash, Alexios Zavras, Thomas Gleixner,
linux-kernel
On 23-04-20, 00:09, leonid.ravich@dell.com wrote:
> From: Leonid Ravich <Leonid.Ravich@emc.com>
>
> moving duplicate code from timeout error handling to common
> function.
Applied all, thanks
--
~Vinod
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2020-04-23 7:22 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-02 9:27 [PATCH 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-02 9:27 ` [PATCH 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
2020-04-02 15:06 ` Dave Jiang
2020-04-02 16:33 ` [PATCH v2 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-02 16:33 ` [PATCH v2 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M -> 512K leonid.ravich
2020-04-15 15:53 ` Vinod Koul
2020-04-16 17:06 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency leonid.ravich
2020-04-16 17:06 ` [PATCH v3 2/2] dmaengine: ioat: Decreasing allocation chunk size 2M->512K leonid.ravich
2020-04-22 19:25 ` [PATCH v2 1/3] dmaengine: ioat: removing duplicate code from timeout handler leonid.ravich
2020-04-22 19:25 ` [PATCH v2 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich
2020-04-22 19:25 ` [PATCH v2 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
2020-04-22 19:58 ` Dave Jiang
2020-04-22 21:09 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " leonid.ravich
2020-04-22 21:09 ` [PATCH v3 2/3] dmaengine: ioat: remove unnesesery double complition timer modification leonid.ravich
2020-04-22 21:09 ` [PATCH v3 3/3] dmaengine: ioat: adding missed issue_pending to timeout handler leonid.ravich
2020-04-23 7:21 ` [PATCH v3 1/3] dmaengine: ioat: removing duplicate code from " Vinod Koul
2020-04-17 11:58 ` [PATCH v3 1/2] dmaengine: ioat: fixing chunk sizing macros dependency Vinod Koul
2020-04-02 16:42 ` [PATCH v2 " Dave Jiang
2020-04-02 16:46 ` Ravich, Leonid
2020-04-02 16:49 ` Dave Jiang
2020-04-15 15:52 ` Vinod Koul
2020-04-02 15:04 ` [PATCH " Dave Jiang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.