All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/3] coccinelle: catchup on memory allocation functions
@ 2016-02-16 17:01 ` Yann Droneaud
  0 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:01 UTC (permalink / raw)
  To: Julia Lawall, Gilles Muller, Nicolas Palix, Michal Marek
  Cc: Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel,
	Yann Droneaud, Daniel Thompson, Eli Billauer, Geert Uytterhoeven,
	Himangi Saraogi, Joe Perches, Johannes Weiner, Manish Badarkhe,
	Pekka Enberg, Srinivas Pandruvada, Wolfram Sang

Hi,

Please find few patches that remove coccicheck's blindness
regarding the following functions:
- kzfree()
- krealloc()
- __krealloc()
- devm_kmalloc()
- devm_kvasprintf()
- devm_kasprintf()
- devm_kmalloc_array()
- devm_kcalloc()
- devm_kstrdup()
- devm_kmemdup()
- devm_get_free_pages()
- free_pages()
- free_page()

I've ran coccicheck on drivers/staging and found no new issue,
which is great, but that questions those patches' usefulness,
YMMV.

Regards.

Yann Droneaud (3):
  coccinelle: also catch kzfree() issues
  coccinelle: recognize more devm_* memory allocation functions
  coccinelle: catch krealloc() on devm_*() allocated memory

 scripts/coccinelle/free/devm_free.cocci | 26 ++++++++++++++++++++++++++
 scripts/coccinelle/free/kfree.cocci     |  3 +++
 scripts/coccinelle/free/kfreeaddr.cocci |  6 +++++-
 3 files changed, 34 insertions(+), 1 deletion(-)

-- 
2.5.0

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] [PATCH 0/3] coccinelle: catchup on memory allocation functions
@ 2016-02-16 17:01 ` Yann Droneaud
  0 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:01 UTC (permalink / raw)
  To: cocci

Hi,

Please find few patches that remove coccicheck's blindness
regarding the following functions:
- kzfree()
- krealloc()
- __krealloc()
- devm_kmalloc()
- devm_kvasprintf()
- devm_kasprintf()
- devm_kmalloc_array()
- devm_kcalloc()
- devm_kstrdup()
- devm_kmemdup()
- devm_get_free_pages()
- free_pages()
- free_page()

I've ran coccicheck on drivers/staging and found no new issue,
which is great, but that questions those patches' usefulness,
YMMV.

Regards.

Yann Droneaud (3):
  coccinelle: also catch kzfree() issues
  coccinelle: recognize more devm_* memory allocation functions
  coccinelle: catch krealloc() on devm_*() allocated memory

 scripts/coccinelle/free/devm_free.cocci | 26 ++++++++++++++++++++++++++
 scripts/coccinelle/free/kfree.cocci     |  3 +++
 scripts/coccinelle/free/kfreeaddr.cocci |  6 +++++-
 3 files changed, 34 insertions(+), 1 deletion(-)

-- 
2.5.0

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [PATCH 1/3] coccinelle: also catch kzfree() issues
  2016-02-16 17:01 ` [Cocci] " Yann Droneaud
@ 2016-02-16 17:06   ` Yann Droneaud
  -1 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:06 UTC (permalink / raw)
  To: Julia Lawall, Gilles Muller, Nicolas Palix, Michal Marek
  Cc: Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel,
	Yann Droneaud, Johannes Weiner

Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
kfree() is no more the only function to be considered.

Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
 scripts/coccinelle/free/devm_free.cocci | 2 ++
 scripts/coccinelle/free/kfree.cocci     | 3 +++
 scripts/coccinelle/free/kfreeaddr.cocci | 6 +++++-
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 3d9349012bb3..83c03adec1c5 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -48,6 +48,8 @@ position p;
 (
 * kfree@p(x)
 |
+* kzfree@p(x)
+|
 * free_irq@p(x)
 |
 * iounmap@p(x)
diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
index 577b78056990..c3246b47f51f 100644
--- a/scripts/coccinelle/free/kfree.cocci
+++ b/scripts/coccinelle/free/kfree.cocci
@@ -16,6 +16,7 @@ virtual org
 virtual report
 
 @free@
+identifier kfree =~ "kz?free";
 expression E;
 position p1;
 @@
@@ -54,6 +55,7 @@ position p;
  sizeof(<+...E@p...+>)
 
 @loop exists@
+identifier kfree =~ "kz?free";
 expression E;
 identifier l;
 position ok;
@@ -67,6 +69,7 @@ while (1) { ...
 }
 
 @r exists@
+identifier kfree =~ "kz?free";
 expression free.E, subE<=free.E, E2;
 expression E1;
 iterator iter;
diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
index ce8aacc314cb..d46063b1db8b 100644
--- a/scripts/coccinelle/free/kfreeaddr.cocci
+++ b/scripts/coccinelle/free/kfreeaddr.cocci
@@ -16,7 +16,11 @@ identifier f;
 position p;
 @@
 
+(
 * kfree@p(&e->f)
+|
+* kzfree@p(&e->f)
+)
 
 @script:python depends on org@
 p << r.p;
@@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
 p << r.p;
 @@
 
-msg = "ERROR: kfree of structure field"
+msg = "ERROR: invalid free of structure field"
 coccilib.report.print_report(p[0],msg)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Cocci] [PATCH 1/3] coccinelle: also catch kzfree() issues
@ 2016-02-16 17:06   ` Yann Droneaud
  0 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:06 UTC (permalink / raw)
  To: cocci

Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
kfree() is no more the only function to be considered.

Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
 scripts/coccinelle/free/devm_free.cocci | 2 ++
 scripts/coccinelle/free/kfree.cocci     | 3 +++
 scripts/coccinelle/free/kfreeaddr.cocci | 6 +++++-
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 3d9349012bb3..83c03adec1c5 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -48,6 +48,8 @@ position p;
 (
 * kfree at p(x)
 |
+* kzfree at p(x)
+|
 * free_irq at p(x)
 |
 * iounmap at p(x)
diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
index 577b78056990..c3246b47f51f 100644
--- a/scripts/coccinelle/free/kfree.cocci
+++ b/scripts/coccinelle/free/kfree.cocci
@@ -16,6 +16,7 @@ virtual org
 virtual report
 
 @free@
+identifier kfree =~ "kz?free";
 expression E;
 position p1;
 @@
@@ -54,6 +55,7 @@ position p;
  sizeof(<+...E@p...+>)
 
 @loop exists@
+identifier kfree =~ "kz?free";
 expression E;
 identifier l;
 position ok;
@@ -67,6 +69,7 @@ while (1) { ...
 }
 
 @r exists@
+identifier kfree =~ "kz?free";
 expression free.E, subE<=free.E, E2;
 expression E1;
 iterator iter;
diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
index ce8aacc314cb..d46063b1db8b 100644
--- a/scripts/coccinelle/free/kfreeaddr.cocci
+++ b/scripts/coccinelle/free/kfreeaddr.cocci
@@ -16,7 +16,11 @@ identifier f;
 position p;
 @@
 
+(
 * kfree@p(&e->f)
+|
+* kzfree at p(&e->f)
+)
 
 @script:python depends on org@
 p << r.p;
@@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
 p << r.p;
 @@
 
-msg = "ERROR: kfree of structure field"
+msg = "ERROR: invalid free of structure field"
 coccilib.report.print_report(p[0],msg)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH 2/3] coccinelle: recognize more devm_* memory allocation functions
  2016-02-16 17:01 ` [Cocci] " Yann Droneaud
@ 2016-02-16 17:06   ` Yann Droneaud
  -1 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:06 UTC (permalink / raw)
  To: Julia Lawall, Gilles Muller, Nicolas Palix, Michal Marek
  Cc: Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel,
	Yann Droneaud, Joe Perches, Manish Badarkhe, Srinivas Pandruvada,
	Eli Billauer, Himangi Saraogi, Geert Uytterhoeven, Wolfram Sang,
	Daniel Thompson

Updates free/devm_free.cocci to recognize functions added by:

- commit 64c862a839a8 ('devres: add kernel standard devm_k.alloc functions')
- commit e31108cad3de ('devres: introduce API "devm_kstrdup"')
- commit 3046365bb470 ('devres: introduce API "devm_kmemdup')
- commit 43339bed7010 ('devres: Add devm_get_free_pages API')
- commit 75f2a4ead5d5 ('devres: Add devm_kasprintf and devm_kvasprintf API')

See also Documentation/driver-model/devres.txt

Cc: Joe Perches <joe@perches.com>
Cc: Manish Badarkhe <badarkhe.manish@gmail.com>
Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
Cc: Eli Billauer <eli.billauer@gmail.com>
Cc: Himangi Saraogi <himangi774@gmail.com>
Cc: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: Wolfram Sang <w.sang@pengutronix.de>
Cc: Daniel Thompson <daniel.thompson@linaro.org>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
 scripts/coccinelle/free/devm_free.cocci | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 83c03adec1c5..3794cd97494b 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -29,8 +29,24 @@ expression x;
 @@
 
 (
+ x = devm_kmalloc(...)
+|
+ x = devm_kvasprintf(...)
+|
+ x = devm_kasprintf(...)
+|
  x = devm_kzalloc(...)
 |
+ x = devm_kmalloc_array(...)
+|
+ x = devm_kcalloc(...)
+|
+ x = devm_kstrdup(...)
+|
+ x = devm_kmemdup(...)
+|
+ x = devm_get_free_pages(...)
+|
  x = devm_request_irq(...)
 |
  x = devm_ioremap(...)
@@ -50,6 +66,10 @@ position p;
 |
 * kzfree@p(x)
 |
+* free_pages@p(x, ...)
+|
+* free_page@p(x)
+|
 * free_irq@p(x)
 |
 * iounmap@p(x)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Cocci] [PATCH 2/3] coccinelle: recognize more devm_* memory allocation functions
@ 2016-02-16 17:06   ` Yann Droneaud
  0 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:06 UTC (permalink / raw)
  To: cocci

Updates free/devm_free.cocci to recognize functions added by:

- commit 64c862a839a8 ('devres: add kernel standard devm_k.alloc functions')
- commit e31108cad3de ('devres: introduce API "devm_kstrdup"')
- commit 3046365bb470 ('devres: introduce API "devm_kmemdup')
- commit 43339bed7010 ('devres: Add devm_get_free_pages API')
- commit 75f2a4ead5d5 ('devres: Add devm_kasprintf and devm_kvasprintf API')

See also Documentation/driver-model/devres.txt

Cc: Joe Perches <joe@perches.com>
Cc: Manish Badarkhe <badarkhe.manish@gmail.com>
Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
Cc: Eli Billauer <eli.billauer@gmail.com>
Cc: Himangi Saraogi <himangi774@gmail.com>
Cc: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: Wolfram Sang <w.sang@pengutronix.de>
Cc: Daniel Thompson <daniel.thompson@linaro.org>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
 scripts/coccinelle/free/devm_free.cocci | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 83c03adec1c5..3794cd97494b 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -29,8 +29,24 @@ expression x;
 @@
 
 (
+ x = devm_kmalloc(...)
+|
+ x = devm_kvasprintf(...)
+|
+ x = devm_kasprintf(...)
+|
  x = devm_kzalloc(...)
 |
+ x = devm_kmalloc_array(...)
+|
+ x = devm_kcalloc(...)
+|
+ x = devm_kstrdup(...)
+|
+ x = devm_kmemdup(...)
+|
+ x = devm_get_free_pages(...)
+|
  x = devm_request_irq(...)
 |
  x = devm_ioremap(...)
@@ -50,6 +66,10 @@ position p;
 |
 * kzfree at p(x)
 |
+* free_pages at p(x, ...)
+|
+* free_page at p(x)
+|
 * free_irq at p(x)
 |
 * iounmap at p(x)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH 3/3] coccinelle: catch krealloc() on devm_*() allocated memory
  2016-02-16 17:01 ` [Cocci] " Yann Droneaud
@ 2016-02-16 17:06   ` Yann Droneaud
  -1 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:06 UTC (permalink / raw)
  To: Julia Lawall, Gilles Muller, Nicolas Palix, Michal Marek
  Cc: Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel,
	Yann Droneaud, Pekka Enberg

krealloc() must not be used against devm_*() allocated
memory regions:

- if a bigger memory is to be allocated, krealloc() and
  __krealloc() could return a different pointer than the
  one given to them, creating a memory region which is not
  managed, thus it will not be automatically released on
  device removal.

- if a bigger memory is to be allocated, krealloc() could
  kfree() the managed memory region which is passed to it.
  The old pointer is left registered as a resource for the
  device. On device removal, this dangling pointer will be
  used and an unrelated memory region could be released.

- if the requested size is equal to 0, krealloc() can also
  just behave like kfree(). Here too, the old pointer is
  kept associated with the device. On device removal, this
  invalid pointer will be used and an unrelated memory
  region could be released.

For all these reasons, krealloc() must not be used on a
pointer returned by devm_*() functions.

Cc: Tejun Heo <tj@kernel.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
 scripts/coccinelle/free/devm_free.cocci | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 3794cd97494b..c990d2c7ee16 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -66,6 +66,10 @@ position p;
 |
 * kzfree@p(x)
 |
+* __krealloc@p(x, ...)
+|
+* krealloc@p(x, ...)
+|
 * free_pages@p(x, ...)
 |
 * free_page@p(x)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Cocci] [PATCH 3/3] coccinelle: catch krealloc() on devm_*() allocated memory
@ 2016-02-16 17:06   ` Yann Droneaud
  0 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-16 17:06 UTC (permalink / raw)
  To: cocci

krealloc() must not be used against devm_*() allocated
memory regions:

- if a bigger memory is to be allocated, krealloc() and
  __krealloc() could return a different pointer than the
  one given to them, creating a memory region which is not
  managed, thus it will not be automatically released on
  device removal.

- if a bigger memory is to be allocated, krealloc() could
  kfree() the managed memory region which is passed to it.
  The old pointer is left registered as a resource for the
  device. On device removal, this dangling pointer will be
  used and an unrelated memory region could be released.

- if the requested size is equal to 0, krealloc() can also
  just behave like kfree(). Here too, the old pointer is
  kept associated with the device. On device removal, this
  invalid pointer will be used and an unrelated memory
  region could be released.

For all these reasons, krealloc() must not be used on a
pointer returned by devm_*() functions.

Cc: Tejun Heo <tj@kernel.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
 scripts/coccinelle/free/devm_free.cocci | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 3794cd97494b..c990d2c7ee16 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -66,6 +66,10 @@ position p;
 |
 * kzfree at p(x)
 |
+* __krealloc at p(x, ...)
+|
+* krealloc at p(x, ...)
+|
 * free_pages at p(x, ...)
 |
 * free_page at p(x)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* Re: [PATCH 1/3] coccinelle: also catch kzfree() issues
  2016-02-16 17:06   ` [Cocci] " Yann Droneaud
@ 2016-02-16 17:16     ` Julia Lawall
  -1 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 17:16 UTC (permalink / raw)
  To: Yann Droneaud
  Cc: Gilles Muller, Nicolas Palix, Michal Marek, Tejun Heo,
	Greg Kroah-Hartman, cocci, linux-kernel, Johannes Weiner



On Tue, 16 Feb 2016, Yann Droneaud wrote:

> Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
> kfree() is no more the only function to be considered.
>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
> ---
>  scripts/coccinelle/free/devm_free.cocci | 2 ++
>  scripts/coccinelle/free/kfree.cocci     | 3 +++
>  scripts/coccinelle/free/kfreeaddr.cocci | 6 +++++-
>  3 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 3d9349012bb3..83c03adec1c5 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -48,6 +48,8 @@ position p;
>  (
>  * kfree@p(x)
>  |
> +* kzfree@p(x)
> +|
>  * free_irq@p(x)
>  |
>  * iounmap@p(x)
> diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
> index 577b78056990..c3246b47f51f 100644
> --- a/scripts/coccinelle/free/kfree.cocci
> +++ b/scripts/coccinelle/free/kfree.cocci
> @@ -16,6 +16,7 @@ virtual org
>  virtual report
>
>  @free@
> +identifier kfree =~ "kz?free";

Thanks for the suggestions.  However, the regular expression is not such a
good idea.  Coccinelle doesn't make any optimizations based on regulat
expressions.  It would be better to put a disjunction with kfree and
kzfree explicitly, as in the other cases.

julia

>  expression E;
>  position p1;
>  @@
> @@ -54,6 +55,7 @@ position p;
>   sizeof(<+...E@p...+>)
>
>  @loop exists@
> +identifier kfree =~ "kz?free";
>  expression E;
>  identifier l;
>  position ok;
> @@ -67,6 +69,7 @@ while (1) { ...
>  }
>
>  @r exists@
> +identifier kfree =~ "kz?free";
>  expression free.E, subE<=free.E, E2;
>  expression E1;
>  iterator iter;
> diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
> index ce8aacc314cb..d46063b1db8b 100644
> --- a/scripts/coccinelle/free/kfreeaddr.cocci
> +++ b/scripts/coccinelle/free/kfreeaddr.cocci
> @@ -16,7 +16,11 @@ identifier f;
>  position p;
>  @@
>
> +(
>  * kfree@p(&e->f)
> +|
> +* kzfree@p(&e->f)
> +)
>
>  @script:python depends on org@
>  p << r.p;
> @@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
>  p << r.p;
>  @@
>
> -msg = "ERROR: kfree of structure field"
> +msg = "ERROR: invalid free of structure field"
>  coccilib.report.print_report(p[0],msg)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] [PATCH 1/3] coccinelle: also catch kzfree() issues
@ 2016-02-16 17:16     ` Julia Lawall
  0 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 17:16 UTC (permalink / raw)
  To: cocci



On Tue, 16 Feb 2016, Yann Droneaud wrote:

> Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
> kfree() is no more the only function to be considered.
>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
> ---
>  scripts/coccinelle/free/devm_free.cocci | 2 ++
>  scripts/coccinelle/free/kfree.cocci     | 3 +++
>  scripts/coccinelle/free/kfreeaddr.cocci | 6 +++++-
>  3 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 3d9349012bb3..83c03adec1c5 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -48,6 +48,8 @@ position p;
>  (
>  * kfree at p(x)
>  |
> +* kzfree at p(x)
> +|
>  * free_irq at p(x)
>  |
>  * iounmap at p(x)
> diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
> index 577b78056990..c3246b47f51f 100644
> --- a/scripts/coccinelle/free/kfree.cocci
> +++ b/scripts/coccinelle/free/kfree.cocci
> @@ -16,6 +16,7 @@ virtual org
>  virtual report
>
>  @free@
> +identifier kfree =~ "kz?free";

Thanks for the suggestions.  However, the regular expression is not such a
good idea.  Coccinelle doesn't make any optimizations based on regulat
expressions.  It would be better to put a disjunction with kfree and
kzfree explicitly, as in the other cases.

julia

>  expression E;
>  position p1;
>  @@
> @@ -54,6 +55,7 @@ position p;
>   sizeof(<+...E@p...+>)
>
>  @loop exists@
> +identifier kfree =~ "kz?free";
>  expression E;
>  identifier l;
>  position ok;
> @@ -67,6 +69,7 @@ while (1) { ...
>  }
>
>  @r exists@
> +identifier kfree =~ "kz?free";
>  expression free.E, subE<=free.E, E2;
>  expression E1;
>  iterator iter;
> diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
> index ce8aacc314cb..d46063b1db8b 100644
> --- a/scripts/coccinelle/free/kfreeaddr.cocci
> +++ b/scripts/coccinelle/free/kfreeaddr.cocci
> @@ -16,7 +16,11 @@ identifier f;
>  position p;
>  @@
>
> +(
>  * kfree at p(&e->f)
> +|
> +* kzfree at p(&e->f)
> +)
>
>  @script:python depends on org@
>  p << r.p;
> @@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
>  p << r.p;
>  @@
>
> -msg = "ERROR: kfree of structure field"
> +msg = "ERROR: invalid free of structure field"
>  coccilib.report.print_report(p[0],msg)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH 2/3] coccinelle: recognize more devm_* memory allocation functions
  2016-02-16 17:06   ` [Cocci] " Yann Droneaud
@ 2016-02-16 17:18     ` Julia Lawall
  -1 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 17:18 UTC (permalink / raw)
  To: Yann Droneaud
  Cc: Julia Lawall, Gilles Muller, Nicolas Palix, Michal Marek,
	Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel, Joe Perches,
	Manish Badarkhe, Srinivas Pandruvada, Eli Billauer,
	Himangi Saraogi, Geert Uytterhoeven, Wolfram Sang,
	Daniel Thompson


On Tue, 16 Feb 2016, Yann Droneaud wrote:

> Updates free/devm_free.cocci to recognize functions added by:
>
> - commit 64c862a839a8 ('devres: add kernel standard devm_k.alloc functions')
> - commit e31108cad3de ('devres: introduce API "devm_kstrdup"')
> - commit 3046365bb470 ('devres: introduce API "devm_kmemdup')
> - commit 43339bed7010 ('devres: Add devm_get_free_pages API')
> - commit 75f2a4ead5d5 ('devres: Add devm_kasprintf and devm_kvasprintf API')
>
> See also Documentation/driver-model/devres.txt
>
> Cc: Joe Perches <joe@perches.com>
> Cc: Manish Badarkhe <badarkhe.manish@gmail.com>
> Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
> Cc: Eli Billauer <eli.billauer@gmail.com>
> Cc: Himangi Saraogi <himangi774@gmail.com>
> Cc: Geert Uytterhoeven <geert+renesas@glider.be>
> Cc: Wolfram Sang <w.sang@pengutronix.de>
> Cc: Daniel Thompson <daniel.thompson@linaro.org>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>

> ---
>  scripts/coccinelle/free/devm_free.cocci | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 83c03adec1c5..3794cd97494b 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -29,8 +29,24 @@ expression x;
>  @@
>
>  (
> + x = devm_kmalloc(...)
> +|
> + x = devm_kvasprintf(...)
> +|
> + x = devm_kasprintf(...)
> +|
>   x = devm_kzalloc(...)
>  |
> + x = devm_kmalloc_array(...)
> +|
> + x = devm_kcalloc(...)
> +|
> + x = devm_kstrdup(...)
> +|
> + x = devm_kmemdup(...)
> +|
> + x = devm_get_free_pages(...)
> +|
>   x = devm_request_irq(...)
>  |
>   x = devm_ioremap(...)
> @@ -50,6 +66,10 @@ position p;
>  |
>  * kzfree@p(x)
>  |
> +* free_pages@p(x, ...)
> +|
> +* free_page@p(x)
> +|
>  * free_irq@p(x)
>  |
>  * iounmap@p(x)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] [PATCH 2/3] coccinelle: recognize more devm_* memory allocation functions
@ 2016-02-16 17:18     ` Julia Lawall
  0 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 17:18 UTC (permalink / raw)
  To: cocci


On Tue, 16 Feb 2016, Yann Droneaud wrote:

> Updates free/devm_free.cocci to recognize functions added by:
>
> - commit 64c862a839a8 ('devres: add kernel standard devm_k.alloc functions')
> - commit e31108cad3de ('devres: introduce API "devm_kstrdup"')
> - commit 3046365bb470 ('devres: introduce API "devm_kmemdup')
> - commit 43339bed7010 ('devres: Add devm_get_free_pages API')
> - commit 75f2a4ead5d5 ('devres: Add devm_kasprintf and devm_kvasprintf API')
>
> See also Documentation/driver-model/devres.txt
>
> Cc: Joe Perches <joe@perches.com>
> Cc: Manish Badarkhe <badarkhe.manish@gmail.com>
> Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
> Cc: Eli Billauer <eli.billauer@gmail.com>
> Cc: Himangi Saraogi <himangi774@gmail.com>
> Cc: Geert Uytterhoeven <geert+renesas@glider.be>
> Cc: Wolfram Sang <w.sang@pengutronix.de>
> Cc: Daniel Thompson <daniel.thompson@linaro.org>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>

> ---
>  scripts/coccinelle/free/devm_free.cocci | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 83c03adec1c5..3794cd97494b 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -29,8 +29,24 @@ expression x;
>  @@
>
>  (
> + x = devm_kmalloc(...)
> +|
> + x = devm_kvasprintf(...)
> +|
> + x = devm_kasprintf(...)
> +|
>   x = devm_kzalloc(...)
>  |
> + x = devm_kmalloc_array(...)
> +|
> + x = devm_kcalloc(...)
> +|
> + x = devm_kstrdup(...)
> +|
> + x = devm_kmemdup(...)
> +|
> + x = devm_get_free_pages(...)
> +|
>   x = devm_request_irq(...)
>  |
>   x = devm_ioremap(...)
> @@ -50,6 +66,10 @@ position p;
>  |
>  * kzfree at p(x)
>  |
> +* free_pages at p(x, ...)
> +|
> +* free_page at p(x)
> +|
>  * free_irq at p(x)
>  |
>  * iounmap at p(x)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH 3/3] coccinelle: catch krealloc() on devm_*() allocated memory
  2016-02-16 17:06   ` [Cocci] " Yann Droneaud
@ 2016-02-16 17:19     ` Julia Lawall
  -1 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 17:19 UTC (permalink / raw)
  To: Yann Droneaud
  Cc: Julia Lawall, Gilles Muller, Nicolas Palix, Michal Marek,
	Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel, Pekka Enberg



On Tue, 16 Feb 2016, Yann Droneaud wrote:

> krealloc() must not be used against devm_*() allocated
> memory regions:
>
> - if a bigger memory is to be allocated, krealloc() and
>   __krealloc() could return a different pointer than the
>   one given to them, creating a memory region which is not
>   managed, thus it will not be automatically released on
>   device removal.
>
> - if a bigger memory is to be allocated, krealloc() could
>   kfree() the managed memory region which is passed to it.
>   The old pointer is left registered as a resource for the
>   device. On device removal, this dangling pointer will be
>   used and an unrelated memory region could be released.
>
> - if the requested size is equal to 0, krealloc() can also
>   just behave like kfree(). Here too, the old pointer is
>   kept associated with the device. On device removal, this
>   invalid pointer will be used and an unrelated memory
>   region could be released.
>
> For all these reasons, krealloc() must not be used on a
> pointer returned by devm_*() functions.
>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Pekka Enberg <penberg@cs.helsinki.fi>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>

> ---
>  scripts/coccinelle/free/devm_free.cocci | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 3794cd97494b..c990d2c7ee16 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -66,6 +66,10 @@ position p;
>  |
>  * kzfree@p(x)
>  |
> +* __krealloc@p(x, ...)
> +|
> +* krealloc@p(x, ...)
> +|
>  * free_pages@p(x, ...)
>  |
>  * free_page@p(x)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] [PATCH 3/3] coccinelle: catch krealloc() on devm_*() allocated memory
@ 2016-02-16 17:19     ` Julia Lawall
  0 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 17:19 UTC (permalink / raw)
  To: cocci



On Tue, 16 Feb 2016, Yann Droneaud wrote:

> krealloc() must not be used against devm_*() allocated
> memory regions:
>
> - if a bigger memory is to be allocated, krealloc() and
>   __krealloc() could return a different pointer than the
>   one given to them, creating a memory region which is not
>   managed, thus it will not be automatically released on
>   device removal.
>
> - if a bigger memory is to be allocated, krealloc() could
>   kfree() the managed memory region which is passed to it.
>   The old pointer is left registered as a resource for the
>   device. On device removal, this dangling pointer will be
>   used and an unrelated memory region could be released.
>
> - if the requested size is equal to 0, krealloc() can also
>   just behave like kfree(). Here too, the old pointer is
>   kept associated with the device. On device removal, this
>   invalid pointer will be used and an unrelated memory
>   region could be released.
>
> For all these reasons, krealloc() must not be used on a
> pointer returned by devm_*() functions.
>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Pekka Enberg <penberg@cs.helsinki.fi>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>

> ---
>  scripts/coccinelle/free/devm_free.cocci | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 3794cd97494b..c990d2c7ee16 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -66,6 +66,10 @@ position p;
>  |
>  * kzfree at p(x)
>  |
> +* __krealloc at p(x, ...)
> +|
> +* krealloc at p(x, ...)
> +|
>  * free_pages at p(x, ...)
>  |
>  * free_page at p(x)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: coccinelle: also catch kzfree() issues
  2016-02-16 17:16     ` [Cocci] " Julia Lawall
@ 2016-02-16 20:02       ` SF Markus Elfring
  -1 siblings, 0 replies; 26+ messages in thread
From: SF Markus Elfring @ 2016-02-16 20:02 UTC (permalink / raw)
  To: Julia Lawall
  Cc: Yann Droneaud, Gilles Muller, Nicolas Palix, Michal Marek,
	Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel,
	Johannes Weiner

>>  @free@
>> +identifier kfree =~ "kz?free";
> 
> Thanks for the suggestions.  However, the regular expression is not such a
> good idea.

How much is such a SmPL constraint still usable then?


> Coccinelle doesn't make any optimizations based on regulat expressions.

Where can your software optimise the source code search?


> It would be better to put a disjunction with kfree and kzfree explicitly,
> as in the other cases.

What are the circumstances for corresponding benefits?

Regards,
Markus

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] coccinelle: also catch kzfree() issues
@ 2016-02-16 20:02       ` SF Markus Elfring
  0 siblings, 0 replies; 26+ messages in thread
From: SF Markus Elfring @ 2016-02-16 20:02 UTC (permalink / raw)
  To: cocci

>>  @free@
>> +identifier kfree =~ "kz?free";
> 
> Thanks for the suggestions.  However, the regular expression is not such a
> good idea.

How much is such a SmPL constraint still usable then?


> Coccinelle doesn't make any optimizations based on regulat expressions.

Where can your software optimise the source code search?


> It would be better to put a disjunction with kfree and kzfree explicitly,
> as in the other cases.

What are the circumstances for corresponding benefits?

Regards,
Markus

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: coccinelle: also catch kzfree() issues
  2016-02-16 20:02       ` [Cocci] " SF Markus Elfring
@ 2016-02-16 20:17         ` Julia Lawall
  -1 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 20:17 UTC (permalink / raw)
  To: SF Markus Elfring
  Cc: Julia Lawall, Yann Droneaud, Gilles Muller, Nicolas Palix,
	Michal Marek, Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel,
	Johannes Weiner



On Tue, 16 Feb 2016, SF Markus Elfring wrote:

> >>  @free@
> >> +identifier kfree =~ "kz?free";
> >
> > Thanks for the suggestions.  However, the regular expression is not such a
> > good idea.
>
> How much is such a SmPL constraint still usable then?
>
>
> > Coccinelle doesn't make any optimizations based on regulat expressions.
>
> Where can your software optimise the source code search?

When the name appears explicitly in the matching code, Coccinelle will
parse and process only files that contain that name.

julia

>
>
> > It would be better to put a disjunction with kfree and kzfree explicitly,
> > as in the other cases.
>
> What are the circumstances for corresponding benefits?
>
> Regards,
> Markus
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] coccinelle: also catch kzfree() issues
@ 2016-02-16 20:17         ` Julia Lawall
  0 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-16 20:17 UTC (permalink / raw)
  To: cocci



On Tue, 16 Feb 2016, SF Markus Elfring wrote:

> >>  @free@
> >> +identifier kfree =~ "kz?free";
> >
> > Thanks for the suggestions.  However, the regular expression is not such a
> > good idea.
>
> How much is such a SmPL constraint still usable then?
>
>
> > Coccinelle doesn't make any optimizations based on regulat expressions.
>
> Where can your software optimise the source code search?

When the name appears explicitly in the matching code, Coccinelle will
parse and process only files that contain that name.

julia

>
>
> > It would be better to put a disjunction with kfree and kzfree explicitly,
> > as in the other cases.
>
> What are the circumstances for corresponding benefits?
>
> Regards,
> Markus
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: coccinelle: also catch kzfree() issues
  2016-02-16 20:17         ` [Cocci] " Julia Lawall
@ 2016-02-16 20:22           ` SF Markus Elfring
  -1 siblings, 0 replies; 26+ messages in thread
From: SF Markus Elfring @ 2016-02-16 20:22 UTC (permalink / raw)
  To: Julia Lawall
  Cc: Yann Droneaud, Gilles Muller, Nicolas Palix, Michal Marek,
	Tejun Heo, Greg Kroah-Hartman, cocci, linux-kernel,
	Johannes Weiner

>>> Coccinelle doesn't make any optimizations based on regulat expressions.
>>
>> Where can your software optimise the source code search?
> 
> When the name appears explicitly in the matching code, Coccinelle will
> parse and process only files that contain that name.

Does your software perform any file filtering on a passed selection
under special circumstances?

Regards,
Markus

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] coccinelle: also catch kzfree() issues
@ 2016-02-16 20:22           ` SF Markus Elfring
  0 siblings, 0 replies; 26+ messages in thread
From: SF Markus Elfring @ 2016-02-16 20:22 UTC (permalink / raw)
  To: cocci

>>> Coccinelle doesn't make any optimizations based on regulat expressions.
>>
>> Where can your software optimise the source code search?
> 
> When the name appears explicitly in the matching code, Coccinelle will
> parse and process only files that contain that name.

Does your software perform any file filtering on a passed selection
under special circumstances?

Regards,
Markus

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [PATCHv1 1/3] coccinelle: also catch kzfree() issues
  2016-02-16 17:16     ` [Cocci] " Julia Lawall
@ 2016-02-22 14:09       ` Yann Droneaud
  -1 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-22 14:09 UTC (permalink / raw)
  To: Julia Lawall
  Cc: Yann Droneaud, Gilles Muller, Nicolas Palix, Michal Marek,
	Tejun Heo, Greg Kroah-Hartman, Johannes Weiner, cocci,
	linux-kernel

Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
kfree() is no more the only function to be considered.

In particular, kzfree() must not be called on memory
allocated through devm_*() functions.

Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
Hi Julia,

As you suggested, I've use disjunctions instead of regular
expressions (which I managed to use incorrectly: eg.
without ^...$ they catch other functions than kfree(),
such as kfree_skb()).

I've think we should also catch krealloc(, size), where size
is 0, but it's beyond my understanding of coccinelle if size
is not a plain 0 constant.

Perhaps you could help me for this one.

Regards.

 scripts/coccinelle/free/devm_free.cocci |  2 ++
 scripts/coccinelle/free/kfree.cocci     | 18 +++++++++++++++---
 scripts/coccinelle/free/kfreeaddr.cocci |  6 +++++-
 3 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 3d9349012bb3..83c03adec1c5 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -48,6 +48,8 @@ position p;
 (
 * kfree@p(x)
 |
+* kzfree@p(x)
+|
 * free_irq@p(x)
 |
 * iounmap@p(x)
diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
index 577b78056990..ac438da4fd7b 100644
--- a/scripts/coccinelle/free/kfree.cocci
+++ b/scripts/coccinelle/free/kfree.cocci
@@ -20,7 +20,11 @@ expression E;
 position p1;
 @@
 
-kfree@p1(E)
+(
+* kfree@p1(E)
+|
+* kzfree@p1(E)
+)
 
 @print expression@
 constant char [] c;
@@ -60,7 +64,11 @@ position ok;
 @@
 
 while (1) { ...
-  kfree@ok(E)
+(
+* kfree@ok(E)
+|
+* kzfree@ok(E)
+)
   ... when != break;
       when != goto l;
       when forall
@@ -74,7 +82,11 @@ statement S;
 position free.p1!=loop.ok,p2!={print.p,sz.p};
 @@
 
-kfree@p1(E,...)
+(
+* kfree@p1(E,...)
+|
+* kzfree@p1(E,...)
+)
 ...
 (
  iter(...,subE,...) S // no use
diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
index ce8aacc314cb..d46063b1db8b 100644
--- a/scripts/coccinelle/free/kfreeaddr.cocci
+++ b/scripts/coccinelle/free/kfreeaddr.cocci
@@ -16,7 +16,11 @@ identifier f;
 position p;
 @@
 
+(
 * kfree@p(&e->f)
+|
+* kzfree@p(&e->f)
+)
 
 @script:python depends on org@
 p << r.p;
@@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
 p << r.p;
 @@
 
-msg = "ERROR: kfree of structure field"
+msg = "ERROR: invalid free of structure field"
 coccilib.report.print_report(p[0],msg)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Cocci] [PATCHv1 1/3] coccinelle: also catch kzfree() issues
@ 2016-02-22 14:09       ` Yann Droneaud
  0 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-22 14:09 UTC (permalink / raw)
  To: cocci

Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
kfree() is no more the only function to be considered.

In particular, kzfree() must not be called on memory
allocated through devm_*() functions.

Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
---
Hi Julia,

As you suggested, I've use disjunctions instead of regular
expressions (which I managed to use incorrectly: eg.
without ^...$ they catch other functions than kfree(),
such as kfree_skb()).

I've think we should also catch krealloc(, size), where size
is 0, but it's beyond my understanding of coccinelle if size
is not a plain 0 constant.

Perhaps you could help me for this one.

Regards.

 scripts/coccinelle/free/devm_free.cocci |  2 ++
 scripts/coccinelle/free/kfree.cocci     | 18 +++++++++++++++---
 scripts/coccinelle/free/kfreeaddr.cocci |  6 +++++-
 3 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 3d9349012bb3..83c03adec1c5 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -48,6 +48,8 @@ position p;
 (
 * kfree at p(x)
 |
+* kzfree at p(x)
+|
 * free_irq at p(x)
 |
 * iounmap at p(x)
diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
index 577b78056990..ac438da4fd7b 100644
--- a/scripts/coccinelle/free/kfree.cocci
+++ b/scripts/coccinelle/free/kfree.cocci
@@ -20,7 +20,11 @@ expression E;
 position p1;
 @@
 
-kfree at p1(E)
+(
+* kfree at p1(E)
+|
+* kzfree at p1(E)
+)
 
 @print expression@
 constant char [] c;
@@ -60,7 +64,11 @@ position ok;
 @@
 
 while (1) { ...
-  kfree at ok(E)
+(
+* kfree at ok(E)
+|
+* kzfree at ok(E)
+)
   ... when != break;
       when != goto l;
       when forall
@@ -74,7 +82,11 @@ statement S;
 position free.p1!=loop.ok,p2!={print.p,sz.p};
 @@
 
-kfree at p1(E,...)
+(
+* kfree at p1(E,...)
+|
+* kzfree at p1(E,...)
+)
 ...
 (
  iter(...,subE,...) S // no use
diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
index ce8aacc314cb..d46063b1db8b 100644
--- a/scripts/coccinelle/free/kfreeaddr.cocci
+++ b/scripts/coccinelle/free/kfreeaddr.cocci
@@ -16,7 +16,11 @@ identifier f;
 position p;
 @@
 
+(
 * kfree at p(&e->f)
+|
+* kzfree at p(&e->f)
+)
 
 @script:python depends on org@
 p << r.p;
@@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
 p << r.p;
 @@
 
-msg = "ERROR: kfree of structure field"
+msg = "ERROR: invalid free of structure field"
 coccilib.report.print_report(p[0],msg)
-- 
2.5.0

^ permalink raw reply related	[flat|nested] 26+ messages in thread

* Re: [PATCHv1 1/3] coccinelle: also catch kzfree() issues
  2016-02-22 14:09       ` [Cocci] " Yann Droneaud
@ 2016-02-22 14:20         ` Julia Lawall
  -1 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-22 14:20 UTC (permalink / raw)
  To: Yann Droneaud
  Cc: Gilles Muller, Nicolas Palix, Michal Marek, Tejun Heo,
	Greg Kroah-Hartman, Johannes Weiner, cocci, linux-kernel



On Mon, 22 Feb 2016, Yann Droneaud wrote:

> Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
> kfree() is no more the only function to be considered.
>
> In particular, kzfree() must not be called on memory
> allocated through devm_*() functions.
>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
> ---
> Hi Julia,
>
> As you suggested, I've use disjunctions instead of regular
> expressions (which I managed to use incorrectly: eg.
> without ^...$ they catch other functions than kfree(),
> such as kfree_skb()).
>
> I've think we should also catch krealloc(, size), where size
> is 0, but it's beyond my understanding of coccinelle if size
> is not a plain 0 constant.
>
> Perhaps you could help me for this one.

Do you have some examples?  Coccinelle is not very good at tracking
values.  You can say something like:

size = 0
... when != size = e
krealloc(...,size)

I don't know if that would be useful in practice though.

> Regards.
>
>  scripts/coccinelle/free/devm_free.cocci |  2 ++
>  scripts/coccinelle/free/kfree.cocci     | 18 +++++++++++++++---
>  scripts/coccinelle/free/kfreeaddr.cocci |  6 +++++-
>  3 files changed, 22 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 3d9349012bb3..83c03adec1c5 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -48,6 +48,8 @@ position p;
>  (
>  * kfree@p(x)
>  |
> +* kzfree@p(x)
> +|
>  * free_irq@p(x)
>  |
>  * iounmap@p(x)
> diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
> index 577b78056990..ac438da4fd7b 100644
> --- a/scripts/coccinelle/free/kfree.cocci
> +++ b/scripts/coccinelle/free/kfree.cocci
> @@ -20,7 +20,11 @@ expression E;
>  position p1;
>  @@
>
> -kfree@p1(E)
> +(
> +* kfree@p1(E)
> +|
> +* kzfree@p1(E)
> +)
>
>  @print expression@
>  constant char [] c;
> @@ -60,7 +64,11 @@ position ok;
>  @@
>
>  while (1) { ...
> -  kfree@ok(E)
> +(
> +* kfree@ok(E)
> +|
> +* kzfree@ok(E)
> +)
>    ... when != break;
>        when != goto l;
>        when forall
> @@ -74,7 +82,11 @@ statement S;
>  position free.p1!=loop.ok,p2!={print.p,sz.p};
>  @@
>
> -kfree@p1(E,...)
> +(
> +* kfree@p1(E,...)
> +|
> +* kzfree@p1(E,...)
> +)
>  ...
>  (
>   iter(...,subE,...) S // no use
> diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
> index ce8aacc314cb..d46063b1db8b 100644
> --- a/scripts/coccinelle/free/kfreeaddr.cocci
> +++ b/scripts/coccinelle/free/kfreeaddr.cocci
> @@ -16,7 +16,11 @@ identifier f;
>  position p;
>  @@
>
> +(
>  * kfree@p(&e->f)
> +|
> +* kzfree@p(&e->f)
> +)
>
>  @script:python depends on org@
>  p << r.p;
> @@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
>  p << r.p;
>  @@
>
> -msg = "ERROR: kfree of structure field"
> +msg = "ERROR: invalid free of structure field"
>  coccilib.report.print_report(p[0],msg)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] [PATCHv1 1/3] coccinelle: also catch kzfree() issues
@ 2016-02-22 14:20         ` Julia Lawall
  0 siblings, 0 replies; 26+ messages in thread
From: Julia Lawall @ 2016-02-22 14:20 UTC (permalink / raw)
  To: cocci



On Mon, 22 Feb 2016, Yann Droneaud wrote:

> Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
> kfree() is no more the only function to be considered.
>
> In particular, kzfree() must not be called on memory
> allocated through devm_*() functions.
>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
> ---
> Hi Julia,
>
> As you suggested, I've use disjunctions instead of regular
> expressions (which I managed to use incorrectly: eg.
> without ^...$ they catch other functions than kfree(),
> such as kfree_skb()).
>
> I've think we should also catch krealloc(, size), where size
> is 0, but it's beyond my understanding of coccinelle if size
> is not a plain 0 constant.
>
> Perhaps you could help me for this one.

Do you have some examples?  Coccinelle is not very good at tracking
values.  You can say something like:

size = 0
... when != size = e
krealloc(...,size)

I don't know if that would be useful in practice though.

> Regards.
>
>  scripts/coccinelle/free/devm_free.cocci |  2 ++
>  scripts/coccinelle/free/kfree.cocci     | 18 +++++++++++++++---
>  scripts/coccinelle/free/kfreeaddr.cocci |  6 +++++-
>  3 files changed, 22 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
> index 3d9349012bb3..83c03adec1c5 100644
> --- a/scripts/coccinelle/free/devm_free.cocci
> +++ b/scripts/coccinelle/free/devm_free.cocci
> @@ -48,6 +48,8 @@ position p;
>  (
>  * kfree at p(x)
>  |
> +* kzfree at p(x)
> +|
>  * free_irq at p(x)
>  |
>  * iounmap at p(x)
> diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
> index 577b78056990..ac438da4fd7b 100644
> --- a/scripts/coccinelle/free/kfree.cocci
> +++ b/scripts/coccinelle/free/kfree.cocci
> @@ -20,7 +20,11 @@ expression E;
>  position p1;
>  @@
>
> -kfree at p1(E)
> +(
> +* kfree at p1(E)
> +|
> +* kzfree at p1(E)
> +)
>
>  @print expression@
>  constant char [] c;
> @@ -60,7 +64,11 @@ position ok;
>  @@
>
>  while (1) { ...
> -  kfree at ok(E)
> +(
> +* kfree at ok(E)
> +|
> +* kzfree at ok(E)
> +)
>    ... when != break;
>        when != goto l;
>        when forall
> @@ -74,7 +82,11 @@ statement S;
>  position free.p1!=loop.ok,p2!={print.p,sz.p};
>  @@
>
> -kfree at p1(E,...)
> +(
> +* kfree at p1(E,...)
> +|
> +* kzfree at p1(E,...)
> +)
>  ...
>  (
>   iter(...,subE,...) S // no use
> diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
> index ce8aacc314cb..d46063b1db8b 100644
> --- a/scripts/coccinelle/free/kfreeaddr.cocci
> +++ b/scripts/coccinelle/free/kfreeaddr.cocci
> @@ -16,7 +16,11 @@ identifier f;
>  position p;
>  @@
>
> +(
>  * kfree at p(&e->f)
> +|
> +* kzfree at p(&e->f)
> +)
>
>  @script:python depends on org@
>  p << r.p;
> @@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
>  p << r.p;
>  @@
>
> -msg = "ERROR: kfree of structure field"
> +msg = "ERROR: invalid free of structure field"
>  coccilib.report.print_report(p[0],msg)
> --
> 2.5.0
>
>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCHv1 1/3] coccinelle: also catch kzfree() issues
  2016-02-22 14:20         ` [Cocci] " Julia Lawall
@ 2016-02-22 15:24           ` Yann Droneaud
  -1 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-22 15:24 UTC (permalink / raw)
  To: Julia Lawall
  Cc: Gilles Muller, Nicolas Palix, Michal Marek, Tejun Heo,
	Greg Kroah-Hartman, Johannes Weiner, cocci, linux-kernel,
	Yann Droneaud

Le lundi 22 février 2016 à 09:20 -0500, Julia Lawall a écrit :
> On Mon, 22 Feb 2016, Yann Droneaud wrote:
> 
> > Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
> > kfree() is no more the only function to be considered.
> >
> > In particular, kzfree() must not be called on memory
> > allocated through devm_*() functions.
> >
> > Cc: Johannes Weiner <hannes@cmpxchg.org>
> > Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
> > ---
> > Hi Julia,
> >
> > As you suggested, I've use disjunctions instead of regular
> > expressions (which I managed to use incorrectly: eg.
> > without ^...$ they catch other functions than kfree(),
> > such as kfree_skb()).
> >
> > I've think we should also catch krealloc(, size), where size
> > is 0, but it's beyond my understanding of coccinelle if size
> > is not a plain 0 constant.
> >
> > Perhaps you could help me for this one.
> 
> Do you have some examples?

I don't have any real world examples (hopefully) and I don't think it's
going to catch issues, as it's unlikely someone would write
krealloc(ptr, 0) instead of kfree().

>   Coccinelle is not very good at tracking
> values.  You can say something like:
> 
> size = 0
> ... when != size = e
> krealloc(...,size)
> 

It works for the most simple cases I can think of. Thanks a lot !

> I don't know if that would be useful in practice though.
> 

It will be difficult to shoehorn such construct in the dijunctions
added here.

Perhaps we could add a new cocci rules file that would translate such
call to krealloc() to kfree():

@@
expression e;
expression p;
identifier size;
@@
  size = 0
  ... when != size = e
-  krealloc(p,size)
+  kfree(p)


@@
expression p;
@@
-  krealloc(p, 0)
+  kfree(p)

But I'm not sure it worth it.

> > Regards.
> >
> >  scripts/coccinelle/free/devm_free.cocci |  2 ++
> >  scripts/coccinelle/free/kfree.cocci     | 18 +++++++++++++++---
> >  scripts/coccinelle/free/kfreeaddr.cocci |  6 +++++-
> >  3 files changed, 22 insertions(+), 4 deletions(-)
> >
> > diff --git a/scripts/coccinelle/free/devm_free.cocci
> b/scripts/coccinelle/free/devm_free.cocci
> > index 3d9349012bb3..83c03adec1c5 100644
> > --- a/scripts/coccinelle/free/devm_free.cocci
> > +++ b/scripts/coccinelle/free/devm_free.cocci
> > @@ -48,6 +48,8 @@ position p;
> >  (
> >  * kfree@p(x)
> >  |
> > +* kzfree@p(x)
> > +|
> >  * free_irq@p(x)
> >  |
> >  * iounmap@p(x)
> > diff --git a/scripts/coccinelle/free/kfree.cocci
> b/scripts/coccinelle/free/kfree.cocci
> > index 577b78056990..ac438da4fd7b 100644
> > --- a/scripts/coccinelle/free/kfree.cocci
> > +++ b/scripts/coccinelle/free/kfree.cocci
> > @@ -20,7 +20,11 @@ expression E;
> >  position p1;
> >  @@
> >
> > -kfree@p1(E)
> > +(
> > +* kfree@p1(E)
> > +|
> > +* kzfree@p1(E)
> > +)
> >
> >  @print expression@
> >  constant char [] c;
> > @@ -60,7 +64,11 @@ position ok;
> >  @@
> >
> >  while (1) { ...
> > -  kfree@ok(E)
> > +(
> > +* kfree@ok(E)
> > +|
> > +* kzfree@ok(E)
> > +)
> >    ... when != break;
> >        when != goto l;
> >        when forall
> > @@ -74,7 +82,11 @@ statement S;
> >  position free.p1!=loop.ok,p2!={print.p,sz.p};
> >  @@
> >
> > -kfree@p1(E,...)
> > +(
> > +* kfree@p1(E,...)
> > +|
> > +* kzfree@p1(E,...)
> > +)
> >  ...
> >  (
> >   iter(...,subE,...) S // no use
> > diff --git a/scripts/coccinelle/free/kfreeaddr.cocci
> b/scripts/coccinelle/free/kfreeaddr.cocci
> > index ce8aacc314cb..d46063b1db8b 100644
> > --- a/scripts/coccinelle/free/kfreeaddr.cocci
> > +++ b/scripts/coccinelle/free/kfreeaddr.cocci
> > @@ -16,7 +16,11 @@ identifier f;
> >  position p;
> >  @@
> >
> > +(
> >  * kfree@p(&e->f)
> > +|
> > +* kzfree@p(&e->f)
> > +)
> >
> >  @script:python depends on org@
> >  p << r.p;
> > @@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
> >  p << r.p;
> >  @@
> >
> > -msg = "ERROR: kfree of structure field"
> > +msg = "ERROR: invalid free of structure field"
> >  coccilib.report.print_report(p[0],msg)
> > --
> > 2.5.0
> >
> >

Regards.

-- 
Yann Droneaud
OPTEYA

^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Cocci] [PATCHv1 1/3] coccinelle: also catch kzfree() issues
@ 2016-02-22 15:24           ` Yann Droneaud
  0 siblings, 0 replies; 26+ messages in thread
From: Yann Droneaud @ 2016-02-22 15:24 UTC (permalink / raw)
  To: cocci

Le lundi 22 f?vrier 2016 ? 09:20 -0500, Julia Lawall a ?crit?:
> On Mon, 22 Feb 2016, Yann Droneaud wrote:
> 
> > Since commit 3ef0e5ba4673 ('slab: introduce kzfree()'),
> > kfree() is no more the only function to be considered.
> >
> > In particular, kzfree() must not be called on memory
> > allocated through devm_*() functions.
> >
> > Cc: Johannes Weiner <hannes@cmpxchg.org>
> > Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
> > ---
> > Hi Julia,
> >
> > As you suggested, I've use disjunctions instead of regular
> > expressions (which I managed to use incorrectly: eg.
> > without ^...$ they catch other functions than kfree(),
> > such as kfree_skb()).
> >
> > I've think we should also catch krealloc(, size), where size
> > is 0, but it's beyond my understanding of coccinelle if size
> > is not a plain 0 constant.
> >
> > Perhaps you could help me for this one.
> 
> Do you have some examples?

I don't have any real world examples (hopefully) and I don't think it's
going to catch issues, as it's unlikely someone would write
krealloc(ptr, 0) instead of kfree().

> ? Coccinelle is not very good at tracking
> values.? You can say something like:
> 
> size = 0
> ... when != size = e
> krealloc(...,size)
> 

It works for the most simple cases I can think of. Thanks a lot !

> I don't know if that would be useful in practice though.
> 

It will be difficult to shoehorn such construct in the dijunctions
added here.

Perhaps we could add a new cocci rules file that would translate such
call to krealloc() to kfree():

@@
expression e;
expression p;
identifier size;
@@
? size = 0
? ... when != size = e
-??krealloc(p,size)
+??kfree(p)


@@
expression p;
@@
-??krealloc(p, 0)
+??kfree(p)

But I'm not sure it worth it.

> > Regards.
> >
> >? scripts/coccinelle/free/devm_free.cocci |? 2 ++
> >? scripts/coccinelle/free/kfree.cocci???? | 18 +++++++++++++++---
> >? scripts/coccinelle/free/kfreeaddr.cocci |? 6 +++++-
> >? 3 files changed, 22 insertions(+), 4 deletions(-)
> >
> > diff --git a/scripts/coccinelle/free/devm_free.cocci
> b/scripts/coccinelle/free/devm_free.cocci
> > index 3d9349012bb3..83c03adec1c5 100644
> > --- a/scripts/coccinelle/free/devm_free.cocci
> > +++ b/scripts/coccinelle/free/devm_free.cocci
> > @@ -48,6 +48,8 @@ position p;
> >? (
> >? * kfree at p(x)
> >? |
> > +* kzfree at p(x)
> > +|
> >? * free_irq at p(x)
> >? |
> >? * iounmap at p(x)
> > diff --git a/scripts/coccinelle/free/kfree.cocci
> b/scripts/coccinelle/free/kfree.cocci
> > index 577b78056990..ac438da4fd7b 100644
> > --- a/scripts/coccinelle/free/kfree.cocci
> > +++ b/scripts/coccinelle/free/kfree.cocci
> > @@ -20,7 +20,11 @@ expression E;
> >? position p1;
> >? @@
> >
> > -kfree at p1(E)
> > +(
> > +* kfree at p1(E)
> > +|
> > +* kzfree at p1(E)
> > +)
> >
> >? @print expression@
> >? constant char [] c;
> > @@ -60,7 +64,11 @@ position ok;
> >? @@
> >
> >? while (1) { ...
> > -? kfree at ok(E)
> > +(
> > +* kfree at ok(E)
> > +|
> > +* kzfree at ok(E)
> > +)
> >??? ... when != break;
> >??????? when != goto l;
> >??????? when forall
> > @@ -74,7 +82,11 @@ statement S;
> >? position free.p1!=loop.ok,p2!={print.p,sz.p};
> >? @@
> >
> > -kfree at p1(E,...)
> > +(
> > +* kfree at p1(E,...)
> > +|
> > +* kzfree at p1(E,...)
> > +)
> >? ...
> >? (
> >?? iter(...,subE,...) S // no use
> > diff --git a/scripts/coccinelle/free/kfreeaddr.cocci
> b/scripts/coccinelle/free/kfreeaddr.cocci
> > index ce8aacc314cb..d46063b1db8b 100644
> > --- a/scripts/coccinelle/free/kfreeaddr.cocci
> > +++ b/scripts/coccinelle/free/kfreeaddr.cocci
> > @@ -16,7 +16,11 @@ identifier f;
> >? position p;
> >? @@
> >
> > +(
> >? * kfree at p(&e->f)
> > +|
> > +* kzfree at p(&e->f)
> > +)
> >
> >? @script:python depends on org@
> >? p << r.p;
> > @@ -28,5 +32,5 @@ cocci.print_main("kfree",p)
> >? p << r.p;
> >? @@
> >
> > -msg = "ERROR: kfree of structure field"
> > +msg = "ERROR: invalid free of structure field"
> >? coccilib.report.print_report(p[0],msg)
> > --
> > 2.5.0
> >
> >

Regards.

--?
Yann Droneaud
OPTEYA

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2016-02-22 15:24 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-16 17:01 [PATCH 0/3] coccinelle: catchup on memory allocation functions Yann Droneaud
2016-02-16 17:01 ` [Cocci] " Yann Droneaud
2016-02-16 17:06 ` [PATCH 1/3] coccinelle: also catch kzfree() issues Yann Droneaud
2016-02-16 17:06   ` [Cocci] " Yann Droneaud
2016-02-16 17:16   ` Julia Lawall
2016-02-16 17:16     ` [Cocci] " Julia Lawall
2016-02-16 20:02     ` SF Markus Elfring
2016-02-16 20:02       ` [Cocci] " SF Markus Elfring
2016-02-16 20:17       ` Julia Lawall
2016-02-16 20:17         ` [Cocci] " Julia Lawall
2016-02-16 20:22         ` SF Markus Elfring
2016-02-16 20:22           ` [Cocci] " SF Markus Elfring
2016-02-22 14:09     ` [PATCHv1 1/3] " Yann Droneaud
2016-02-22 14:09       ` [Cocci] " Yann Droneaud
2016-02-22 14:20       ` Julia Lawall
2016-02-22 14:20         ` [Cocci] " Julia Lawall
2016-02-22 15:24         ` Yann Droneaud
2016-02-22 15:24           ` [Cocci] " Yann Droneaud
2016-02-16 17:06 ` [PATCH 2/3] coccinelle: recognize more devm_* memory allocation functions Yann Droneaud
2016-02-16 17:06   ` [Cocci] " Yann Droneaud
2016-02-16 17:18   ` Julia Lawall
2016-02-16 17:18     ` [Cocci] " Julia Lawall
2016-02-16 17:06 ` [PATCH 3/3] coccinelle: catch krealloc() on devm_*() allocated memory Yann Droneaud
2016-02-16 17:06   ` [Cocci] " Yann Droneaud
2016-02-16 17:19   ` Julia Lawall
2016-02-16 17:19     ` [Cocci] " Julia Lawall

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.