All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Keeping <john@metanate.com>
To: Corentin Labbe <clabbe@baylibre.com>
Cc: heiko@sntech.de, herbert@gondor.apana.org.au, krzk+dt@kernel.org,
	robh+dt@kernel.org, devicetree@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-rockchip@lists.infradead.org
Subject: Re: [PATCH v4 06/33] crypto: rockchip: add fallback for cipher
Date: Mon, 4 Apr 2022 12:26:15 +0100	[thread overview]
Message-ID: <YkrV1z5GPVXc+d/X@donbot> (raw)
In-Reply-To: <20220401201804.2867154-7-clabbe@baylibre.com>

On Fri, Apr 01, 2022 at 08:17:37PM +0000, Corentin Labbe wrote:
> The hardware does not handle 0 size length request, let's add a
> fallback.
> Furthermore fallback will be used for all unaligned case the hardware
> cannot handle.
> 
> Fixes: ce0183cb6464b ("crypto: rockchip - switch to skcipher API")
> Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
> ---
> diff --git a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> index bbd0bf52bf07..c6b601086c04 100644
> --- a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> +++ b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> @@ -13,6 +13,71 @@
>  
>  #define RK_CRYPTO_DEC			BIT(0)
>  
> +static int rk_cipher_need_fallback(struct skcipher_request *req)
> +{
> +	struct scatterlist *sgs, *sgd;
> +	unsigned int todo, len;
> +	unsigned int bs = crypto_skcipher_blocksize(tfm);
> +
> +	if (!req->cryptlen)
> +		return true;
> +
> +	len = req->cryptlen;
> +	sgs = req->src;
> +	while (sgs) {
> +		if (!IS_ALIGNED(sgs->offset, sizeof(u32))) {
> +			return true;
> +		}
> +		todo = min(len, sgs->length);
> +		if (todo % bs) {
> +			return true;
> +		}
> +		len -= todo;
> +		sgs = sg_next(sgs);
> +	}
> +	len = req->cryptlen;
> +	sgd = req->dst;
> +	while (sgd) {
> +		if (!IS_ALIGNED(sgd->offset, sizeof(u32))) {
> +			return true;
> +		}
> +		todo = min(len, sgd->length);
> +		if (todo % bs) {
> +			return true;
> +		}
> +		len -= todo;
> +		sgd = sg_next(sgd);
> +	}
> +	sgs = req->src;
> +	sgd = req->dst;
> +	while (sgs && sgd) {
> +		if (sgs->length != sgd->length)

This check still seems to be triggering the fallback when it is not
needed.

I've done some testing with fscrypt and the series is working great, but
the stats show the fallback triggering more than I'd expect.  With some
extra logging here I see output like:

	sgs->length=32 sgd->length=255 req->cryptlen=16

In this case sgs and sgd are both the first (and only) entries in the
list.  Should this take account of req->cryptlen as well?

In fact, can't this whole function be folded into one loop over src and
dst at the same time, since all the checks must be the same?  Something
like this (untested):

	while (sgs && sgd) {
		if (!IS_ALIGNED(sgs->offset, sizeof(u32)) ||
		    !IS_ALIGNED(sgd->offset, sizeof(u32)))
			return true;

		todo = min(len, sgs->length);
		if (todo % bs)
			return true;

		if (sgd->length < todo)
			return true;

		len -= todo;
		sgs = sg_next(sgs);
		sgd = sg_next(sgd);
	}

	if (len)
		return true;

> +			return true;
> +		sgs = sg_next(sgs);
> +		sgd = sg_next(sgd);
> +	}
> +	return false;
> +}

WARNING: multiple messages have this Message-ID (diff)
From: John Keeping <john@metanate.com>
To: Corentin Labbe <clabbe@baylibre.com>
Cc: heiko@sntech.de, herbert@gondor.apana.org.au, krzk+dt@kernel.org,
	robh+dt@kernel.org, devicetree@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-rockchip@lists.infradead.org
Subject: Re: [PATCH v4 06/33] crypto: rockchip: add fallback for cipher
Date: Mon, 4 Apr 2022 12:26:15 +0100	[thread overview]
Message-ID: <YkrV1z5GPVXc+d/X@donbot> (raw)
In-Reply-To: <20220401201804.2867154-7-clabbe@baylibre.com>

On Fri, Apr 01, 2022 at 08:17:37PM +0000, Corentin Labbe wrote:
> The hardware does not handle 0 size length request, let's add a
> fallback.
> Furthermore fallback will be used for all unaligned case the hardware
> cannot handle.
> 
> Fixes: ce0183cb6464b ("crypto: rockchip - switch to skcipher API")
> Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
> ---
> diff --git a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> index bbd0bf52bf07..c6b601086c04 100644
> --- a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> +++ b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> @@ -13,6 +13,71 @@
>  
>  #define RK_CRYPTO_DEC			BIT(0)
>  
> +static int rk_cipher_need_fallback(struct skcipher_request *req)
> +{
> +	struct scatterlist *sgs, *sgd;
> +	unsigned int todo, len;
> +	unsigned int bs = crypto_skcipher_blocksize(tfm);
> +
> +	if (!req->cryptlen)
> +		return true;
> +
> +	len = req->cryptlen;
> +	sgs = req->src;
> +	while (sgs) {
> +		if (!IS_ALIGNED(sgs->offset, sizeof(u32))) {
> +			return true;
> +		}
> +		todo = min(len, sgs->length);
> +		if (todo % bs) {
> +			return true;
> +		}
> +		len -= todo;
> +		sgs = sg_next(sgs);
> +	}
> +	len = req->cryptlen;
> +	sgd = req->dst;
> +	while (sgd) {
> +		if (!IS_ALIGNED(sgd->offset, sizeof(u32))) {
> +			return true;
> +		}
> +		todo = min(len, sgd->length);
> +		if (todo % bs) {
> +			return true;
> +		}
> +		len -= todo;
> +		sgd = sg_next(sgd);
> +	}
> +	sgs = req->src;
> +	sgd = req->dst;
> +	while (sgs && sgd) {
> +		if (sgs->length != sgd->length)

This check still seems to be triggering the fallback when it is not
needed.

I've done some testing with fscrypt and the series is working great, but
the stats show the fallback triggering more than I'd expect.  With some
extra logging here I see output like:

	sgs->length=32 sgd->length=255 req->cryptlen=16

In this case sgs and sgd are both the first (and only) entries in the
list.  Should this take account of req->cryptlen as well?

In fact, can't this whole function be folded into one loop over src and
dst at the same time, since all the checks must be the same?  Something
like this (untested):

	while (sgs && sgd) {
		if (!IS_ALIGNED(sgs->offset, sizeof(u32)) ||
		    !IS_ALIGNED(sgd->offset, sizeof(u32)))
			return true;

		todo = min(len, sgs->length);
		if (todo % bs)
			return true;

		if (sgd->length < todo)
			return true;

		len -= todo;
		sgs = sg_next(sgs);
		sgd = sg_next(sgd);
	}

	if (len)
		return true;

> +			return true;
> +		sgs = sg_next(sgs);
> +		sgd = sg_next(sgd);
> +	}
> +	return false;
> +}

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

WARNING: multiple messages have this Message-ID (diff)
From: John Keeping <john@metanate.com>
To: Corentin Labbe <clabbe@baylibre.com>
Cc: heiko@sntech.de, herbert@gondor.apana.org.au, krzk+dt@kernel.org,
	robh+dt@kernel.org, devicetree@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-rockchip@lists.infradead.org
Subject: Re: [PATCH v4 06/33] crypto: rockchip: add fallback for cipher
Date: Mon, 4 Apr 2022 12:26:15 +0100	[thread overview]
Message-ID: <YkrV1z5GPVXc+d/X@donbot> (raw)
In-Reply-To: <20220401201804.2867154-7-clabbe@baylibre.com>

On Fri, Apr 01, 2022 at 08:17:37PM +0000, Corentin Labbe wrote:
> The hardware does not handle 0 size length request, let's add a
> fallback.
> Furthermore fallback will be used for all unaligned case the hardware
> cannot handle.
> 
> Fixes: ce0183cb6464b ("crypto: rockchip - switch to skcipher API")
> Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
> ---
> diff --git a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> index bbd0bf52bf07..c6b601086c04 100644
> --- a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> +++ b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
> @@ -13,6 +13,71 @@
>  
>  #define RK_CRYPTO_DEC			BIT(0)
>  
> +static int rk_cipher_need_fallback(struct skcipher_request *req)
> +{
> +	struct scatterlist *sgs, *sgd;
> +	unsigned int todo, len;
> +	unsigned int bs = crypto_skcipher_blocksize(tfm);
> +
> +	if (!req->cryptlen)
> +		return true;
> +
> +	len = req->cryptlen;
> +	sgs = req->src;
> +	while (sgs) {
> +		if (!IS_ALIGNED(sgs->offset, sizeof(u32))) {
> +			return true;
> +		}
> +		todo = min(len, sgs->length);
> +		if (todo % bs) {
> +			return true;
> +		}
> +		len -= todo;
> +		sgs = sg_next(sgs);
> +	}
> +	len = req->cryptlen;
> +	sgd = req->dst;
> +	while (sgd) {
> +		if (!IS_ALIGNED(sgd->offset, sizeof(u32))) {
> +			return true;
> +		}
> +		todo = min(len, sgd->length);
> +		if (todo % bs) {
> +			return true;
> +		}
> +		len -= todo;
> +		sgd = sg_next(sgd);
> +	}
> +	sgs = req->src;
> +	sgd = req->dst;
> +	while (sgs && sgd) {
> +		if (sgs->length != sgd->length)

This check still seems to be triggering the fallback when it is not
needed.

I've done some testing with fscrypt and the series is working great, but
the stats show the fallback triggering more than I'd expect.  With some
extra logging here I see output like:

	sgs->length=32 sgd->length=255 req->cryptlen=16

In this case sgs and sgd are both the first (and only) entries in the
list.  Should this take account of req->cryptlen as well?

In fact, can't this whole function be folded into one loop over src and
dst at the same time, since all the checks must be the same?  Something
like this (untested):

	while (sgs && sgd) {
		if (!IS_ALIGNED(sgs->offset, sizeof(u32)) ||
		    !IS_ALIGNED(sgd->offset, sizeof(u32)))
			return true;

		todo = min(len, sgs->length);
		if (todo % bs)
			return true;

		if (sgd->length < todo)
			return true;

		len -= todo;
		sgs = sg_next(sgs);
		sgd = sg_next(sgd);
	}

	if (len)
		return true;

> +			return true;
> +		sgs = sg_next(sgs);
> +		sgd = sg_next(sgd);
> +	}
> +	return false;
> +}

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2022-04-04 11:26 UTC|newest]

Thread overview: 157+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-04-01 20:17 [PATCH v4 00/33] crypto: rockchip: permit to pass self-tests Corentin Labbe
2022-04-01 20:17 ` Corentin Labbe
2022-04-01 20:17 ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 01/33] crypto: rockchip: use dev_err for error message about interrupt Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 02/33] crypto: rockchip: do not use uninitialized variable Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 03/33] crypto: rockchip: do not do custom power management Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 04/33] crypto: rockchip: fix privete/private typo Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 05/33] crypto: rockchip: do not store mode globally Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 06/33] crypto: rockchip: add fallback for cipher Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-04 11:26   ` John Keeping [this message]
2022-04-04 11:26     ` John Keeping
2022-04-04 11:26     ` John Keeping
2022-04-11  8:42     ` LABBE Corentin
2022-04-11  8:42       ` LABBE Corentin
2022-04-11  8:42       ` LABBE Corentin
2022-04-01 20:17 ` [PATCH v4 07/33] crypto: rockchip: add fallback for ahash Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 08/33] crypto: rockchip: better handle cipher key Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-04 11:31   ` John Keeping
2022-04-04 11:31     ` John Keeping
2022-04-04 11:31     ` John Keeping
2022-04-11  8:51     ` LABBE Corentin
2022-04-11  8:51       ` LABBE Corentin
2022-04-11  8:51       ` LABBE Corentin
2022-04-01 20:17 ` [PATCH v4 09/33] crypto: rockchip: remove non-aligned handling Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 10/33] crypto: rockchip: rework by using crypto_engine Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-02  8:22   ` kernel test robot
2022-04-02  8:22     ` kernel test robot
2022-04-02  8:22     ` kernel test robot
2022-04-02 20:04     ` LABBE Corentin
2022-04-02 20:04       ` LABBE Corentin
2022-04-02 20:04       ` LABBE Corentin
2022-04-02 20:04       ` LABBE Corentin
2022-04-01 20:17 ` [PATCH v4 11/33] crypto: rockchip: rewrite type Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 12/33] crypto: rockchip: add debugfs Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 13/33] crypto: rockchip: introduce PM Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 14/33] crypto: rockchip: handle reset also in PM Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 15/33] crypto: rockchip: use clk_bulk to simplify clock management Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 16/33] crypto: rockchip: add myself as maintainer Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 17/33] crypto: rockchip: use read_poll_timeout Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-04 11:38   ` John Keeping
2022-04-04 11:38     ` John Keeping
2022-04-04 11:38     ` John Keeping
2022-04-11  8:33     ` LABBE Corentin
2022-04-11  8:33       ` LABBE Corentin
2022-04-11  8:33       ` LABBE Corentin
2022-04-01 20:17 ` [PATCH v4 18/33] crypto: rockchip: fix style issue Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-04 11:39   ` John Keeping
2022-04-04 11:39     ` John Keeping
2022-04-04 11:39     ` John Keeping
2022-04-11  7:40     ` LABBE Corentin
2022-04-11  7:40       ` LABBE Corentin
2022-04-11  7:40       ` LABBE Corentin
2022-04-01 20:17 ` [PATCH v4 19/33] crypto: rockchip: add support for rk3328 Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 20/33] crypto: rockchip: rename ablk functions to cipher Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 21/33] crypto: rockchip: rework rk_handle_req function Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 22/33] crypto: rockchip: use a rk_crypto_info variable instead of lot of indirection Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 23/33] crypto: rockchip: use the rk_crypto_info given as parameter Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 24/33] crypto: rockchip: rename crypto_info to main in TFM context Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 25/33] crypto: rockchip: store crypto_info in request context Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 26/33] crypto: rockchip: Add support for rk3399 Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17 ` [PATCH v4 27/33] dt-bindings: crypto: convert rockchip-crypto to yaml Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-02 12:10   ` Krzysztof Kozlowski
2022-04-02 12:10     ` Krzysztof Kozlowski
2022-04-02 12:10     ` Krzysztof Kozlowski
2022-04-02 20:11     ` LABBE Corentin
2022-04-02 20:11       ` LABBE Corentin
2022-04-02 20:11       ` LABBE Corentin
2022-04-01 20:17 ` [PATCH v4 28/33] dt-bindings: crypto: rockchip: convert to new driver bindings Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-01 20:17   ` Corentin Labbe
2022-04-02 11:53   ` Krzysztof Kozlowski
2022-04-02 11:53     ` Krzysztof Kozlowski
2022-04-02 11:53     ` Krzysztof Kozlowski
2022-04-02 12:07     ` Krzysztof Kozlowski
2022-04-02 12:07       ` Krzysztof Kozlowski
2022-04-02 12:07       ` Krzysztof Kozlowski
2022-04-03 18:58       ` LABBE Corentin
2022-04-03 18:58         ` LABBE Corentin
2022-04-03 18:58         ` LABBE Corentin
2022-04-02 20:10     ` LABBE Corentin
2022-04-02 20:10       ` LABBE Corentin
2022-04-02 20:10       ` LABBE Corentin
2022-04-02 20:19       ` Krzysztof Kozlowski
2022-04-02 20:19         ` Krzysztof Kozlowski
2022-04-02 20:19         ` Krzysztof Kozlowski
2022-04-01 20:18 ` [PATCH v4 29/33] clk: rk3399: use proper crypto0 name Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-06 16:07   ` Rob Herring
2022-04-06 16:07     ` Rob Herring
2022-04-06 16:07     ` Rob Herring
2022-04-01 20:18 ` [PATCH v4 30/33] ARM: dts: rk3288: crypto does not need reset-names anymore Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18 ` [PATCH v4 31/33] arm64: dts: rockchip: add rk3328 crypto node Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18 ` [PATCH v4 32/33] arm64: dts: rockchip: rk3399: add " Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18 ` [PATCH v4 33/33] crypto: rockchip: Check for clocks numbers and their frequencies Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe
2022-04-01 20:18   ` Corentin Labbe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YkrV1z5GPVXc+d/X@donbot \
    --to=john@metanate.com \
    --cc=clabbe@baylibre.com \
    --cc=devicetree@vger.kernel.org \
    --cc=heiko@sntech.de \
    --cc=herbert@gondor.apana.org.au \
    --cc=krzk+dt@kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rockchip@lists.infradead.org \
    --cc=robh+dt@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.