All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mike Leach <mike.leach@linaro.org>
To: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
	Coresight ML <coresight@lists.linaro.org>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	"Suzuki K. Poulose" <suzuki.poulose@arm.com>,
	Linu Cherian <lcherian@marvell.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Leo Yan <leo.yan@linaro.org>
Subject: Re: [PATCH V3 14/14] coresight: etm-perf: Add support for trace buffer format
Date: Tue, 16 Feb 2021 11:01:42 +0000	[thread overview]
Message-ID: <CAJ9a7Vi-mJ_sccLFyDQrUo8opEfGnZG-s0Atw=c0Ukt4kVX5xA@mail.gmail.com> (raw)
In-Reply-To: <1611737738-1493-15-git-send-email-anshuman.khandual@arm.com>

On Wed, 27 Jan 2021 at 08:56, Anshuman Khandual
<anshuman.khandual@arm.com> wrote:
>
> From: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> CoreSight PMU supports aux-buffer for the ETM tracing. The trace
> generated by the ETM (associated with individual CPUs, like Intel PT)
> is captured by a separate IP (CoreSight TMC-ETR/ETF until now).
>
> The TMC-ETR applies formatting of the raw ETM trace data, as it
> can collect traces from multiple ETMs, with the TraceID to indicate
> the source of a given trace packet.
>
> Arm Trace Buffer Extension is new "sink" IP, attached to individual
> CPUs and thus do not provide additional formatting, like TMC-ETR.
>
> Additionally, a system could have both TRBE *and* TMC-ETR for
> the trace collection. e.g, TMC-ETR could be used as a single
> trace buffer to collect data from multiple ETMs to correlate
> the traces from different CPUs. It is possible to have a
> perf session where some events end up collecting the trace
> in TMC-ETR while the others in TRBE. Thus we need a way
> to identify the type of the trace for each AUX record.
>
> Define the trace formats exported by the CoreSight PMU.
> We don't define the flags following the "ETM" as this
> information is available to the user when issuing
> the session. What is missing is the additional
> formatting applied by the "sink" which is decided
> at the runtime and the user may not have a control on.
>
> So we define :
>  - CORESIGHT format (indicates the Frame format)
>  - RAW format (indicates the format of the source)
>
> The default value is CORESIGHT format for all the records
> (i,e == 0). Add the RAW format for the TRBE sink driver.
>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Mike Leach <mike.leach@linaro.org>
> Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
> Cc: Leo Yan <leo.yan@linaro.org>
> Cc: Anshuman Khandual <anshuman.khandual@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
> ---
>  drivers/hwtracing/coresight/coresight-trbe.c | 2 ++
>  include/uapi/linux/perf_event.h              | 4 ++++
>  2 files changed, 6 insertions(+)
>
> diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c
> index 1464d8b..7c0e691 100644
> --- a/drivers/hwtracing/coresight/coresight-trbe.c
> +++ b/drivers/hwtracing/coresight/coresight-trbe.c
> @@ -511,6 +511,7 @@ static unsigned long arm_trbe_update_buffer(struct coresight_device *csdev,
>         if (cpudata->mode != CS_MODE_PERF)
>                 return -EINVAL;
>
> +       perf_aux_output_flag(handle, PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW);
>         /*
>          * If the TRBE was disabled due to lack of space in the AUX buffer or a
>          * spurious fault, the driver leaves it disabled, truncating the buffer.
> @@ -606,6 +607,7 @@ static void trbe_handle_overflow(struct perf_output_handle *handle)
>         size = offset - PERF_IDX2OFF(handle->head, buf);
>         if (buf->snapshot)
>                 handle->head = offset;
> +       perf_aux_output_flag(handle, PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW);
>         perf_aux_output_end(handle, size);
>
>         event_data = perf_aux_output_begin(handle, event);
> diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> index 9a5ca45..169e6b3 100644
> --- a/include/uapi/linux/perf_event.h
> +++ b/include/uapi/linux/perf_event.h
> @@ -1111,6 +1111,10 @@ enum perf_callchain_context {
>  #define PERF_AUX_FLAG_COLLISION                        0x08    /* sample collided with another */
>  #define PERF_AUX_FLAG_PMU_FORMAT_TYPE_MASK     0xff00  /* PMU specific trace format type */
>
> +/* CoreSight PMU AUX buffer formats */
> +#define PERF_AUX_FLAG_CORESIGHT_FORMAT_CORESIGHT       0x0000 /* Default for backward compatibility */
> +#define PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW             0x0100 /* Raw format of the source */
> +
>  #define PERF_FLAG_FD_NO_GROUP          (1UL << 0)
>  #define PERF_FLAG_FD_OUTPUT            (1UL << 1)
>  #define PERF_FLAG_PID_CGROUP           (1UL << 2) /* pid=cgroup id, per-cpu mode only */
> --
> 2.7.4
>

Reviewed-by: Mike Leach <mike.leach@linaro.org>
-- 
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK

WARNING: multiple messages have this Message-ID (diff)
From: Mike Leach <mike.leach@linaro.org>
To: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>,
	"Suzuki K. Poulose" <suzuki.poulose@arm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Coresight ML <coresight@lists.linaro.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
	Leo Yan <leo.yan@linaro.org>, Linu Cherian <lcherian@marvell.com>
Subject: Re: [PATCH V3 14/14] coresight: etm-perf: Add support for trace buffer format
Date: Tue, 16 Feb 2021 11:01:42 +0000	[thread overview]
Message-ID: <CAJ9a7Vi-mJ_sccLFyDQrUo8opEfGnZG-s0Atw=c0Ukt4kVX5xA@mail.gmail.com> (raw)
In-Reply-To: <1611737738-1493-15-git-send-email-anshuman.khandual@arm.com>

On Wed, 27 Jan 2021 at 08:56, Anshuman Khandual
<anshuman.khandual@arm.com> wrote:
>
> From: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> CoreSight PMU supports aux-buffer for the ETM tracing. The trace
> generated by the ETM (associated with individual CPUs, like Intel PT)
> is captured by a separate IP (CoreSight TMC-ETR/ETF until now).
>
> The TMC-ETR applies formatting of the raw ETM trace data, as it
> can collect traces from multiple ETMs, with the TraceID to indicate
> the source of a given trace packet.
>
> Arm Trace Buffer Extension is new "sink" IP, attached to individual
> CPUs and thus do not provide additional formatting, like TMC-ETR.
>
> Additionally, a system could have both TRBE *and* TMC-ETR for
> the trace collection. e.g, TMC-ETR could be used as a single
> trace buffer to collect data from multiple ETMs to correlate
> the traces from different CPUs. It is possible to have a
> perf session where some events end up collecting the trace
> in TMC-ETR while the others in TRBE. Thus we need a way
> to identify the type of the trace for each AUX record.
>
> Define the trace formats exported by the CoreSight PMU.
> We don't define the flags following the "ETM" as this
> information is available to the user when issuing
> the session. What is missing is the additional
> formatting applied by the "sink" which is decided
> at the runtime and the user may not have a control on.
>
> So we define :
>  - CORESIGHT format (indicates the Frame format)
>  - RAW format (indicates the format of the source)
>
> The default value is CORESIGHT format for all the records
> (i,e == 0). Add the RAW format for the TRBE sink driver.
>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Mike Leach <mike.leach@linaro.org>
> Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
> Cc: Leo Yan <leo.yan@linaro.org>
> Cc: Anshuman Khandual <anshuman.khandual@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
> ---
>  drivers/hwtracing/coresight/coresight-trbe.c | 2 ++
>  include/uapi/linux/perf_event.h              | 4 ++++
>  2 files changed, 6 insertions(+)
>
> diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c
> index 1464d8b..7c0e691 100644
> --- a/drivers/hwtracing/coresight/coresight-trbe.c
> +++ b/drivers/hwtracing/coresight/coresight-trbe.c
> @@ -511,6 +511,7 @@ static unsigned long arm_trbe_update_buffer(struct coresight_device *csdev,
>         if (cpudata->mode != CS_MODE_PERF)
>                 return -EINVAL;
>
> +       perf_aux_output_flag(handle, PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW);
>         /*
>          * If the TRBE was disabled due to lack of space in the AUX buffer or a
>          * spurious fault, the driver leaves it disabled, truncating the buffer.
> @@ -606,6 +607,7 @@ static void trbe_handle_overflow(struct perf_output_handle *handle)
>         size = offset - PERF_IDX2OFF(handle->head, buf);
>         if (buf->snapshot)
>                 handle->head = offset;
> +       perf_aux_output_flag(handle, PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW);
>         perf_aux_output_end(handle, size);
>
>         event_data = perf_aux_output_begin(handle, event);
> diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> index 9a5ca45..169e6b3 100644
> --- a/include/uapi/linux/perf_event.h
> +++ b/include/uapi/linux/perf_event.h
> @@ -1111,6 +1111,10 @@ enum perf_callchain_context {
>  #define PERF_AUX_FLAG_COLLISION                        0x08    /* sample collided with another */
>  #define PERF_AUX_FLAG_PMU_FORMAT_TYPE_MASK     0xff00  /* PMU specific trace format type */
>
> +/* CoreSight PMU AUX buffer formats */
> +#define PERF_AUX_FLAG_CORESIGHT_FORMAT_CORESIGHT       0x0000 /* Default for backward compatibility */
> +#define PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW             0x0100 /* Raw format of the source */
> +
>  #define PERF_FLAG_FD_NO_GROUP          (1UL << 0)
>  #define PERF_FLAG_FD_OUTPUT            (1UL << 1)
>  #define PERF_FLAG_PID_CGROUP           (1UL << 2) /* pid=cgroup id, per-cpu mode only */
> --
> 2.7.4
>

Reviewed-by: Mike Leach <mike.leach@linaro.org>
-- 
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  parent reply	other threads:[~2021-02-16 11:05 UTC|newest]

Thread overview: 181+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-27  8:55 [PATCH V3 00/14] arm64: coresight: Enable ETE and TRBE Anshuman Khandual
2021-01-27  8:55 ` Anshuman Khandual
2021-01-27  8:55 ` [PATCH V3 01/14] coresight: etm-perf: Allow an event to use different sinks Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-01 23:17   ` Mathieu Poirier
2021-02-01 23:17     ` Mathieu Poirier
2021-02-02  9:42     ` Suzuki K Poulose
2021-02-02  9:42       ` Suzuki K Poulose
2021-02-02 16:33       ` Mike Leach
2021-02-02 16:33         ` Mike Leach
2021-02-02 22:41         ` Suzuki K Poulose
2021-02-02 22:41           ` Suzuki K Poulose
2021-02-04 12:27           ` Mike Leach
2021-02-04 12:27             ` Mike Leach
2021-02-02 16:37       ` Mathieu Poirier
2021-02-02 16:37         ` Mathieu Poirier
2021-01-27  8:55 ` [PATCH V3 02/14] coresight: Do not scan for graph if none is present Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-01 23:44   ` Mathieu Poirier
2021-02-01 23:44     ` Mathieu Poirier
2021-02-02 11:10   ` Mike Leach
2021-02-02 11:10     ` Mike Leach
2021-02-02 14:36     ` Suzuki K Poulose
2021-02-02 14:36       ` Suzuki K Poulose
2021-01-27  8:55 ` [PATCH V3 03/14] coresight: etm4x: Add support for PE OS lock Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-02 17:40   ` Mathieu Poirier
2021-02-02 17:40     ` Mathieu Poirier
2021-02-02 18:03   ` Mathieu Poirier
2021-02-02 18:03     ` Mathieu Poirier
2021-02-15 14:08   ` Mike Leach
2021-02-15 14:08     ` Mike Leach
2021-01-27  8:55 ` [PATCH V3 04/14] coresight: ete: Add support for ETE sysreg access Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-02 17:52   ` Mathieu Poirier
2021-02-02 17:52     ` Mathieu Poirier
2021-02-03 15:51     ` Suzuki K Poulose
2021-02-03 15:51       ` Suzuki K Poulose
2021-02-15 14:08   ` Mike Leach
2021-02-15 14:08     ` Mike Leach
2021-01-27  8:55 ` [PATCH V3 05/14] coresight: ete: Add support for ETE tracing Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-02 18:56   ` Mathieu Poirier
2021-02-02 18:56     ` Mathieu Poirier
2021-02-02 22:50     ` Suzuki K Poulose
2021-02-02 22:50       ` Suzuki K Poulose
2021-02-15 13:21     ` Mike Leach
2021-02-15 13:21       ` Mike Leach
2021-02-15 14:08       ` Mike Leach
2021-02-15 14:08         ` Mike Leach
2021-01-27  8:55 ` [PATCH V3 06/14] dts: bindings: Document device tree bindings for ETE Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-09 19:00   ` Rob Herring
2021-02-09 19:00     ` Rob Herring
2021-02-10 12:33     ` Suzuki K Poulose
2021-02-10 12:33       ` Suzuki K Poulose
2021-02-18 18:33       ` Rob Herring
2021-02-18 18:33         ` Rob Herring
2021-02-18 22:51         ` Suzuki K Poulose
2021-02-18 22:51           ` Suzuki K Poulose
2021-01-27  8:55 ` [PATCH V3 07/14] coresight: etm-perf: Handle stale output handles Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-03 19:05   ` Mathieu Poirier
2021-02-03 19:05     ` Mathieu Poirier
2021-02-03 23:36     ` Suzuki K Poulose
2021-02-03 23:36       ` Suzuki K Poulose
2021-02-15 16:27   ` Mike Leach
2021-02-15 16:27     ` Mike Leach
2021-01-27  8:55 ` [PATCH V3 08/14] coresight: core: Add support for dedicated percpu sinks Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-01-28  9:16   ` Suzuki K Poulose
2021-01-28  9:16     ` Suzuki K Poulose
2021-02-04 18:34     ` Mathieu Poirier
2021-02-04 18:34       ` Mathieu Poirier
2021-02-16 10:40       ` Anshuman Khandual
2021-02-16 10:40         ` Anshuman Khandual
2021-02-16 20:44         ` Mathieu Poirier
2021-02-16 20:44           ` Mathieu Poirier
2021-02-16 10:21     ` Anshuman Khandual
2021-02-16 10:21       ` Anshuman Khandual
2021-02-15 16:27   ` Mike Leach
2021-02-15 16:27     ` Mike Leach
2021-02-15 16:56     ` Mathieu Poirier
2021-02-15 16:56       ` Mathieu Poirier
2021-02-15 17:58       ` Mike Leach
2021-02-15 17:58         ` Mike Leach
2021-02-16 20:30         ` Mathieu Poirier
2021-02-16 20:30           ` Mathieu Poirier
2021-01-27  8:55 ` [PATCH V3 09/14] arm64: Add TRBE definitions Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-01-28  9:31   ` Suzuki K Poulose
2021-01-28  9:31     ` Suzuki K Poulose
2021-01-28 17:18   ` Catalin Marinas
2021-01-28 17:18     ` Catalin Marinas
2021-02-15 18:06     ` Mike Leach
2021-02-15 18:06       ` Mike Leach
2021-01-27  8:55 ` [PATCH V3 10/14] arm64: nvhe: Allow TRBE access at EL1 Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-01-27  9:58   ` Marc Zyngier
2021-01-27  9:58     ` Marc Zyngier
2021-01-28  9:34     ` Suzuki K Poulose
2021-01-28  9:34       ` Suzuki K Poulose
2021-01-28  9:46       ` Marc Zyngier
2021-01-28  9:46         ` Marc Zyngier
2021-01-28  9:48         ` Suzuki K Poulose
2021-01-28  9:48           ` Suzuki K Poulose
2021-01-27  8:55 ` [PATCH V3 11/14] coresight: sink: Add TRBE driver Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-01-29 10:23   ` Suzuki K Poulose
2021-02-02  5:55     ` Anshuman Khandual
2021-02-02  5:55       ` Anshuman Khandual
2021-02-05 17:53   ` Mathieu Poirier
2021-02-05 17:53     ` Mathieu Poirier
2021-02-08  4:20     ` Anshuman Khandual
2021-02-08  4:20       ` Anshuman Khandual
2021-02-09 17:39     ` Mathieu Poirier
2021-02-09 17:39       ` Mathieu Poirier
2021-02-10  4:12       ` Anshuman Khandual
2021-02-10  4:12         ` Anshuman Khandual
2021-02-10 16:54         ` Mathieu Poirier
2021-02-10 16:54           ` Mathieu Poirier
2021-02-10 19:00   ` Mathieu Poirier
2021-02-10 19:00     ` Mathieu Poirier
2021-02-12  5:43     ` Anshuman Khandual
2021-02-12  5:43       ` Anshuman Khandual
2021-02-12 17:02       ` Mathieu Poirier
2021-02-12 17:02         ` Mathieu Poirier
2021-02-11 19:00   ` Mathieu Poirier
2021-02-11 19:00     ` Mathieu Poirier
2021-02-12  3:31     ` Anshuman Khandual
2021-02-12  3:31       ` Anshuman Khandual
2021-02-12 16:57       ` Mathieu Poirier
2021-02-12 16:57         ` Mathieu Poirier
2021-02-15  9:26         ` Anshuman Khandual
2021-02-15  9:26           ` Anshuman Khandual
2021-02-12 20:26   ` Mathieu Poirier
2021-02-12 20:26     ` Mathieu Poirier
2021-02-15  9:46     ` Anshuman Khandual
2021-02-15  9:46       ` Anshuman Khandual
2021-02-16  9:00       ` Mike Leach
2021-02-16  9:00         ` Mike Leach
2021-02-16  9:44         ` Anshuman Khandual
2021-02-16  9:44           ` Anshuman Khandual
2021-02-16 12:12           ` Mike Leach
2021-02-16 12:12             ` Mike Leach
2021-02-18  7:50         ` Suzuki K Poulose
2021-02-18  7:50           ` Suzuki K Poulose
2021-02-18 14:30           ` Mike Leach
2021-02-18 14:30             ` Mike Leach
2021-02-18 15:14             ` Suzuki K Poulose
2021-02-18 15:14               ` Suzuki K Poulose
2021-02-22 10:42               ` Mike Leach
2021-02-22 10:42                 ` Mike Leach
2021-01-27  8:55 ` [PATCH V3 12/14] dts: bindings: Document device tree bindings for Arm TRBE Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-02-09 19:04   ` Rob Herring
2021-02-09 19:04     ` Rob Herring
2021-01-27  8:55 ` [PATCH V3 13/14] perf: aux: Add flags for the buffer format Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-01-27 12:51   ` Peter Zijlstra
2021-01-27 12:51     ` Peter Zijlstra
2021-02-16 10:59   ` Mike Leach
2021-02-16 10:59     ` Mike Leach
2021-01-27  8:55 ` [PATCH V3 14/14] coresight: etm-perf: Add support for trace " Anshuman Khandual
2021-01-27  8:55   ` Anshuman Khandual
2021-01-27 12:54   ` Peter Zijlstra
2021-01-27 12:54     ` Peter Zijlstra
2021-01-27 13:00     ` Al Grant
2021-01-27 13:00       ` Al Grant
2021-02-18  3:05       ` Anshuman Khandual
2021-02-18  3:05         ` Anshuman Khandual
2021-01-27 14:12     ` Suzuki K Poulose
2021-01-27 14:12       ` Suzuki K Poulose
2021-02-16 11:01   ` Mike Leach [this message]
2021-02-16 11:01     ` Mike Leach
2021-01-27 18:50 ` [PATCH V3 00/14] arm64: coresight: Enable ETE and TRBE Mathieu Poirier
2021-01-27 18:50   ` Mathieu Poirier
2021-02-01 18:44 ` Mathieu Poirier
2021-02-01 18:44   ` Mathieu Poirier
2021-02-18  4:23   ` Anshuman Khandual
2021-02-18  4:23     ` Anshuman Khandual

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJ9a7Vi-mJ_sccLFyDQrUo8opEfGnZG-s0Atw=c0Ukt4kVX5xA@mail.gmail.com' \
    --to=mike.leach@linaro.org \
    --cc=anshuman.khandual@arm.com \
    --cc=coresight@lists.linaro.org \
    --cc=lcherian@marvell.com \
    --cc=leo.yan@linaro.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mathieu.poirier@linaro.org \
    --cc=peterz@infradead.org \
    --cc=suzuki.poulose@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.