All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ram Pai <linuxram@us.ibm.com>
To: David Gibson <david@gibson.dropbear.id.au>
Cc: Christoph Hellwig <hch@lst.de>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Thiago Jung Bauermann <bauerman@linux.ibm.com>,
	virtualization@lists.linux-foundation.org,
	linuxppc-devel@lists.ozlabs.org,
	iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org,
	Jason Wang <jasowang@redhat.com>,
	Alexey Kardashevskiy <aik@linux.ibm.com>,
	Paul Mackerras <paulus@ozlabs.org>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>
Subject: RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
Date: Tue, 13 Aug 2019 08:45:37 -0700	[thread overview]
Message-ID: <20190813154537.GE5964@ram.ibm.com> (raw)
In-Reply-To: <20190813142439.GO3947@umbus.fritz.box>

On Wed, Aug 14, 2019 at 12:24:39AM +1000, David Gibson wrote:
> On Tue, Aug 13, 2019 at 03:26:17PM +0200, Christoph Hellwig wrote:
> > On Mon, Aug 12, 2019 at 07:51:56PM +1000, David Gibson wrote:
> > > AFAICT we already kind of abuse this for the VIRTIO_F_IOMMU_PLATFORM,
> > > because to handle for cases where it *is* a device limitation, we
> > > assume that if the hypervisor presents VIRTIO_F_IOMMU_PLATFORM then
> > > the guest *must* select it.
> > > 
> > > What we actually need here is for the hypervisor to present
> > > VIRTIO_F_IOMMU_PLATFORM as available, but not required.  Then we need
> > > a way for the platform core code to communicate to the virtio driver
> > > that *it* requires the IOMMU to be used, so that the driver can select
> > > or not the feature bit on that basis.
> > 
> > I agree with the above, but that just brings us back to the original
> > issue - the whole bypass of the DMA OPS should be an option that the
> > device can offer, not the other way around.  And we really need to
> > fix that root cause instead of doctoring around it.
> 
> I'm not exactly sure what you mean by "device" in this context.  Do
> you mean the hypervisor (qemu) side implementation?
> 
> You're right that this was the wrong way around to begin with, but as
> well as being hard to change now, I don't see how it really addresses
> the current problem.  The device could default to IOMMU and allow
> bypass, but the driver would still need to get information from the
> platform to know that it *can't* accept that option in the case of a
> secure VM.  Reversed sense, but the same basic problem.
> 
> The hypervisor does not, and can not be aware of the secure VM
> restrictions - only the guest side platform code knows that.

This statement is almost entirely right. I will rephrase it to make it
entirely right.   

The hypervisor does not, and can not be aware of the secure VM
requirement that it needs to do some special processing that has nothing
to do with DMA address translation - only the guest side platform code
know that.

BTW: I do not consider 'bounce buffering' as 'DMA address translation'.
DMA address translation, translates CPU address to DMA address.  Bounce
buffering moves the data from one buffer at a given CPU address to
another buffer at a different CPU address.  Unfortunately the current
DMA ops conflates the two.  The need to do 'DMA address translation' 
is something the device can enforce.  But the need to do bounce
buffering, is something that the device should not be aware and should be
entirely a decision made locally by the kernel/driver in the secure VM.

RP

> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson



-- 
Ram Pai


WARNING: multiple messages have this Message-ID (diff)
From: Ram Pai <linuxram@us.ibm.com>
To: David Gibson <david@gibson.dropbear.id.au>
Cc: "Michael S. Tsirkin" <mst@redhat.com>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Jason Wang <jasowang@redhat.com>,
	Alexey Kardashevskiy <aik@linux.ibm.com>,
	linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	Paul Mackerras <paulus@ozlabs.org>,
	iommu@lists.linux-foundation.org,
	linuxppc-devel@lists.ozlabs.org, Christoph Hellwig <hch@lst.de>
Subject: RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
Date: Tue, 13 Aug 2019 08:45:37 -0700	[thread overview]
Message-ID: <20190813154537.GE5964@ram.ibm.com> (raw)
In-Reply-To: <20190813142439.GO3947@umbus.fritz.box>

On Wed, Aug 14, 2019 at 12:24:39AM +1000, David Gibson wrote:
> On Tue, Aug 13, 2019 at 03:26:17PM +0200, Christoph Hellwig wrote:
> > On Mon, Aug 12, 2019 at 07:51:56PM +1000, David Gibson wrote:
> > > AFAICT we already kind of abuse this for the VIRTIO_F_IOMMU_PLATFORM,
> > > because to handle for cases where it *is* a device limitation, we
> > > assume that if the hypervisor presents VIRTIO_F_IOMMU_PLATFORM then
> > > the guest *must* select it.
> > > 
> > > What we actually need here is for the hypervisor to present
> > > VIRTIO_F_IOMMU_PLATFORM as available, but not required.  Then we need
> > > a way for the platform core code to communicate to the virtio driver
> > > that *it* requires the IOMMU to be used, so that the driver can select
> > > or not the feature bit on that basis.
> > 
> > I agree with the above, but that just brings us back to the original
> > issue - the whole bypass of the DMA OPS should be an option that the
> > device can offer, not the other way around.  And we really need to
> > fix that root cause instead of doctoring around it.
> 
> I'm not exactly sure what you mean by "device" in this context.  Do
> you mean the hypervisor (qemu) side implementation?
> 
> You're right that this was the wrong way around to begin with, but as
> well as being hard to change now, I don't see how it really addresses
> the current problem.  The device could default to IOMMU and allow
> bypass, but the driver would still need to get information from the
> platform to know that it *can't* accept that option in the case of a
> secure VM.  Reversed sense, but the same basic problem.
> 
> The hypervisor does not, and can not be aware of the secure VM
> restrictions - only the guest side platform code knows that.

This statement is almost entirely right. I will rephrase it to make it
entirely right.   

The hypervisor does not, and can not be aware of the secure VM
requirement that it needs to do some special processing that has nothing
to do with DMA address translation - only the guest side platform code
know that.

BTW: I do not consider 'bounce buffering' as 'DMA address translation'.
DMA address translation, translates CPU address to DMA address.  Bounce
buffering moves the data from one buffer at a given CPU address to
another buffer at a different CPU address.  Unfortunately the current
DMA ops conflates the two.  The need to do 'DMA address translation' 
is something the device can enforce.  But the need to do bounce
buffering, is something that the device should not be aware and should be
entirely a decision made locally by the kernel/driver in the secure VM.

RP

> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson



-- 
Ram Pai

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

WARNING: multiple messages have this Message-ID (diff)
From: Ram Pai <linuxram@us.ibm.com>
To: David Gibson <david@gibson.dropbear.id.au>
Cc: "Michael S. Tsirkin" <mst@redhat.com>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Alexey Kardashevskiy <aik@linux.ibm.com>,
	linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	Paul Mackerras <paulus@ozlabs.org>,
	iommu@lists.linux-foundation.org,
	linuxppc-devel@lists.ozlabs.org, Christoph Hellwig <hch@lst.de>
Subject: RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
Date: Tue, 13 Aug 2019 08:45:37 -0700	[thread overview]
Message-ID: <20190813154537.GE5964@ram.ibm.com> (raw)
In-Reply-To: <20190813142439.GO3947@umbus.fritz.box>

On Wed, Aug 14, 2019 at 12:24:39AM +1000, David Gibson wrote:
> On Tue, Aug 13, 2019 at 03:26:17PM +0200, Christoph Hellwig wrote:
> > On Mon, Aug 12, 2019 at 07:51:56PM +1000, David Gibson wrote:
> > > AFAICT we already kind of abuse this for the VIRTIO_F_IOMMU_PLATFORM,
> > > because to handle for cases where it *is* a device limitation, we
> > > assume that if the hypervisor presents VIRTIO_F_IOMMU_PLATFORM then
> > > the guest *must* select it.
> > > 
> > > What we actually need here is for the hypervisor to present
> > > VIRTIO_F_IOMMU_PLATFORM as available, but not required.  Then we need
> > > a way for the platform core code to communicate to the virtio driver
> > > that *it* requires the IOMMU to be used, so that the driver can select
> > > or not the feature bit on that basis.
> > 
> > I agree with the above, but that just brings us back to the original
> > issue - the whole bypass of the DMA OPS should be an option that the
> > device can offer, not the other way around.  And we really need to
> > fix that root cause instead of doctoring around it.
> 
> I'm not exactly sure what you mean by "device" in this context.  Do
> you mean the hypervisor (qemu) side implementation?
> 
> You're right that this was the wrong way around to begin with, but as
> well as being hard to change now, I don't see how it really addresses
> the current problem.  The device could default to IOMMU and allow
> bypass, but the driver would still need to get information from the
> platform to know that it *can't* accept that option in the case of a
> secure VM.  Reversed sense, but the same basic problem.
> 
> The hypervisor does not, and can not be aware of the secure VM
> restrictions - only the guest side platform code knows that.

This statement is almost entirely right. I will rephrase it to make it
entirely right.   

The hypervisor does not, and can not be aware of the secure VM
requirement that it needs to do some special processing that has nothing
to do with DMA address translation - only the guest side platform code
know that.

BTW: I do not consider 'bounce buffering' as 'DMA address translation'.
DMA address translation, translates CPU address to DMA address.  Bounce
buffering moves the data from one buffer at a given CPU address to
another buffer at a different CPU address.  Unfortunately the current
DMA ops conflates the two.  The need to do 'DMA address translation' 
is something the device can enforce.  But the need to do bounce
buffering, is something that the device should not be aware and should be
entirely a decision made locally by the kernel/driver in the secure VM.

RP

> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson



-- 
Ram Pai

  reply	other threads:[~2019-08-13 15:45 UTC|newest]

Thread overview: 198+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-29 17:08 [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted Thiago Jung Bauermann
2019-01-29 17:08 ` Thiago Jung Bauermann
2019-01-29 17:42 ` Thiago Jung Bauermann
2019-01-29 17:42   ` Thiago Jung Bauermann
2019-01-29 19:02   ` Michael S. Tsirkin
2019-01-29 19:02     ` Michael S. Tsirkin
2019-01-30  2:24     ` Jason Wang
2019-01-30  2:24       ` Jason Wang
2019-01-30  2:36       ` Michael S. Tsirkin
2019-01-30  2:36       ` Michael S. Tsirkin
2019-01-30  2:36         ` Michael S. Tsirkin
2019-01-30  3:05         ` Jason Wang
2019-01-30  3:05         ` Jason Wang
2019-01-30  3:05           ` Jason Wang
2019-01-30  3:26           ` Michael S. Tsirkin
2019-01-30  3:26           ` Michael S. Tsirkin
2019-01-30  3:26             ` Michael S. Tsirkin
2019-01-30  7:44         ` Christoph Hellwig
2019-01-30  7:44         ` Christoph Hellwig
2019-01-30  7:44           ` Christoph Hellwig
2019-02-04 18:15           ` Thiago Jung Bauermann
2019-02-04 18:15           ` Thiago Jung Bauermann
2019-02-04 18:15             ` Thiago Jung Bauermann
2019-02-04 21:38             ` Michael S. Tsirkin
2019-02-04 21:38               ` Michael S. Tsirkin
2019-02-05  7:24               ` Christoph Hellwig
2019-02-05  7:24                 ` Christoph Hellwig
2019-02-05 16:13                 ` Michael S. Tsirkin
2019-02-05 16:13                 ` Michael S. Tsirkin
2019-02-05 16:13                   ` Michael S. Tsirkin
2019-02-05 16:13                   ` Michael S. Tsirkin
2019-02-05  7:24               ` Christoph Hellwig
2019-02-04 21:38             ` Michael S. Tsirkin
2019-03-26 16:53           ` Michael S. Tsirkin
2019-03-26 16:53           ` Michael S. Tsirkin
2019-03-26 16:53             ` Michael S. Tsirkin
2019-01-30  2:24     ` Jason Wang
2019-02-04 18:14     ` Thiago Jung Bauermann
2019-02-04 18:14     ` Thiago Jung Bauermann
2019-02-04 18:14       ` Thiago Jung Bauermann
2019-02-04 20:23       ` Michael S. Tsirkin
2019-02-04 20:23         ` Michael S. Tsirkin
2019-03-20 16:13         ` Thiago Jung Bauermann
2019-03-20 16:13           ` Thiago Jung Bauermann
2019-03-20 16:13           ` Thiago Jung Bauermann
2019-03-20 21:17           ` Michael S. Tsirkin
2019-03-20 21:17           ` Michael S. Tsirkin
2019-03-20 21:17             ` Michael S. Tsirkin
2019-03-22  0:05             ` Thiago Jung Bauermann
2019-03-22  0:05             ` Thiago Jung Bauermann
2019-03-22  0:05               ` Thiago Jung Bauermann
2019-03-23 21:01               ` Michael S. Tsirkin
2019-03-23 21:01                 ` Michael S. Tsirkin
2019-03-23 21:01                 ` Michael S. Tsirkin
2019-03-25  0:57                 ` David Gibson
2019-03-25  0:57                   ` David Gibson
2019-03-25  0:57                   ` David Gibson
2019-04-17 21:42                   ` Thiago Jung Bauermann
2019-04-17 21:42                     ` Thiago Jung Bauermann
2019-04-17 21:42                     ` Thiago Jung Bauermann
2019-04-17 21:42                     ` Thiago Jung Bauermann
2019-04-17 21:42                 ` Thiago Jung Bauermann
2019-04-17 21:42                 ` Thiago Jung Bauermann
2019-04-17 21:42                   ` Thiago Jung Bauermann
2019-04-17 21:42                   ` Thiago Jung Bauermann
2019-04-17 21:42                   ` Thiago Jung Bauermann
2019-04-19 23:09                   ` Michael S. Tsirkin
2019-04-19 23:09                   ` Michael S. Tsirkin
2019-04-19 23:09                     ` Michael S. Tsirkin
2019-04-19 23:09                     ` Michael S. Tsirkin
2019-04-25  1:01                     ` Thiago Jung Bauermann
2019-04-25  1:01                       ` Thiago Jung Bauermann
2019-04-25  1:01                       ` Thiago Jung Bauermann
2019-04-25  1:18                       ` Michael S. Tsirkin
2019-04-25  1:18                       ` Michael S. Tsirkin
2019-04-25  1:18                         ` Michael S. Tsirkin
2019-04-25  1:18                         ` Michael S. Tsirkin
2019-04-25  1:18                         ` Michael S. Tsirkin
2019-04-26 23:56                         ` Thiago Jung Bauermann
2019-04-26 23:56                         ` Thiago Jung Bauermann
2019-04-26 23:56                           ` Thiago Jung Bauermann
2019-04-26 23:56                           ` Thiago Jung Bauermann
2019-05-20 13:08                           ` Michael S. Tsirkin
2019-05-20 13:08                             ` Michael S. Tsirkin
2019-05-20 13:08                             ` Michael S. Tsirkin
2019-05-20 13:08                             ` Michael S. Tsirkin
2019-04-25  1:01                     ` Thiago Jung Bauermann
2019-05-20 13:16                   ` Michael S. Tsirkin
2019-05-20 13:16                   ` Michael S. Tsirkin
2019-05-20 13:16                     ` Michael S. Tsirkin
2019-05-20 13:16                     ` Michael S. Tsirkin
2019-06-04  1:13                     ` Thiago Jung Bauermann
2019-06-04  1:13                       ` Thiago Jung Bauermann
2019-06-04  1:13                       ` Thiago Jung Bauermann
2019-06-04  1:42                       ` Michael S. Tsirkin
2019-06-04  1:42                         ` Michael S. Tsirkin
2019-06-04  1:42                         ` Michael S. Tsirkin
2019-06-04  1:42                         ` Michael S. Tsirkin
2019-06-28  1:58                         ` Thiago Jung Bauermann
2019-06-28  1:58                         ` Thiago Jung Bauermann
2019-06-28  1:58                           ` Thiago Jung Bauermann
2019-06-28  1:58                           ` Thiago Jung Bauermann
2019-07-01 14:17                           ` Michael S. Tsirkin
2019-07-01 14:17                           ` Michael S. Tsirkin
2019-07-01 14:17                             ` Michael S. Tsirkin
2019-07-01 14:17                             ` Michael S. Tsirkin
2019-07-14  5:51                             ` Thiago Jung Bauermann
2019-07-14  5:51                             ` Thiago Jung Bauermann
2019-07-14  5:51                               ` Thiago Jung Bauermann
2019-07-14  5:51                               ` Thiago Jung Bauermann
2019-07-15 14:35                               ` Michael S. Tsirkin
2019-07-15 14:35                               ` Michael S. Tsirkin
2019-07-15 14:35                                 ` Michael S. Tsirkin
2019-07-15 14:35                                 ` Michael S. Tsirkin
2019-07-15 20:29                                 ` Thiago Jung Bauermann
2019-07-15 20:29                                   ` Thiago Jung Bauermann
2019-07-15 20:29                                   ` Thiago Jung Bauermann
2019-07-15 20:36                                   ` Michael S. Tsirkin
2019-07-15 20:36                                   ` Michael S. Tsirkin
2019-07-15 20:36                                     ` Michael S. Tsirkin
2019-07-15 20:36                                     ` Michael S. Tsirkin
2019-07-15 22:03                                     ` Thiago Jung Bauermann
2019-07-15 22:03                                       ` Thiago Jung Bauermann
2019-07-15 22:03                                       ` Thiago Jung Bauermann
2019-07-15 22:03                                       ` Thiago Jung Bauermann
2019-07-15 22:16                                       ` Michael S. Tsirkin
2019-07-15 22:16                                       ` Michael S. Tsirkin
2019-07-15 22:16                                         ` Michael S. Tsirkin
2019-07-15 22:16                                         ` Michael S. Tsirkin
2019-07-15 23:05                                         ` Thiago Jung Bauermann
2019-07-15 23:05                                           ` Thiago Jung Bauermann
2019-07-15 23:05                                           ` Thiago Jung Bauermann
2019-07-15 23:05                                         ` Thiago Jung Bauermann
2019-07-15 23:24                                       ` Benjamin Herrenschmidt
2019-07-15 23:24                                       ` Benjamin Herrenschmidt
2019-07-15 23:24                                         ` Benjamin Herrenschmidt
2019-07-15 23:24                                         ` Benjamin Herrenschmidt
2019-07-15 20:29                                 ` Thiago Jung Bauermann
2019-07-18  3:39                               ` Thiago Jung Bauermann
2019-07-18  3:39                                 ` Thiago Jung Bauermann
2019-07-18  3:39                               ` Thiago Jung Bauermann
2019-06-04  1:13                     ` Thiago Jung Bauermann
2019-03-20 16:13         ` Thiago Jung Bauermann
2019-02-04 20:23       ` Michael S. Tsirkin
2019-01-29 19:02   ` Michael S. Tsirkin
2019-01-29 17:42 ` Thiago Jung Bauermann
2019-08-10 18:57 ` Michael S. Tsirkin
2019-08-10 18:57   ` Michael S. Tsirkin
2019-08-10 18:57   ` Michael S. Tsirkin
2019-08-10 22:07   ` Ram Pai
2019-08-10 22:07   ` Ram Pai
2019-08-10 22:07     ` Ram Pai
2019-08-11  5:56     ` Christoph Hellwig
2019-08-11  5:56       ` Christoph Hellwig
2019-08-11  5:56       ` Christoph Hellwig
2019-08-11  6:46       ` Ram Pai
2019-08-11  6:46         ` Ram Pai
2019-08-11  6:46         ` Ram Pai
2019-08-11  8:44         ` Michael S. Tsirkin
2019-08-11  8:44           ` Michael S. Tsirkin
2019-08-11  8:44           ` Michael S. Tsirkin
2019-08-12 12:13         ` Christoph Hellwig
2019-08-12 12:13         ` Christoph Hellwig
2019-08-12 12:13           ` Christoph Hellwig
2019-08-12 20:29           ` Ram Pai
2019-08-12 20:29             ` Ram Pai
2019-08-12 20:29           ` Ram Pai
2019-08-11  8:42       ` Michael S. Tsirkin
2019-08-11  8:42         ` Michael S. Tsirkin
2019-08-11  8:42       ` Michael S. Tsirkin
2019-08-11  8:55       ` Michael S. Tsirkin
2019-08-11  8:55       ` Michael S. Tsirkin
2019-08-11  8:55         ` Michael S. Tsirkin
2019-08-12 12:15         ` Christoph Hellwig
2019-08-12 12:15         ` Christoph Hellwig
2019-08-12 12:15           ` Christoph Hellwig
2019-09-06  5:07           ` Michael S. Tsirkin
2019-09-06  5:07             ` Michael S. Tsirkin
2019-09-06  5:07             ` Michael S. Tsirkin
2019-08-12  9:51       ` David Gibson
2019-08-12  9:51       ` David Gibson
2019-08-12  9:51         ` David Gibson
2019-08-13 13:26         ` Christoph Hellwig
2019-08-13 13:26           ` Christoph Hellwig
2019-08-13 14:24           ` David Gibson
2019-08-13 14:24             ` David Gibson
2019-08-13 15:45             ` Ram Pai [this message]
2019-08-13 15:45               ` Ram Pai
2019-08-13 15:45               ` Ram Pai
2019-08-26 17:48               ` Ram Pai
2019-08-26 17:48                 ` Ram Pai
2019-08-26 17:48               ` Ram Pai
2019-08-13 14:24           ` David Gibson
2019-08-13 13:26         ` Christoph Hellwig
2019-08-11  8:12     ` Michael S. Tsirkin
2019-08-11  8:12       ` Michael S. Tsirkin
2019-08-11  8:12       ` Michael S. Tsirkin
  -- strict thread matches above, loose matches on Subject: below --
2019-01-29 17:08 Thiago Jung Bauermann

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190813154537.GE5964@ram.ibm.com \
    --to=linuxram@us.ibm.com \
    --cc=aik@linux.ibm.com \
    --cc=bauerman@linux.ibm.com \
    --cc=benh@kernel.crashing.org \
    --cc=david@gibson.dropbear.id.au \
    --cc=hch@lst.de \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jasowang@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxppc-devel@lists.ozlabs.org \
    --cc=mst@redhat.com \
    --cc=paulus@ozlabs.org \
    --cc=virtualization@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.